Defect Detection Method and Device for Inner Glass Substrate of Display Panel

Abstract

The present application provides a method and device for defect detection of the internal glass substrate of a display panel, which are used for the defect detection accuracy of the glass substrate. The method mainly includes: obtaining multi-angle polarized light field data through a high-speed polarization camera; generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data; the three-dimensional refractive index anomaly map includes the refractive index perturbation values corresponding to each voxel; determining the abnormal position coordinates through the refractive index perturbation values in the three-dimensional refractive index anomaly map, and intercepting an abnormal region image from the two-dimensional spectral image obtained through a hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes the spectral data of each position point; mapping the spectral data in the abnormal region image to the three-dimensional refractive index anomaly map through coordinate transformation; determining the defect type and defect cause of the glass substrate according to the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data.

Description

Technical Field

[0001] This application relates to the field of optical detection technology, and in particular, to a method and device for detecting defects in a glass substrate inside a display panel. Background Art

[0002] With the development of society, electronic display devices are being updated and replaced more and more quickly. The types and sizes of display devices are also increasing, and the requirements for display devices are getting higher and higher. As the main raw material of liquid crystal displays, glass substrates are extremely prone to defects during the production process, and their product quality and utilization rate are directly related to the imaging effect of liquid crystal displays.

[0003] Traditional methods for detecting defects in glass substrates usually rely on single optical detection means, such as brightness detection, contrast detection, or simple surface defect detection. These methods have limitations in detecting complex defects (such as internal bubbles, impurities, stress non-uniformity, etc.), resulting in insufficient detection accuracy. Summary of the Invention

[0004] Embodiments of this application provide a method and device for detecting defects in a glass substrate inside a display panel, which are used to improve the accuracy of defect detection of glass substrates.

[0005] Embodiments of the present invention provide a method for detecting defects in a glass substrate inside a display panel. The method is applied to a glass substrate defect detection system, and the system includes: a refraction image module, a spectral image module, a collaborative control module, and a glass substrate defect detection device; the refraction image module includes a multi-angle polarized light source array, a high-speed polarized camera, and a white light interferometer; the spectral image module includes a hyperspectral imaging camera and an LED uniform light source; the collaborative control module includes a six-axis motion platform and a six-axis robotic arm. A vacuum chuck for adsorbing the glass substrate is provided on the six-axis motion platform; the multi-angle polarized light source array and the hyperspectral imaging camera are provided on the six-axis robotic arm, and the high-speed polarized camera and the LED uniform light source are provided directly above the glass substrate; the glass substrate defect detection device is connected to the refraction image module, the spectral image module, and the collaborative control module, and the glass substrate defect detection device is used to determine the defect type and defect cause of the glass substrate through the image data obtained by the refraction image module and the spectral image module.

[0006] Embodiments of the present invention provide a glass substrate defect detection device. The device is applied to execute the method for detecting defects in a glass substrate inside a display panel according to any one of claims 1 to 7. The device includes:

[0007] An acquisition module, configured to set the light source parameters of the multi-angle polarization light source array, so as to acquire multi-angle polarization light field data I(θ, λ, P, x, y) through the high-speed polarization camera; where θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) are the position coordinates;

[0008] A generation module, configured to generate a three-dimensional refractive index anomaly map according to the multi-angle polarization light field data I(θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes the refractive index perturbation values corresponding to each voxel;

[0009] A determination module, configured to determine the abnormal position coordinates determined by the refractive index perturbation values in the three-dimensional refractive index anomaly map, and intercept the abnormal region image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes the spectral data of each position point;

[0010] A mapping module, configured to map the spectral data in the abnormal region image to the three-dimensional refractive index anomaly map through coordinate transformation;

[0011] A prediction module, configured to determine the defect type and defect cause of the glass substrate according to the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data.

[0012] A computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the above-mentioned defect detection method for the glass substrate inside the display panel.

[0013] The present invention provides a method and device for defect detection of a glass substrate inside a display panel. First, the light source parameters of the multi-angle polarization light source array are set to obtain multi-angle polarization light field data I(θ, λ, P, x, y) through the high-speed polarization camera; then, a three-dimensional refractive index anomaly map is generated based on the multi-angle polarization light field data I(θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes the refractive index perturbation values corresponding to each voxel; the abnormal position coordinates are determined by the refractive index perturbation values in the three-dimensional refractive index anomaly map, and an abnormal region image is intercepted from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes the spectral data of each position point; the spectral data in the abnormal region image is mapped into the three-dimensional refractive index anomaly map through coordinate transformation; the defect type and defect cause of the glass substrate are determined according to the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map with the spectral data mapped. Compared with the prior art that relies on a single optical detection means to detect the glass substrate, the present application combines two technologies of polarization imaging and hyperspectral imaging, realizes the fusion analysis of multi-modal data, and significantly improves the accuracy of glass substrate defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a structural diagram of a defect detection system for a glass substrate inside a display panel provided by the present application;

[0015] Figure 2 It is a flowchart of a method for defect detection of a glass substrate inside a display panel provided by the present application;

[0016] Figure 3 It is a schematic structural diagram of a device for defect detection of a glass substrate provided by the present application;

[0017] Figure 4 It is a schematic diagram of a computer device provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to better understand the above technical solutions, the technical solutions of the embodiments of the present application will be described in detail below through the drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present application and the embodiments are detailed descriptions of the technical solutions of the embodiments of the present application, rather than limitations on the technical solutions of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0019] Please refer to Figure 1, a defect detection system for a glass substrate inside a display panel provided by an embodiment of the present invention. The system includes: a refraction image module, a spectral image module, a collaborative control module, and a glass substrate defect detection device; the refraction image module includes a multi-angle polarized light source array, a high-speed polarized camera, and a white light interferometer; the spectral image module includes a hyperspectral imaging camera and an LED uniform light source; the collaborative control module includes a six-axis motion platform and a six-axis robotic arm, and a vacuum chuck for adsorbing the glass substrate is arranged on the six-axis motion platform; the multi-angle polarized light source array and the hyperspectral imaging camera are arranged on the six-axis robotic arm, and the high-speed polarized camera and the LED uniform light source are arranged directly above the glass substrate; the glass substrate defect detection device is connected to the refraction image module, the spectral image module, and the collaborative control module. Among them, the parameters of the multi-angle polarized light source array in this embodiment can be 6 groups of lasers with a wavelength of 405 - 1550 nm; the parameters of the high-speed polarized camera can be 4096×2160 pixels and 1000 fps; a white light interferometer (Zygo Verifire); a hyperspectral imaging camera (HySpex VS-6200, 400 - 2500 nm); an LED uniform light source (polarization controllable); a six-axis motion platform (XYZ translation + tilt).

[0020] The refraction image module is located in the upper half of the housing, arranged in parallel with the spectral image module, and fixed by an isolation bracket to reduce optical interference. Multi-angle polarized light source array: Horizontally arranged along the top of the housing, the light source incident angle covers 0° - 90°, and is adjusted by the six-axis robotic arm. High-speed polarized camera: Facing the detection area of the glass substrate, forming a 45° angle with the light source array, and fine-tuning the focus through a guide rail. Interferometer calibration unit: Integrated below the camera, real-time monitoring of the substrate surface deformation, and data feedback to the collaborative control module. The spectral image module is located in the lower half of the housing, sharing the same optical path channel with the refraction image module, and realizing synchronous acquisition of multi-modal data through a beam splitter. The hyperspectral imaging camera is arranged on the six-axis robotic arm, supporting XYZ-axis translation to adapt to different sizes of substrates. The LED uniform light source is installed around the camera lens, and uniformly illuminates through a diffuser plate to eliminate specular reflection noise. The collaborative control module is centrally arranged, connecting the refraction image module and the spectral image module through a backplane. A vacuum chuck for adsorbing the glass substrate is arranged on the six-axis motion platform, driven by a stepping motor to realize uniform pushing and scanning of the substrate at a speed of 0.1 m / s.

[0021] Please refer to Figure 2 , a defect detection method for a glass substrate inside a display panel provided by an embodiment of the present invention. This method is applied to the glass substrate defect detection device in the above-mentioned defect detection system for the glass substrate inside the display panel. The method specifically includes S101 - S105:

[0022] S101. Set the light source parameters of the multi - angle polarized light source array to obtain multi - angle polarized light field data I(θ, λ, P, x, y) through the high - speed polarization camera.

[0023] In this embodiment, the multi - angle polarized light source array can adopt 6 groups of tunable semiconductor lasers (wavelengths 405nm, 532nm, 635nm, 780nm, 1064nm, 1550nm), covering the visible light to near - infrared band. Each group of light sources integrates a liquid - crystal polarization controller, supporting dynamic switching between 0° - 180° linear polarization and circular polarization. The multi - angle polarized light source array can support continuous adjustment of the incident angle from 0° to 90°, covering the omnidirectional illumination requirements. The high - speed polarization camera is equipped with a quantum - dot - enhanced CMOS sensor and integrates a four - way polarization filter (0°, 45°, 90°, 135°), with a frame rate of 1000fps. The white - light interferometer monitors the surface deformation and refractive index fluctuation of the glass substrate in real time.

[0024] Specifically, in this embodiment, each combination of preset incident angles (such as incident angles of 0°, 30°, 60°) and polarization states (0° linear polarization, right - hand circular polarization) is scanned to collect the intensity and polarization state distribution of the reflected / transmitted light field, obtaining multi - angle polarized light field data I(θ, λ, P, x, y), where θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) is the position coordinate.

[0025] S102. Generate a three - dimensional refractive index anomaly map based on the multi - angle polarized light field data I(θ, λ, P, x, y); the three - dimensional refractive index anomaly map includes the refractive index perturbation values corresponding to each voxel.

[0026] In an optional embodiment provided by the present application, generating a three - dimensional refractive index anomaly map based on the multi - angle polarized light field data I(θ, λ, P, x, y) includes:

[0027] S1021. Obtain the surface height distribution of the glass substrate through the white - light interferometer, and calculate the theoretical optical path according to the surface height distribution; calculate the actual optical path according to the multi - angle polarized light field data I(θ, λ, P, x, y) and the reference light intensity of the defect - free glass substrate.

[0028] In this embodiment, first, the surface height distribution h(x, y) of the substrate is obtained through the white - light interferometer, and then combined with the material refractive index n(λ) calculate the theoretical optical path and calculate the measured optical path where is the defect - free reference value, and then a three - dimensional refractive index anomaly map is generated according to the theoretical optical path and the measured optical path.

[0029] It should be noted that the change in the refractive index of the material due to temperature change (for example, when ΔT = 1 °C, the refractive index of soda-lime glass changes ) will affect the calculation of the theoretical optical path. To solve this problem, in this embodiment, the calculation of the theoretical optical path according to the surface height distribution includes: calculating the actual refractive index corresponding to the current temperature according to the current temperature, the reference temperature, the thermo-optic coefficient of the glass substrate, and the standard refractive index corresponding to the reference temperature; calculating the theoretical optical path according to the actual refractive index corresponding to the current temperature and the surface height distribution.

[0030] Specifically, in this embodiment, the actual refractive index corresponding to the current temperature can be calculated by the formula . Where n(Tλ) is the actual refractive index of the material λ at temperature T, is the nominal refractive index of the material at the reference temperature (for example, ≈1.52 for soda-lime glass, ≈1.46 for fused silica glass), β is the thermo-optic coefficient of the material, which is used to represent the change in refractive index per 1 °C increase in temperature; T is the current ambient temperature, is the reference temperature (the reference temperature during calibration). Then, the actual refractive index corresponding to the current temperature is calculated according to the formula .

[0031] Furthermore, in order to adapt to a wide temperature range (such as -50 °C to 150 °C) or high-precision scenarios in this embodiment, a higher-order term correction needs to be introduced, that is, the actual refractive index corresponding to the current temperature is calculated by the formula , where γ is the second-order thermo-optic coefficient.

[0032] S1022. Obtain the three-dimensional refractive index anomaly map according to the theoretical optical path and the actual optical path.

[0033] In this embodiment, an iterative algebraic reconstruction algorithm can be used to solve the three-dimensional refractive index anomaly map Δ n ( x , y , z )

[0034] ,

[0035] where is the change in the optical path difference, Δ n ( x , y , z) represents the refractive index perturbation value at a certain point (x, y, z) in the glass substrate. z is the depth coordinate along the light propagation direction, representing the current calculated depth position, which is used to locate the position of the defect in three-dimensional space, and is the total sum of the refractive index perturbations accumulated from z0 to z1 along the light propagation direction (z-axis).

[0036] S103. Determine the abnormal position coordinates determined by the refractive index perturbation value in the three-dimensional refractive index anomaly map, and intercept the abnormal region image from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates.

[0037] Among them, the two-dimensional spectral image includes the spectral data of each position point. Specifically, in this embodiment, the refractive index perturbation value can be compared with a predetermined threshold to determine the abnormal position coordinates. If a certain region (the typical refractive index fluctuation threshold of the glass) is determined as a physical defect.

[0038] In an optional embodiment provided by the present application, before intercepting the abnormal region image from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates, the method further includes: setting the band range of the hyperspectral imaging camera and the shooting parameters of the LED uniform light source, and adjusting the angle of the six-axis robotic arm to enable the hyperspectral imaging camera to collect two-dimensional spectral images at multiple angles; performing data preprocessing on the two-dimensional spectral images collected by the hyperspectral imaging camera, and performing feature extraction on the two-dimensional spectral images after data preprocessing to obtain spectral feature vectors; correspondingly, intercepting the abnormal region image from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates includes: intercepting the abnormal region image from the spectral feature vectors according to the abnormal position coordinates.

[0039] Specifically, control the glass substrate to pass through the field of view at a constant speed, and the hyperspectral imaging camera scans row by row to generate two-dimensional spectral image data. Data is collected at multi-angle incident angles of 15°, 45°, and 75° respectively. Among them, data preprocessing can include dark current correction, normalization, spectral smoothing and denoising, etc., and this embodiment does not make specific limitations on this.

[0040] S104. Map the spectral data in the abnormal region image to the three-dimensional refractive index anomaly map through coordinate transformation.

[0041] It should be noted that before obtaining the multi-angle polarized light field data and two-dimensional spectral images through the high-speed polarization camera and the hyperspectral imaging camera in this embodiment, it is also necessary to align the fields of view of the high-speed polarization camera and the hyperspectral imaging camera using a calibration plate (such as a checkerboard + fluorescent marker) so that the error between the two is < 2 μm. Then, the high-speed polarization camera and the hyperspectral imaging camera are ensured to be strictly synchronized in imaging through a trigger signal. After that, the spectral data in the abnormal region image is mapped into a three-dimensional refractive index anomaly map through coordinate transformation.

[0042] S105. Determine the defect type and defect cause of the glass substrate according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data.

[0043] In this embodiment, the three-dimensional refractive index anomaly map Δ n ( x , y , z ) is stored in the form of a three-dimensional array, and each voxel corresponds to the refractive index perturbation value at the spatial coordinates ( x , y , z ). The spectral data can be a 256-dimensional feature vector extracted by the hyperspectral imaging system and the HySpectraNet model, which is used to characterize the spectral-spatial joint features of the defect. Specifically, in this embodiment, the defect type and defect cause of the glass substrate can be determined by comparing the refractive index perturbation value and spectral data with the corresponding preset thresholds. Table 1 below shows the defect type and defect cause of the glass substrate determined according to the refractive index perturbation value, and Table 2 shows the defect type and defect cause of the glass substrate determined according to the spectral data.

[0044] Table 1

[0045]

[0046] Table 2

[0047] 。

[0048] In an alternative embodiment provided by the present application, the determining the defect type and defect cause of the glass substrate according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data includes:

[0049] S1051. Normalize the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data.

[0050] Specifically, in this embodiment, the refractive index perturbation value Δn in the three-dimensional refractive index anomaly map is normalized to [-1, 1] through the following formula, and the spectral data (spectral feature vector ν) is Z-score standardized.

[0051] (Truncated to ±1).

[0052] S1052. Construct graph structure data based on the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map with the normalized mapping spectral data.

[0053] In this embodiment, determine the node features of each node in the graph structure data according to the refractive index perturbation values and spectral data. The node features may include the mean value of Δn, variance, and spatial gradient, as well as the spectral features in the spectral data.

[0054] S1053. Determine the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data.

[0055] Specifically, the determination of the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data includes: performing data preprocessing on the refractive index perturbation values, spectral data, and connection edge weights of the defect nodes, and determining the defect node feature vector according to the refractive index perturbation values and spectral data of the defect nodes after data preprocessing, and determining the connection edge weight feature vector from the connection edge weights after data preprocessing; inputting the defect node feature vector and the connection edge weight feature vector into the glass substrate recognition model to obtain the defect type and defect cause of the glass substrate.

[0056] Among them, data preprocessing can be data standardization and normalization, etc. Then, splice the refractive index perturbation values and spectral data of the defect nodes to obtain the defect node feature vector ν = [statistic, 256-dimensional spectral vector]. The defect nodes in the graph structure data are determined according to the refractive index perturbation values and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between the defect nodes. If the Euclidean distance between defects < threshold (such as 100 μm), then build an edge, and the weight is the reciprocal of the distance; if the cosine similarity of spectral features > 0.9, then build an edge, and the weight is the similarity value. n

[0057] ​In an alternative embodiment, a defect node feature vector is determined based on the refractive index perturbation value and spectral data of the defect node after data preprocessing, and a connection edge weight feature vector is determined based on the connection edge weight after data preprocessing, including: splicing the refractive index perturbation value and spectral data of the defect node after data preprocessing to obtain a defect node feature vector, and splicing the connection edge weight after data preprocessing to obtain a connection edge weight feature vector. Further, when there are multiple edges between two nodes i and j due to physical proximity and compositional similarity, the determination process of the connection edge weight between nodes i and j is as follows: respectively determine the weight values corresponding to physical proximity and compositional similarity; normalize the two calculated weight values; perform weighted calculation on the two normalized weight values to obtain the final connection edge weight.

[0058] More specifically, inputting the defect node feature vector and the connection edge weight feature vector into the glass substrate recognition model to obtain the defect type and defect cause of the glass substrate, including: inputting the defect node feature vector and the connection edge weight feature vector into the input layer of the glass substrate recognition model; the input layer inputs the defect node feature vector and the connection edge weight feature vector into the convolutional layer, and the convolutional layer calculates the feature vector of the defect node through the following formula; ,

[0059] where, is the feature vector of node i in the L-th layer, the learnable weight matrix of the L-th layer, is the feature vector of the neighbor node j in the L-th layer, is the neighbor node in the L-th layer i 's feature vector, is the connection edge weight feature vector between node i and node j , is the neighbor set of node i, and σ is a non-linear activation function;

[0060] The convolutional layer inputs the feature vector of the defect node into the aggregation layer, and the aggregation layer is used to aggregate the feature vectors of the defect nodes input by all convolutional layers to obtain a defect node aggregation feature vector, and input the defect node aggregation feature vector into the output layer; the output layer is used to predict the defect type and defect cause of the glass substrate based on the defect node aggregation feature vector.

[0061] Scenario: Mixed defect clusters on a glass substrate

[0062] Node A: (bubble), and the spectral data shows peak (metal contamination); Node B: (bubble), the spectral data shows peak; Node C: (crack), the spectral data has no pollution peak. Physical adjacent edges: A - B (distance 50μm, weight 0.9), A - C (distance 200μm, no connection); Composition - similar edges: A - B (similarity 0.95, weight 0.95), A - C (similarity 0.1, no connection). In this example, the Δn of the defect node and the spectral data are used to determine the defect - node feature vector, and the weights of the connecting edges are used to determine the connecting - edge weight feature vector; The defect - node feature vector and the connecting - edge weight feature vector are input into the glass - substrate recognition model to obtain the defect type and the defect cause of the glass substrate. That is, the defect type and the defect cause obtained by the glass - substrate recognition model in this embodiment are:

[0063] Node A receives two types of messages, physical and compositional, from node B, and node C has no connection. Feature update: The feature of node A strengthens the "bubble + pollution" mode, and node C maintains the "crack" feature. Classification result: Node A, Node B: Determined as "metal - pollution bubble cluster"; Node C: Determined as "isolated crack".

[0064] This embodiment provides a method for detecting defects of a glass substrate applied to the inside of a display panel. First, set the light - source parameters of the multi - angle polarized - light source array to obtain multi - angle polarized - light - field data I(θ,λ,P,x,y) through the high - speed polarized camera; Then generate a three - dimensional refractive - index anomaly map according to the multi - angle polarized - light - field data I(θ,λ,P,x,y); The three - dimensional refractive - index anomaly map includes the refractive - index perturbation values corresponding to each voxel; Determine the abnormal - position coordinates through the refractive - index perturbation values in the three - dimensional refractive - index anomaly map, and intercept the abnormal - region image from the two - dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal - position coordinates; The two - dimensional spectral image includes the spectral data of each position point; Map the spectral data in the abnormal - region image to the three - dimensional refractive - index anomaly map through coordinate transformation; Determine the defect type and the defect cause of the glass substrate according to the refractive - index perturbation values and the spectral data in the three - dimensional refractive - index anomaly map with mapped spectral data. Compared with the prior art that relies on a single optical detection method to detect the glass substrate, this application combines two technologies, polarized imaging and hyperspectral imaging, realizes the fusion analysis of multi - modal data, and significantly improves the accuracy of glass - substrate defect detection.

[0065] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0066] In one embodiment, a glass substrate defect detection device is provided, which corresponds one-to-one with the defect detection method for the glass substrate applied inside the display panel in the above embodiment. As Figure 3 shown, the detailed description of each functional module of the glass substrate defect detection device is as follows:

[0067] An acquisition module 31, configured to set the light source parameters of the multi-angle polarization light source array, so as to acquire multi-angle polarization light field data I(θ,λ,P,x,y) through the high-speed polarization camera; where θ is the incident angle, λ is the wavelength, P is the polarization state, and (x,y) is the position coordinate;

[0068] A generation module 32, configured to generate a three-dimensional refractive index anomaly map according to the multi-angle polarization light field data I(θ,λ,P,x,y); the three-dimensional refractive index anomaly map includes the refractive index perturbation values corresponding to each voxel;

[0069] A determination module 33, configured to determine the abnormal position coordinates determined by the refractive index perturbation values in the three-dimensional refractive index anomaly map, and intercept the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes the spectral data of each position point;

[0070] A mapping module 34, configured to map the spectral data in the abnormal area image to the three-dimensional refractive index anomaly map through coordinate transformation;

[0071] A prediction module 35, configured to determine the defect type and defect cause of the glass substrate according to the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data.

[0072] In an optional embodiment, the generation module 32 is specifically configured to:

[0073] Acquire the surface height distribution of the glass substrate through the white light interferometer, and calculate the theoretical optical path according to the surface height distribution; calculate the actual optical path according to the multi-angle polarization light field data I(θ,λ,P,x,y) and the reference light intensity of the defect-free glass substrate;

[0074] Obtain the three-dimensional refractive index anomaly map according to the theoretical optical path and the actual optical path.

[0075] In an optional embodiment, the generation module 32 is specifically configured to:

[0076] Calculate the actual refractive index corresponding to the current temperature according to the current temperature, the reference temperature, the thermo-optic coefficient of the glass substrate, and the standard refractive index corresponding to the reference temperature;

[0077] Calculate the theoretical optical path according to the actual refractive index corresponding to the current temperature and the surface height distribution.

[0078] In an optional embodiment, the glass substrate defect detection device further includes a preprocessing module 36, and the preprocessing module 36 is specifically configured to:

[0079] Set the band range of the hyperspectral imaging camera and the shooting parameters of the LED uniform light source, and adjust the angle of the six-axis robotic arm to enable the hyperspectral imaging camera to collect two-dimensional spectral images at multiple angles;

[0080] Perform data preprocessing on the two-dimensional spectral images collected by the hyperspectral imaging camera, and perform feature extraction on the preprocessed two-dimensional spectral images to obtain spectral feature vectors;

[0081] The determination module 33 is specifically configured to intercept the abnormal region image from the spectral feature vectors according to the abnormal position coordinates.

[0082] In an optional embodiment, the prediction module 35 is specifically configured to:

[0083] Perform data normalization on the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data;

[0084] Construct graph structure data according to the refractive index perturbation values and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data after normalization. The defect nodes in the graph structure data are determined according to the refractive index perturbation values and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between the defect nodes;

[0085] Determine the defect type and defect cause of the glass substrate through the defect nodes and the corresponding connection edges in the graph structure data.

[0086] In an optional embodiment, the prediction module 35 is specifically configured to:

[0087] Perform data preprocessing on the refractive index perturbation values, spectral data, and connection edge weights of the defect nodes, and determine the defect node feature vectors according to the refractive index perturbation values and spectral data of the preprocessed defect nodes. Determine the connection edge weight feature vectors for the preprocessed connection edge weights;

[0088] Input the defect node feature vectors and the connection edge weight feature vectors into the glass substrate recognition model to obtain the defect type and defect cause of the glass substrate.

[0089] In an optional embodiment, the prediction module 35 is specifically configured to:

[0090] The refractive index perturbation value and spectral data of the defective nodes after data preprocessing are spliced to obtain the defective node feature vector, and the connection edge weights after data preprocessing are spliced to obtain the connection edge weight feature vector;

[0091] The defective node feature vector and the connection edge weight feature vector are input into the input layer of the glass substrate recognition model;

[0092] The input layer inputs the defective node feature vector and the connection edge weight feature vector into the convolutional layer, and the convolutional layer calculates the feature vector of the defective node through the following formula;

[0093] ,

[0094] where, is the feature vector of node i in the L-th layer, the learnable weight matrix of the L-th layer, is the feature vector of the L-th neighbor nodes j of, is the neighbor node in the L-th layer i 's feature direction, is the connection edge weight feature vector between node i and node j of, is the neighbor set of node i, and σ is the non-linear activation function;

[0095] The convolutional layer inputs the feature vector of the defective node into the aggregation layer, and the aggregation layer is used to aggregate the feature vectors of the defective nodes input by all convolutional layers to obtain the defective node aggregation feature vector, and input the defective node aggregation feature vector into the output layer;

[0096] The output layer is used to predict the defective type and defective reason of the glass substrate according to the defective node aggregation feature vector.

[0097] For the specific definition of the glass substrate defect detection device, reference can be made to the definition of the glass substrate defect detection method applied to the inside of the display panel in the above text, which will not be elaborated here. Each module in the above device can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0098] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as shown in Figure 4As shown. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a method for detecting defects in a glass substrate inside a display panel.

[0099] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0100] Obtain a spinal endoscope image from a spinal endoscope video stream, where each frame of the spinal endoscope image in the spinal endoscope video stream corresponds to time information, angle information, and position information;

[0101] Filter the spinal endoscope image according to the time information, angle information, and position information of each frame of the spinal endoscope image;

[0102] Perform image preprocessing on each frame of the filtered spinal endoscope image, and input the preprocessed spinal endoscope image into a spinal recognition model to obtain the spinal position area corresponding to each frame of the spinal endoscope image. The spinal position area at least includes the position areas where the vertebral body, intervertebral disc, and nerve root are located;

[0103] Extract a spinal structure image from the spinal endoscope image according to the spinal position area;

[0104] Determine the depth information of each pixel point in the spinal structure image based on the angle information and position information corresponding to each frame of the spinal structure image;

[0105] Project the position information and depth information of each pixel point in each frame of the spinal structure image to the corresponding positions in a three-dimensional space to form point cloud data; and convert the point cloud data into a triangular mesh to obtain a three-dimensional model of the spinal endoscope image.

[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0107] Obtain a spinal endoscope image from a spinal endoscope video stream, where each frame of the spinal endoscope image in the spinal endoscope video stream corresponds to time information, angle information, and position information;

[0108] Filter the spinal endoscope images according to the time information, angle information, and position information of each frame of the spinal endoscope images;

[0109] Perform image preprocessing on each frame of the filtered spinal endoscope images, and input the preprocessed spinal endoscope images into a spinal recognition model to obtain the spinal position regions corresponding to each frame of the spinal endoscope images, where the spinal position regions at least include the position regions where the vertebral bodies, intervertebral discs, and nerve roots are located;

[0110] Extract spinal structure images from the spinal endoscope images according to the spinal position regions;

[0111] Determine the depth information of each pixel point in the spinal structure images based on the angle information and position information corresponding to each frame of the spinal structure images;

[0112] Project the position information and depth information of each pixel point in each frame of the spinal structure images to the corresponding positions in the three-dimensional space to form point cloud data; and convert the point cloud data into a triangular mesh to obtain a three-dimensional model of the spinal endoscope image.

[0113] In one embodiment, a computer program product is provided, where the computer program product includes a computer program, and the computer program is executed by a processor to implement the following steps:

[0114] Obtain spinal endoscope images from a spinal endoscope video stream, where each frame of the spinal endoscope images in the spinal endoscope video stream corresponds to time information, angle information, and position information;

[0115] Filter the spinal endoscope images according to the time information, angle information, and position information of each frame of the spinal endoscope images;

[0116] Perform image preprocessing on each frame of the filtered spinal endoscope images, and input the preprocessed spinal endoscope images into a spinal recognition model to obtain the spinal position regions corresponding to each frame of the spinal endoscope images, where the spinal position regions at least include the position regions where the vertebral bodies, intervertebral discs, and nerve roots are located;

[0117] Extract spinal structure images from the spinal endoscope images according to the spinal position regions;

[0118] Determine the depth information of each pixel point in the spinal structure images based on the angle information and position information corresponding to each frame of the spinal structure images;

[0119] Project the position information and depth information of each pixel point in each frame of the spinal structure image to the corresponding positions in three-dimensional space to form point cloud data; and convert the point cloud data into a triangular mesh to obtain a three-dimensional model of the endoscopic image of the spine.

[0120] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0121] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0122] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting defects of a glass substrate inside a display panel, characterized in that, The method is applied to a glass substrate defect detection system, which includes: a refractive image module, a spectral image module, a collaborative control module, and a glass substrate defect detection device; the refractive image module includes a multi-angle polarized light source array, a high-speed polarized camera, and a white light interferometer; the spectral image module includes a hyperspectral imaging camera and an LED uniform light source; the collaborative control module includes a six-axis motion platform and a six-axis robotic arm, and a vacuum chuck for adsorbing the glass substrate is arranged on the six-axis motion platform; the multi-angle polarized light source array and the hyperspectral imaging camera are arranged on the six-axis robotic arm, and the high-speed polarized camera and the LED uniform light source are arranged directly above the glass substrate; the glass substrate defect detection device is connected to the refractive image module, the spectral image module, and the collaborative control module, and is used to determine the defect type and defect cause of the glass substrate through the image data obtained by the refractive image module and the spectral image module; Determining the defect type and defect cause of the glass substrate through the image data obtained by the refractive image module and the spectral image module includes: Set the light source parameters of the multi-angle polarization light source array to obtain multi-angle polarization light field data through the high-speed polarization camera I(θ,λ,P,x,y) ; where θ is the incident angle, λ is the wavelength, P is the polarization state, ([[]] x,y ) is the position coordinate; Based on the multi-angle polarized light field data I(θ,λ,P,x,y) generate a three-dimensional refractive index anomaly map; the three-dimensional refractive index anomaly map includes the refractive index perturbation values corresponding to each voxel; The abnormal position coordinates determined by the refractive index perturbation value in the three-dimensional refractive index anomaly map, and an abnormal region image is intercepted from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; The spectral data in the abnormal region image is mapped to the three-dimensional refractive index anomaly map through coordinate transformation; The defect type and defect cause of the glass substrate are determined according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data; Determining the defect type and defect cause of the glass substrate according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data includes: Performing data normalization on the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data; Constructing graph structure data according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data after normalization, the defect nodes in the graph structure data are determined according to the refractive index perturbation value and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between defect nodes; The defect type and defect cause of the glass substrate are determined through the defect nodes and corresponding connection edges in the graph structure data.

2. The method according to claim 1, wherein According to the multi-angle polarized light field data I(θ,λ, P,x,y) generating a three-dimensional refractive index anomaly map, including: Obtain the surface height distribution of the glass substrate through the white light interferometer, and calculate the theoretical optical path according to the surface height distribution; according to the multi-angle polarized light field data I(θ,λ,P,x,y) and the reference light intensity of the defect-free glass substrate to calculate the actual optical path; The three-dimensional refractive index anomaly map is obtained according to the theoretical optical path and the actual optical path.

3. The method according to claim 2, characterized in that, Calculating the theoretical optical path according to the surface height distribution includes: Calculating the actual refractive index corresponding to the current temperature according to the current temperature, the reference temperature, the thermo-optic coefficient of the glass substrate, and the standard refractive index corresponding to the reference temperature; Calculating the theoretical optical path according to the actual refractive index corresponding to the current temperature and the surface height distribution.

4. The method according to claim 1, wherein Before intercepting the abnormal area image from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates, the method further includes: Setting the band range of the hyperspectral imaging camera and the shooting parameters of the LED uniform light source, and adjusting the angle of the six-axis robotic arm to enable the hyperspectral imaging camera to collect two-dimensional spectral images at multiple angles; Performing data preprocessing on the two-dimensional spectral images collected by the hyperspectral imaging camera, and performing feature extraction on the two-dimensional spectral images after data preprocessing to obtain spectral feature vectors; Intercepting the abnormal area image from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates includes: intercepting the abnormal area image from the spectral feature vectors according to the abnormal position coordinates.

5. The method according to claim 1, characterized in that, Determining the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data includes: Performing data preprocessing on the refractive index perturbation value, spectral data, and connection edge weight of the defect nodes, determining the defect node feature vectors according to the refractive index perturbation value and spectral data of the defect nodes after data preprocessing, and determining the connection edge weight feature vectors from the connection edge weights after data preprocessing; Inputting the defect node feature vectors and the connection edge weight feature vectors into the glass substrate recognition model to obtain the defect type and defect cause of the glass substrate.

6. A glass substrate defect detection device, characterized in that, The device is applied to execute the defect detection method for the glass substrate inside the display panel according to any one of claims 1 to 5. The device includes: An acquisition module, configured to set the light source parameters of the multi-angle polarization light source array, so as to acquire multi-angle polarization light field data through the high-speed polarization camera I(θ,λ,P,x,y) ; wherein, θ is the incident angle, λ is the wavelength, P is the polarization state, ([[]] x,y ) is the position coordinate; A generation module, configured to generate a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I(θ,λ,P,x,y) The three-dimensional refractive index anomaly map includes refractive index perturbation values corresponding to each voxel; A determination module, configured to determine the abnormal position coordinates through the refractive index perturbation value in the three-dimensional refractive index anomaly map, and intercept the abnormal area image from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; A mapping module, configured to map the spectral data in the abnormal area image to the three-dimensional refractive index anomaly map through coordinate transformation; A prediction module, configured to determine the defect type and defect cause of the glass substrate according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data; Determining the defect type and defect cause of the glass substrate according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data includes: Performing data normalization on the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data; Constructing graph structure data according to the refractive index perturbation value and spectral data in the three-dimensional refractive index anomaly map mapped with spectral data after normalization. The defect nodes in the graph structure data are determined according to the refractive index perturbation value and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between defect nodes; Determining the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method for detecting defects in the glass substrate inside the display panel according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method for detecting defects in the glass substrate inside the display panel according to any one of claims 1 to 5 is implemented.

Images