Defect detection method and device applied to glass substrate in display panel

Abstract

The invention provides a defect detection method and device applied to a glass substrate in a display panel. The defect detection method and device are used for improving the defect detection accuracy of the glass substrate. The method mainly comprises the following steps: acquiring multi-angle polarized light field data through a high-speed polarization camera; generating a three-dimensional refractive index anomaly graph according to the multi-angle polarized light field data; the three-dimensional refractive index anomaly graph comprises a refractive index disturbance value corresponding to each voxel; determining an abnormal position coordinate according to a refractive index disturbance value in the three-dimensional refractive index abnormal graph, and intercepting an abnormal region image from a two-dimensional spectral image acquired by a hyperspectral imaging camera according to the abnormal position coordinate; the two-dimensional spectral image comprises spectral data of each position point; mapping spectral data in the abnormal region image into a three-dimensional refractive index abnormal graph through coordinate transformation; and determining the defect type and the defect reason of the glass substrate according to the refractive index disturbance value in the three-dimensional refractive index anomaly graph mapped with the spectral data and the spectral data.

Description

Technical Field

[0001] The present application relates to the field of optical detection technology, and in particular to a defect detection method and device for a glass substrate inside a display panel. Background Art

[0002] With the development of society, electronic display devices are being updated faster and faster, and the types and sizes of display devices are increasing, and the requirements for display devices are becoming higher and higher. As the main raw material of liquid crystal displays, glass substrates are prone to defects during the production process, and their product quality and utilization rate are directly related to the imaging effect of liquid crystal displays.

[0003] Traditional glass substrate defect detection methods usually rely on a single optical detection method, such as brightness detection, contrast detection or simple surface defect detection. These methods have limitations in detecting complex defects (such as internal bubbles, impurities, stress unevenness, etc.), resulting in insufficient detection accuracy. Summary of the invention

[0004] The embodiments of the present application provide a defect detection method and device for a glass substrate inside a display panel, which are used to improve the accuracy of defect detection of the glass substrate.

[0005] An embodiment of the present invention provides a defect detection method for a glass substrate applied to a display panel, the method being applied to a glass substrate defect detection system, the system comprising: a refraction image module, a spectral image module, a collaborative control module and a glass substrate defect detection device; the refraction image module comprises a multi-angle polarized light source array, a high-speed polarization camera, and a white light interferometer; the spectral image module comprises a hyperspectral imaging camera and an LED uniform light source; the collaborative control module comprises a six-axis motion platform and a six-axis robotic arm, the six-axis motion platform being provided with a vacuum suction cup for adsorbing the glass substrate; the multi-angle polarized light source array and the hyperspectral imaging camera are arranged on the six-axis robotic arm, the high-speed polarization camera and the LED uniform light source are arranged directly above the glass substrate; the glass substrate defect detection device is connected to the refraction image module, the spectral image module and the collaborative control module, the glass substrate defect detection device being used to determine the defect type and defect cause of the glass substrate through image data acquired by the refraction image module and the spectral image module.

[0006] An embodiment of the present invention provides a glass substrate defect detection device, the device is used to perform the defect detection method of a glass substrate applied to a display panel according to any one of claims 1 to 7, the device comprising: An acquisition module, used to set the light source parameters of the multi-angle polarized light source array, so as to acquire multi-angle polarized light field data I(θ, λ, P, x, y) through the high-speed polarization camera; wherein θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) is the position coordinate; A generating module, used for generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I(θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes a refractive index disturbance value corresponding to each voxel; A determination module, used to determine the coordinates of the abnormal position through the refractive index disturbance value in the three-dimensional refractive index abnormality map, and to intercept the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; A mapping module, used for mapping the spectral data in the abnormal region image to the three-dimensional refractive index abnormality map through coordinate transformation; The prediction module is used to determine the defect type and defect cause of the glass substrate according to the refractive index disturbance value in the three-dimensional refractive index anomaly map mapped with the spectral data and the spectral data.

[0007] A computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned defect detection method for a glass substrate inside a display panel.

[0008] The present invention provides a defect detection method and device for a glass substrate inside a display panel. First, light source parameters of the multi-angle polarized light source array are set to obtain multi-angle polarized light field data I(θ, λ, P, x, y) through the high-speed polarization camera; then a three-dimensional refractive index anomaly map is generated according to the multi-angle polarized light field data I(θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes refractive index disturbance values ​​corresponding to each voxel; the coordinates of the abnormal position are determined by the refractive index disturbance values ​​in the three-dimensional refractive index anomaly map, and an abnormal area image is intercepted from a two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; the spectral data in the abnormal area image is mapped to the three-dimensional refractive index anomaly map through coordinate transformation; the defect type and defect cause of the glass substrate are determined according to the refractive index disturbance values ​​and spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data. Compared with the existing technology that relies on a single optical detection method to detect glass substrates, the present application combines polarization imaging and hyperspectral imaging technologies to achieve fusion analysis of multimodal data, significantly improving the accuracy of glass substrate defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A structural diagram of a defect detection system for a glass substrate inside a display panel provided in the present application; Figure 2 A flow chart of a defect detection method for a glass substrate inside a display panel provided in the present application; Figure 3 A schematic diagram of the structure of a glass substrate defect detection device provided in the present application; Figure 4 A schematic diagram of a computer device provided in this application. DETAILED DESCRIPTION

[0010] In order to better understand the above-mentioned technical scheme, the technical scheme of the embodiments of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the embodiments of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0011] See also Figure 1 , a defect detection system for a glass substrate inside a display panel provided by an embodiment of the present invention, the system comprising: a refraction image module, a spectral image module, a collaborative control module and a glass substrate defect detection device; the refraction image module comprises a multi-angle polarized light source array, a high-speed polarization camera, and a white light interferometer; the spectral image module comprises a hyperspectral imaging camera and an LED uniform light source; the collaborative control module comprises a six-axis motion platform and a six-axis robotic arm, and the six-axis motion platform is provided with a vacuum suction cup for adsorbing the glass substrate; the multi-angle polarized light source array and the hyperspectral imaging camera are arranged on the six-axis robotic arm, and the high-speed polarization camera and the LED uniform light source are arranged directly above the glass substrate; the glass substrate defect detection device is connected to the refraction image module, the spectral image module, and the collaborative control module. Among them, the parameters of the multi-angle polarized light source array in this embodiment can be 6 groups of lasers with a wavelength of 405-1550nm; the parameters of the high-speed polarization camera can be 4096×2160 pixels, 1000fps; white light interferometer (Zygo Verifire); hyperspectral imaging camera (HySpex VS-6200, 400-2500nm); LED uniform light source (polarization controllable); six-axis motion platform (XYZ translation + tilt).

[0012] The refraction image module is located in the upper part of the shell, arranged in parallel with the spectral image module, and fixed by an isolation bracket to reduce optical interference. Multi-angle polarized light source array: arranged horizontally along the top of the shell, the incident angle of the light source covers 0°-90°, and is adjusted by a six-axis robotic arm. High-speed polarization camera: facing the glass substrate detection area, at a 45° angle with the light source array, and fine-tuning the focus through the guide rail. Interferometer calibration unit: integrated under the camera, real-time monitoring of substrate surface deformation, data feedback to the collaborative control module. The spectral image module is located in the lower part of the shell, sharing the same optical path channel with the refraction image module, and realizing multi-modal data synchronous acquisition through a spectroscope. The hyperspectral imaging camera is set on a six-axis robotic arm, supporting XYZ axis translation to adapt to substrates of different sizes. The LED uniform light source is installed around the camera lens, and the diffuse reflection plate is used to evenly illuminate and eliminate mirror reflection noise. The collaborative control module is centrally arranged, and the refraction image module and the spectral image module are connected through the back panel. The six-axis motion platform is equipped with a vacuum suction cup for adsorbing the glass substrate, which is driven by a stepper motor to achieve a uniform sweeping speed of 0.1m / s for the substrate.

[0013] See also Figure 2 , a defect detection method for a glass substrate inside a display panel provided by an embodiment of the present invention, the method is applied to a glass substrate defect detection device in the above-mentioned defect detection system for a glass substrate inside a display panel, the method specifically includes S101-S105: S101, setting light source parameters of the multi-angle polarized light source array to acquire multi-angle polarized light field data I(θ, λ, P, x, y) through the high-speed polarization camera.

[0014] In this embodiment, the multi-angle polarized light source array can use 6 groups of tunable semiconductor lasers (wavelengths of 405nm, 532nm, 635nm, 780nm, 1064nm, and 1550nm), covering the visible light to near-infrared band. Each group of light sources integrates a liquid crystal polarization controller to support dynamic switching between 0°-180° linear polarization and circular polarization. The multi-angle polarized light source array can support continuous adjustment of the incident angle from 0° to 90°, covering omnidirectional lighting needs. The high-speed polarization camera is equipped with a quantum dot enhanced CMOS sensor, integrated with a four-way polarization filter (0°, 45°, 90°, 135°), and a frame rate of 1000fps. The white light interferometer monitors the surface deformation and refractive index fluctuations of the glass substrate in real time.

[0015] Specifically, this embodiment scans each preset incident angle (such as incident angles of 0°, 30°, and 60°) and polarization state (0° linear polarization, right-hand circular polarization) combination, collects the intensity and polarization state distribution of the reflected / transmitted light field, and obtains multi-angle polarized light field data I(θ, λ, P, x, y), where θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) is the position coordinate.

[0016] S102, generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I (θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes a refractive index disturbance value corresponding to each voxel.

[0017] In an optional embodiment provided in the present application, generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I (θ, λ, P, x, y) includes: S1021, obtaining the surface height distribution of the glass substrate through the white light interferometer, and calculating the theoretical optical path according to the surface height distribution; calculating the actual optical path according to the multi-angle polarized light field data I(θ, λ, P, x, y) and the reference light intensity of the defect-free glass substrate.

[0018] In this embodiment, the substrate surface height distribution h(x,y) is first obtained by white light interferometry, and then combined with the material refractive index n(λ) Calculate the theoretical optical path , and calculate the measured optical path ,in The defect-free reference value is then used to generate a three-dimensional refractive index anomaly map based on the theoretical optical path and the measured optical path.

[0019] It should be noted that the refractive index of the material changes due to temperature changes (for example, when ΔT=1°C, the refractive index of soda-lime glass changes ) will affect the calculation of the theoretical optical path. To solve this problem, the calculation of the theoretical optical path according to the surface height distribution in this embodiment includes: calculating the actual refractive index corresponding to the current temperature according to the current temperature, the reference temperature, the thermo-optical coefficient of the glass substrate and the standard refractive index corresponding to the reference temperature; and calculating the theoretical optical path according to the actual refractive index corresponding to the current temperature and the surface height distribution.

[0020] Specifically, in this embodiment, the formula Calculate the actual refractive index corresponding to the current temperature. Where n(Tλ) is the actual refractive index of material λ at temperature T, For materials at reference temperature The nominal refractive index of soda-lime glass ≈1.52, quartz glass ≈1.46), β is the thermo-optical coefficient of the material, which is used to indicate the change in refractive index when the temperature rises by 1°C; T is the current ambient temperature, is the reference temperature (the reference temperature during calibration). Then according to the formula The actual refractive index corresponding to the current temperature is calculated.

[0021] Furthermore, in order to adapt to a wide temperature range (such as -50°C to 150°C) or high-precision scenarios, this embodiment needs to introduce high-order correction, that is, through the formula The actual refractive index corresponding to the current temperature is calculated, wherein γ is the second-order thermo-optic coefficient.

[0022] S1022, obtaining the three-dimensional refractive index anomaly map according to the theoretical optical path and the actual optical path.

[0023] In this embodiment, an iterative algebraic reconstruction algorithm can be used to solve the three-dimensional refractive index anomaly map Δ n ( x , y , z ) , in, is the change in optical path difference, Δ n ( x , y , z ) represents the refractive index disturbance value of a certain point (x, y, z) in the glass substrate. z is the depth coordinate along the light propagation direction, which represents the depth position currently calculated and is used to locate the position of the defect in three-dimensional space. It is the sum of the refractive index disturbances accumulated from z0 to z1 along the light propagation direction (z axis).

[0024] S103, determining the coordinates of the abnormal position by the refractive index disturbance value in the three-dimensional refractive index abnormality map, and intercepting the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates.

[0025] The two-dimensional spectral image includes spectral data of each position point. Specifically, in this embodiment, the refractive index disturbance value can be compared with a predetermined threshold to determine the coordinates of the abnormal position. (Typical refractive index fluctuation threshold of glass) is judged as a physical defect.

[0026] In an optional embodiment provided in the present application, before intercepting the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates, the method further includes: setting the band range of the hyperspectral imaging camera and the shooting parameters of the LED uniform light source, and enabling the hyperspectral imaging camera to collect two-dimensional spectral images at multiple angles by adjusting the angle of the six-axis robotic arm; performing data preprocessing on the two-dimensional spectral image acquired by the hyperspectral imaging camera, and performing feature extraction on the two-dimensional spectral image after data preprocessing to obtain a spectral feature vector; accordingly, intercepting the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates includes: intercepting the abnormal area image from the spectral feature vector according to the abnormal position coordinates.

[0027] Specifically, the glass substrate is controlled to pass through the field of view at a uniform speed, and the hyperspectral imaging camera scans line by line to generate two-dimensional spectral image data. Data is collected at multiple incident angles of 15°, 45°, and 75°. Among them, data preprocessing may include dark current correction, normalization, spectral smoothing and denoising, etc., which are not specifically limited in this embodiment.

[0028] S104, mapping the spectral data in the abnormal region image into a three-dimensional refractive index abnormality map through coordinate transformation.

[0029] It should be noted that, before obtaining multi-angle polarized light field data and two-dimensional spectral images through the high-speed polarization camera and the hyperspectral imaging camera, this embodiment also needs to use a calibration plate (such as a checkerboard + fluorescent marker) to align the field of view of the high-speed polarization camera and the hyperspectral imaging camera so that the error between the two is less than 2μm. Then, the trigger signal is used to ensure that the imaging of the high-speed polarization camera and the hyperspectral imaging camera is strictly synchronized. After that, the spectral data in the abnormal area image is mapped to the three-dimensional refractive index anomaly map through coordinate transformation.

[0030] S105, determining the defect type and defect cause of the glass substrate according to the refractive index disturbance value in the three-dimensional refractive index anomaly map mapped with the spectral data and the spectral data.

[0031] In this embodiment, the three-dimensional refractive index anomaly map Δ n ( x , y , z ) is stored in a three-dimensional array, and each voxel corresponds to a spatial coordinate ( x , y , z) at a refractive index disturbance value; the spectral data may be a 256-dimensional feature vector extracted by the hyperspectral imaging system and the HySpectraNet model, which is used to characterize the spectral-spatial joint characteristics of the defect. Specifically, this embodiment can determine the defect type and defect cause of the glass substrate by comparing the refractive index disturbance value and the spectral data with the corresponding preset threshold value. The following Table 1 shows the defect type and defect cause of the glass substrate determined according to the refractive index disturbance value, and Table 2 shows the defect type and defect cause of the glass substrate determined according to the spectral data.

[0032] Table 1

[0033] Table 2 .

[0034] In an optional embodiment provided in the present application, determining the defect type and defect cause of the glass substrate according to the refractive index disturbance value and the spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data includes: S1051, performing data normalization on the refractive index disturbance value and the spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data.

[0035] Specifically, in this embodiment, the refractive index disturbance value Δn in the three-dimensional refractive index anomaly map is normalized to [-1, 1] by the following formula, and the spectral data (spectral feature vector ν) is Z-score standardized.

[0036] (truncated to ±1).

[0037] S1052, constructing graph structure data according to the normalized refractive index disturbance value in the three-dimensional refractive index anomaly graph mapped with the spectral data and the spectral data.

[0038] In this embodiment, the node feature of each node in the graph structure data is determined according to the refractive index disturbance value and the spectral data. The node feature may include the Δn mean, variance and spatial gradient, and the spectral feature in the spectral data.

[0039] S1053: Determine the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data.

[0040] Specifically, the method of determining the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data includes: performing data preprocessing on the refractive index disturbance value, spectral data, and connection edge weight of the defect node, and determining the defect node feature vector according to the refractive index disturbance value and spectral data of the defect node after data preprocessing, and determining the connection edge weight feature vector according to the connection edge weight after data preprocessing; and inputting the defect node feature vector and the connection edge weight feature vector into a glass substrate recognition model to obtain the defect type and defect cause of the glass substrate.

[0041] Among them, data preprocessing can be data standardization and normalization, and then the refractive index disturbance value and spectral data of the defect node are spliced ​​to obtain the defect node feature vector ν=[Δ n Statistics, 256-dimensional spectral vector]. The defect nodes in the graph structure data are determined according to the refractive index disturbance value and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between the defect nodes. If the Euclidean distance between defects is less than the threshold (such as 100 μm), an edge is established, and the weight is the inverse of the distance; if the spectral feature cosine similarity is greater than 0.9, an edge is established, and the weight is the similarity value.

[0042] In an optional embodiment, the defect node feature vector is determined based on the refractive index disturbance value and spectral data of the defect node after data preprocessing, and the connection edge weight feature vector is determined based on the connection edge weight after data preprocessing, including: splicing the defect node feature vector based on the refractive index disturbance value and spectral data of the defect node after data preprocessing, and splicing the connection edge weight feature vector after data preprocessing. Further, when there are multiple edges between two nodes i and j due to physical proximity and component similarity, the process of determining the connection edge weight between node i and node j is: respectively corresponding to the weight values ​​of physical proximity and component similarity; normalizing the two calculated weight values; weighted calculation of the two normalized weight values ​​to obtain the final connection edge weight.

[0043] More specifically, the step of inputting the defect node feature vector and the connection edge weight feature vector into a glass substrate recognition model to obtain the defect type and defect cause of the glass substrate includes: inputting the defect node feature vector and the connection edge weight feature vector into an input layer of the glass substrate recognition model; the input layer inputting the defect node feature vector and the connection edge weight feature vector into a convolution layer, and the convolution layer calculates the feature vector of the defect node by the following formula; , in, is the feature vector of node i in layer L, The learnable weight matrix of layer L, is the neighbor node in layer L j The characteristic vector of is the neighbor node in layer L i The characteristic direction For node i and node j The connection edge weight feature vector of is the neighbor set of node i, σ is the nonlinear activation function; The convolution layer inputs the feature vector of the defect node to the aggregation layer, and the aggregation layer is used to aggregate the feature vectors of the defect nodes input by all convolution layers to obtain a defect node aggregation feature vector, and input the defect node aggregation feature vector to the output layer; the output layer is used to predict the defect type and defect cause of the glass substrate according to the defect node aggregation feature vector.

[0044] Scenario: Mixed defect clusters on a glass substrate Node A: (bubbles), spectral data show Peak (metal contamination); Node B: (bubbles), spectral data show Peak; Node C: (crack), the spectral data has no contamination peak. Physically adjacent edges: AB (distance 50μm, weight 0.9), AC (distance 200μm, no connection); component similarity edges: AB (similarity 0.95, weight 0.95), AC (similarity 0.1, no connection). In this example, the defect node feature vector is determined by the Δn of the defect node and the spectral data, and the connection edge weight feature vector is determined by the connection edge weight; the defect node feature vector and the connection edge weight feature vector are input into the glass substrate recognition model to obtain the defect type and defect cause of the glass substrate. That is, the defect type and defect cause obtained by the glass substrate recognition model in this embodiment are: Node A receives two types of messages, physical and composition, from node B, and node C has no connection. Feature update: The feature of node A strengthens the "bubble + pollution" mode, and node C maintains the "crack" feature. Classification results: Node A and node B: judged as "metal contamination bubble cluster"; node C: judged as "isolated crack".

[0045] The present embodiment provides a defect detection method for a glass substrate inside a display panel, firstly setting light source parameters of the multi-angle polarized light source array to obtain multi-angle polarized light field data I(θ, λ, P, x, y) through the high-speed polarization camera; then generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I(θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes refractive index disturbance values ​​corresponding to each voxel; the coordinates of the abnormal position are determined by the refractive index disturbance values ​​in the three-dimensional refractive index anomaly map, and an abnormal area image is intercepted from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; the spectral data in the abnormal area image is mapped to the three-dimensional refractive index anomaly map through coordinate transformation; the defect type and defect cause of the glass substrate are determined according to the refractive index disturbance values ​​and spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data. Compared with the existing technology that relies on a single optical detection method to detect glass substrates, the present application combines polarization imaging and hyperspectral imaging technologies to achieve fusion analysis of multimodal data, significantly improving the accuracy of glass substrate defect detection.

[0046] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0047] In one embodiment, a glass substrate defect detection device is provided, which corresponds one-to-one to the defect detection method of the glass substrate applied to the inside of the display panel in the above embodiment. Figure 3 As shown, the functional modules of the glass substrate defect detection device are described in detail as follows: The acquisition module 31 is used to set the light source parameters of the multi-angle polarized light source array to acquire multi-angle polarized light field data I(θ, λ, P, x, y) through the high-speed polarization camera; wherein θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) is the position coordinate; A generating module 32, configured to generate a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I(θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes a refractive index disturbance value corresponding to each voxel; A determination module 33 is used to determine the coordinates of the abnormal position by the refractive index disturbance value in the three-dimensional refractive index abnormality map, and to intercept the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; A mapping module 34, used for mapping the spectral data in the abnormal region image to the three-dimensional refractive index abnormality map through coordinate transformation; The prediction module 35 is used to determine the defect type and defect cause of the glass substrate according to the refractive index disturbance value in the three-dimensional refractive index anomaly map mapped with the spectral data and the spectral data.

[0048] In an optional embodiment, the generating module 32 is specifically configured to: The surface height distribution of the glass substrate is obtained by the white light interferometer, and a theoretical optical path is calculated according to the surface height distribution; the actual optical path is calculated according to the multi-angle polarized light field data I(θ, λ, P, x, y) and the reference light intensity of a defect-free glass substrate; The three-dimensional refractive index anomaly map is obtained according to the theoretical optical path and the actual optical path.

[0049] In an optional embodiment, the generating module 32 is specifically configured to: Calculating an actual refractive index corresponding to the current temperature according to the current temperature, the reference temperature, the thermo-optic coefficient of the glass substrate, and the standard refractive index corresponding to the reference temperature; The theoretical optical path is calculated according to the actual refractive index corresponding to the current temperature and the surface height distribution.

[0050] In an optional embodiment, the glass substrate defect detection device further includes a pre-processing module 36, and the pre-processing module 36 is specifically used for: Setting the band range of the hyperspectral imaging camera and the shooting parameters of the LED uniform light source, and adjusting the angle of the six-axis robotic arm to enable the hyperspectral imaging camera to collect two-dimensional spectral images at multiple angles; Performing data preprocessing on the two-dimensional spectral image collected by the hyperspectral imaging camera, and performing feature extraction on the two-dimensional spectral image after data preprocessing to obtain a spectral feature vector; The determination module 33 is specifically configured to intercept an abnormal region image from the spectral feature vector according to the abnormal position coordinates.

[0051] In an optional embodiment, the prediction module 35 is specifically configured to: Normalizing the refractive index disturbance value and the spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data; Constructing graph structure data according to the normalized refractive index disturbance value and the spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data, wherein the defect nodes in the graph structure data are determined according to the refractive index disturbance value and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between the defect nodes; The defect type and defect cause of the glass substrate are determined through the defect nodes and corresponding connection edges in the graph structure data.

[0052] In an optional embodiment, the prediction module 35 is specifically configured to: The refractive index disturbance value, spectral data and connection edge weight of the defect node are subjected to data preprocessing, and a defect node feature vector is determined according to the refractive index disturbance value and spectral data of the defect node subjected to data preprocessing, and a connection edge weight feature vector is determined according to the connection edge weight subjected to data preprocessing; The defect node feature vector and the connection edge weight feature vector are input into a glass substrate recognition model to obtain the defect type and defect cause of the glass substrate.

[0053] In an optional embodiment, the prediction module 35 is specifically configured to: The defect node feature vector is obtained by splicing the refractive index disturbance value and the spectral data of the defect node after data preprocessing, and the connection edge weight feature vector is obtained by splicing the connection edge weight after data preprocessing; Inputting the defect node feature vector and the connection edge weight feature vector into the input layer of the glass substrate recognition model; The input layer inputs the defect node feature vector and the connection edge weight feature vector into the convolution layer, and the convolution layer calculates the defect node feature vector by the following formula; , in, is the feature vector of node i in layer L, The learnable weight matrix of layer L, is the L neighbor nodes in the layer j The characteristic vector of is the neighbor node in layer L i The characteristic direction For node i and node j The connection edge weight feature vector of is the neighbor set of node i, σ is the nonlinear activation function; The convolution layer inputs the feature vector of the defective node to the aggregation layer, and the aggregation layer is used to aggregate the feature vectors of the defective nodes input by all convolution layers to obtain an aggregated feature vector of the defective node, and input the aggregated feature vector of the defective node to the output layer; The output layer is used to predict the defect type and defect cause of the glass substrate according to the defect node aggregation feature vector.

[0054] For the specific definition of the glass substrate defect detection device, please refer to the definition of the defect detection method of the glass substrate applied to the inside of the display panel above, which will not be repeated here. Each module in the above device can be implemented in whole or in part by software, hardware and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0055] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a defect detection method for a glass substrate inside a display panel is implemented.

[0056] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: Acquire a spinal endoscope image from a spinal endoscope video stream, wherein each frame of the spinal endoscope image in the spinal endoscope video stream corresponds to time information, angle information, and position information; Filtering the spinal endoscope image according to the time information, angle information and position information of each frame of the spinal endoscope image; Performing image preprocessing on each frame of the filtered spinal endoscopic image, and inputting the image preprocessed spinal endoscopic image into a spinal recognition model to obtain a spinal position area corresponding to each frame of the spinal endoscopic image, wherein the spinal position area at least includes a position area where a vertebral body, an intervertebral disc, and a nerve root are located; Extracting a spinal structure image from the spinal endoscope image according to the spinal position area; Determine the depth information of each pixel in the spinal structure image based on the angle information and position information corresponding to each frame of the spinal structure image; The position information and depth information of each pixel point in each frame of the spinal structure image are projected to the corresponding position in the three-dimensional space to form point cloud data; and the point cloud data is converted into a triangular mesh to obtain a three-dimensional model of the endoscopic image of the spine.

[0057] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Acquire a spinal endoscope image from a spinal endoscope video stream, wherein each frame of the spinal endoscope image in the spinal endoscope video stream corresponds to time information, angle information, and position information; Filtering the spinal endoscope image according to the time information, angle information and position information of each frame of the spinal endoscope image; Performing image preprocessing on each frame of the filtered spinal endoscopic image, and inputting the image preprocessed spinal endoscopic image into a spinal recognition model to obtain a spinal position area corresponding to each frame of the spinal endoscopic image, wherein the spinal position area at least includes a position area where a vertebral body, an intervertebral disc, and a nerve root are located; Extracting a spinal structure image from the spinal endoscope image according to the spinal position area; Determine the depth information of each pixel in the spinal structure image based on the angle information and position information corresponding to each frame of the spinal structure image; The position information and depth information of each pixel point in each frame of the spinal structure image are projected to the corresponding position in the three-dimensional space to form point cloud data; and the point cloud data is converted into a triangular mesh to obtain a three-dimensional model of the endoscopic image of the spine.

[0058] In one embodiment, a computer program product is provided, the computer program product comprising a computer program, the computer program being executed by a processor to implement the following steps: Acquire a spinal endoscope image from a spinal endoscope video stream, wherein each frame of the spinal endoscope image in the spinal endoscope video stream corresponds to time information, angle information, and position information; Filtering the spinal endoscope image according to the time information, angle information and position information of each frame of the spinal endoscope image; Performing image preprocessing on each frame of the filtered spinal endoscopic image, and inputting the image preprocessed spinal endoscopic image into a spinal recognition model to obtain a spinal position area corresponding to each frame of the spinal endoscopic image, wherein the spinal position area at least includes a position area where a vertebral body, an intervertebral disc, and a nerve root are located; Extracting a spinal structure image from the spinal endoscope image according to the spinal position area; Determine the depth information of each pixel in the spinal structure image based on the angle information and position information corresponding to each frame of the spinal structure image; The position information and depth information of each pixel point in each frame of the spinal structure image are projected to the corresponding position in the three-dimensional space to form point cloud data; and the point cloud data is converted into a triangular mesh to obtain a three-dimensional model of the endoscopic image of the spine.

[0059] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0060] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0061] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A defect detection method for a glass substrate inside a display panel, characterized in that: The method is applied to a glass substrate defect detection system, which comprises: a refraction image module, a spectral image module, a collaborative control module and a glass substrate defect detection device; the refraction image module comprises a multi-angle polarized light source array, a high-speed polarization camera and a white light interferometer; the spectral image module comprises a hyperspectral imaging camera and an LED uniform light source; the collaborative control module comprises a six-axis motion platform and a six-axis mechanical arm, and a vacuum suction cup for adsorbing a glass substrate is arranged on the six-axis motion platform; the multi-angle polarized light source array and the hyperspectral imaging camera are arranged on the six-axis mechanical arm, and the high-speed polarization camera and the LED uniform light source are arranged just above the glass substrate; the glass substrate defect detection device is connected to the refraction image module, the spectral image module and the collaborative control module, and the glass substrate defect detection device is used to determine the defect type and defect cause of the glass substrate through image data acquired by the refraction image module and the spectral image module.

2. The method according to claim 1, characterized in that The method of determining the defect type and defect cause of the glass substrate by using the image data acquired by the refraction image module and the spectrum image module includes: Setting the light source parameters of the multi-angle polarized light source array to obtain multi-angle polarized light field data I (θ, λ, P, x, y) through the high-speed polarization camera; wherein θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) is the position coordinate; Generate a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I (θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes a refractive index disturbance value corresponding to each voxel; The abnormal position coordinates are determined by the refractive index disturbance value in the three-dimensional refractive index anomaly map, and the abnormal area image is intercepted from the two-dimensional spectral image obtained by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; Mapping the spectral data in the abnormal region image to the three-dimensional refractive index anomaly map by coordinate transformation; The defect type and defect cause of the glass substrate are determined according to the refractive index disturbance value in the three-dimensional refractive index anomaly map mapped with the spectral data and the spectral data.

3. The method according to claim 2, characterized in that Generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I (θ, λ, P, x, y) includes: The surface height distribution of the glass substrate is obtained by the white light interferometer, and a theoretical optical path is calculated according to the surface height distribution; the actual optical path is calculated according to the multi-angle polarized light field data I (θ, λ, P, x, y) and the reference light intensity of a defect-free glass substrate; The three-dimensional refractive index anomaly map is obtained according to the theoretical optical path and the actual optical path.

4. The method according to claim 3, characterized in that The calculating of the theoretical optical path according to the surface height distribution comprises: Calculating an actual refractive index corresponding to the current temperature according to the current temperature, the reference temperature, the thermo-optic coefficient of the glass substrate, and the standard refractive index corresponding to the reference temperature; The theoretical optical path is calculated according to the actual refractive index corresponding to the current temperature and the surface height distribution.

5. The method according to claim 2, characterized in that: Before intercepting the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates, the method further includes: Setting the band range of the hyperspectral imaging camera and the shooting parameters of the LED uniform light source, and adjusting the angle of the six-axis robotic arm to enable the hyperspectral imaging camera to collect two-dimensional spectral images at multiple angles; Performing data preprocessing on the two-dimensional spectral image collected by the hyperspectral imaging camera, and performing feature extraction on the two-dimensional spectral image after data preprocessing to obtain a spectral feature vector; The extracting the abnormal region image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates includes: extracting the abnormal region image from the spectral feature vector according to the abnormal position coordinates.

6. The method according to any one of claims 2 to 5, characterized in that: The method of determining the defect type and defect cause of the glass substrate according to the refractive index disturbance value in the three-dimensional refractive index anomaly map mapped with the spectral data and the spectral data comprises: Normalizing the refractive index disturbance value and the spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data; Constructing graph structure data according to the normalized refractive index disturbance value and the spectral data in the three-dimensional refractive index anomaly map mapped with the spectral data, wherein the defect nodes in the graph structure data are determined according to the refractive index disturbance value and the spectral data, and the connection edge weights in the graph structure data are determined according to the Euclidean distance and similarity between the defect nodes; The defect type and defect cause of the glass substrate are determined through the defect nodes and corresponding connection edges in the graph structure data.

7. The method according to claim 6, characterized in that The determining the defect type and defect cause of the glass substrate through the defect nodes and corresponding connection edges in the graph structure data includes: The refractive index disturbance value, spectral data and connection edge weight of the defect node are subjected to data preprocessing, and a defect node feature vector is determined according to the refractive index disturbance value and spectral data of the defect node subjected to data preprocessing, and a connection edge weight feature vector is determined according to the connection edge weight subjected to data preprocessing; The defect node feature vector and the connection edge weight feature vector are input into a glass substrate recognition model to obtain the defect type and defect cause of the glass substrate.

8. A glass substrate defect detection device, characterized in that: The device is used to perform the defect detection method for a glass substrate used in a display panel according to any one of claims 1 to 7, and the device comprises: An acquisition module, used to set the light source parameters of the multi-angle polarized light source array, so as to acquire multi-angle polarized light field data I (θ, λ, P, x, y) through the high-speed polarization camera; wherein θ is the incident angle, λ is the wavelength, P is the polarization state, and (x, y) is the position coordinate; A generating module, used for generating a three-dimensional refractive index anomaly map according to the multi-angle polarized light field data I (θ, λ, P, x, y); the three-dimensional refractive index anomaly map includes a refractive index disturbance value corresponding to each voxel; A determination module, used to determine the coordinates of the abnormal position through the refractive index disturbance value in the three-dimensional refractive index abnormality map, and to intercept the abnormal area image from the two-dimensional spectral image acquired by the hyperspectral imaging camera according to the abnormal position coordinates; the two-dimensional spectral image includes spectral data of each position point; A mapping module, used for mapping the spectral data in the abnormal region image to the three-dimensional refractive index abnormality map through coordinate transformation; The prediction module is used to determine the defect type and defect cause of the glass substrate according to the refractive index disturbance value in the three-dimensional refractive index anomaly map mapped with the spectral data and the spectral data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the defect detection method for a glass substrate inside a display panel is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the defect detection method for a glass substrate inside a display panel according to any one of claims 1 to 7 is implemented.

Images