Vision language model-based training-free multi-light-field substrate glass defect detection method and system

Abstract

This invention relates to the field of glass defect detection technology, specifically to a training-free multi-field substrate glass defect detection method and system based on a visual language model. The method includes: 1. Designing bright-field and dark-field cue words and establishing a text encoder based on a visual language model, inputting the bright-field and dark-field cue words into the text encoder to obtain bright-field text encoding features and dark-field text encoding features respectively; 2. Inputting bright-field and dark-field images into a dual-branch network established based on the visual language model to obtain multi-level bright-field and dark-field image features; obtaining multi-modal, multi-level defect features containing Gaussian noise based on the bright-field text encoding features, dark-field text encoding features, and multi-level bright-field and dark-field image features; 3. Denoising the multi-modal, multi-level defect images using structural sparsity and low-rank priors to obtain the final substrate glass defect image. This invention is used for glass defect detection, and the constructed model requires no training and has strong generalization ability.

Description

Technical Field

[0001] This invention relates to the field of glass defect detection technology, and in particular to a training-free multi-field substrate glass defect detection method and system based on a visual language model. Background Technology

[0002] As a core material in the electronics and information industry, substrate glass is widely used in liquid crystal displays, photovoltaic modules, touch panels, and other fields. Its surface quality directly affects the performance and reliability of end products. Therefore, high-precision detection of defects in substrate glass has become a critical step in the industrial manufacturing process. Due to the characteristics of substrate glass such as high light transmittance, high flatness, and thinness, its defects are usually small, complex in shape, and unevenly distributed, posing a great challenge to automated inspection.

[0003] Currently, defect detection systems for substrate glass typically employ composite lighting with transmitted bright-field and dark-field light sources to enhance the imaging contrast of defects, thereby acquiring multi-field image data with multi-view characteristics. This multi-field imaging method can effectively capture the response characteristics of different defects under different lighting conditions, providing a rich information foundation for subsequent inspection. Traditional glass defect detection methods rely heavily on image processing techniques such as edge detection, template matching, and projection analysis. These methods have good adaptability and interpretability for specific scenarios, but they also heavily depend on manually set rules and prior knowledge, exhibiting poor robustness and limited generalization ability when facing complex backgrounds, weak signal interference, or unknown defects. In recent years, deep learning-based image recognition technology has been gradually introduced into glass defect detection, using large-scale labeled data to train detection models, thus improving the accuracy of identifying complex defects. However, in practical industrial applications, this type of method has significant limitations: on the one hand, the substrate glass is affected by many factors such as process parameters, equipment status and environmental disturbances during the production process, which can easily lead to drastic changes in defect morphology and distribution; on the other hand, the samples used for training the detection system are difficult to cover all possible defect types and working condition changes, resulting in inconsistent data distribution between the training and testing phases, which seriously affects the generalization ability of the model.

[0004] In summary, existing methods generally suffer from performance degradation when faced with dynamically changing industrial environments and unknown defect types, making it difficult to meet the practical needs of high reliability and high flexibility in defect detection. There is an urgent need to explore training-free detection methods with generalization capabilities. Summary of the Invention

[0005] This invention provides a training-free multi-field substrate glass defect detection method and system based on a visual language model to solve the technical problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0007] This invention provides a training-free multi-field substrate glass defect detection method based on a visual language model, comprising the following steps:

[0008] S1. Design bright-field and dark-field cue words, and build a text encoder based on the visual language model. Input the bright-field and dark-field cue words into the text encoder respectively to obtain the bright-field text encoding features and the dark-field text encoding features.

[0009] S2. Input the bright field and dark field images into a dual-branch network based on a visual language model to obtain multi-level bright field and dark field image features; obtain multi-modal and multi-level defect features containing Gaussian noise based on the bright field text encoding features, dark field text encoding features, and multi-level bright field and dark field image features.

[0010] S3. Denoising of multimodal and multilevel defect features is performed using structural sparsity and low-rank priors to obtain the final substrate glass defect image.

[0011] Furthermore, step S1 specifically includes the following steps:

[0012] S11. For substrate glass inspection, describe the bright field and dark field images respectively to obtain bright field and dark field prompt words;

[0013] S12. Select the Transformer network as the visual language model, and then build a text encoder based on the Transformer network;

[0014] S13. Input the bright field and dark field cue words into the text encoder respectively. The text encoder extracts the text features within the bright field and dark field cue words to obtain the bright field text encoding features and the dark field text encoding features.

[0015] Furthermore, step S2 specifically includes the following steps:

[0016] S21. Establish a dual-branch network based on a visual language model;

[0017] S22. Composite lighting is applied to the substrate glass using transmitted bright field and transmitted dark field light sources to obtain bright field and dark field images.

[0018] S23, the bright-field image and the dark-field image are respectively input into the dual-branch network to obtain multi-level bright-field image features. and multi-layered dark field image features ;

[0019] S24. Features of multi-level brightfield images A weighted summation is performed, and then matrix multiplication is carried out with the bright-field text-coded features to obtain the cosine similarity between the multi-level bright-field image features and the bright-field text-coded features. ;

[0020] S25. Features of multi-layered dark-field images A weighted sum is performed, and then matrix multiplication is carried out with the dark field text encoding features to obtain the cosine similarity between the multi-level dark field image features and the dark field text encoding features. ;

[0021] S26. Cosine similarity Similarity to cosine By stacking along the feature dimensions, noisy multimodal multilevel features are obtained. ;

[0022] S27. Noisy multimodal and multilevel features Scaling the spatial dimension yields multimodal, multi-level defect features containing Gaussian noise. .

[0023] Furthermore, the dual-branch network includes two image encoders, and each image encoder contains 24 layers.

[0024] Furthermore, step S3 specifically includes the following steps:

[0025] S31. Based on structural sparsity and low-rank priors, characteristics of multimodal and multi-level defects. Decomposition was performed to obtain multimodal and multi-level defect features. The factorization of is as follows:

[0026] ;

[0027] Where N represents Gaussian noise; X represents multi-scale features, and the spatial dimension of multi-scale feature X is multi-modal and multi-level defect features. The spatial dimensions are the same;

[0028] Structural sparsity refers to the application of organized sparsity constraints on the feature map dimensions of multiple layers in deep neural networks. Sparsity constraints require that feature maps of multiple layers mostly exhibit a zero-value distribution, and that features are only allowed to remain non-zero when important information shared across layers is activated. Low-rank prior refers to the low-rank matrix represented by the feature map, that is, information with a proportion higher than a first set ratio can be expressed by principal components with a proportion lower than a second set ratio.

[0029] S32. Based on multimodal and multi-level defect characteristics The decomposition forms the first optimization objective, which estimates the optimal value using the maximum a posteriori probability optimization method. ,in Represents the multi-scale features of the prediction;

[0030] S33. Based on the monotonically increasing property of the logarithmic function, the first optimization objective is transformed into a second optimization objective, which is to minimize the negative logarithm.

[0031] S34. Based on the structural sparsity and low-rank prior, the second optimization objective is transformed to obtain the optimization task;

[0032] S35. Decouple the optimization task into a sparse problem and a low-rank term problem. A sparse problem refers to a situation in the spatial domain where the number of zero or near-zero elements in the defect is much greater than the number of non-zero elements. A low-rank term problem refers to a situation in the spatial domain where the background is structurally highly similar and is of low rank.

[0033] S36. Solve the sparse problem using the Vector Soft Thresholding (VST) algorithm, and then use the Vector SVD algorithm to calculate the low-rank term problem based on the calculation results of the sparse problem, to obtain the final substrate glass defect prediction map. .

[0034] Furthermore, the multi-scale features predicted in S32 The formula for calculation is:

[0035] ;

[0036] in, For data fidelity items; Indicates prior terms; This refers to finding the value of the multi-scale feature X that maximizes the corresponding function. The corresponding function in the calculation formula is .

[0037] Furthermore, the expression for the second optimization objective in S33 is as follows:

[0038] ;

[0039] in, This indicates the search for the value of the multi-scale feature X that minimizes the corresponding function on the multi-scale feature X. The expression for the second optimization objective contains the corresponding function as follows: .

[0040] Furthermore, the expression for optimizing the task in S34 is as follows:

[0041] ;

[0042] in, It is a balancing factor; Denotes the square of the F-norm; This means taking the 2-norm of each column of the matrix and then summing the 1-norms of these norms. This indicates the tensor expansion operation. This indicates taking the nuclear norm.

[0043] Furthermore, the expression for the sparsity problem in S35 is:

[0044] ;

[0045] The expression for the low-rank term problem is:

[0046] ;

[0047] Where H represents the actual defect diagram; This represents the final predicted defect diagram for the substrate glass. This represents finding the value of the actual defect map H that minimizes the corresponding function on the actual defect map H. The corresponding function in the expression of the low-rank term problem is... ;

[0048] The specific calculation results for the sparse problem are as follows:

[0049] ;

[0050] in, Representing multimodal and multilevel defect characteristics The OK; This represents the L2 norm.

[0051] In another aspect, the present invention provides a training-free multi-field substrate glass defect detection system based on a visual language model, including a detection device that detects defects in the substrate glass according to a training-free multi-field substrate glass defect detection method.

[0052] The beneficial effects of this invention are:

[0053] This invention discloses a training-free multi-light field substrate glass defect detection method based on a visual language model. This method introduces a block-by-block attention mechanism (i.e., an image encoder) to replace global attention mapping and integrates low-rank optimization and structural sparsity to achieve feature map and bi-branch multi-light field complementarity. This method requires no training, and experiments verify its excellent detection performance in complex scenes. Attached Figure Description

[0054] Figure 1 This is a flowchart of the training-free multi-field substrate glass defect detection method of the present invention;

[0055] Figure 2This is a schematic diagram illustrating the comparative results of the comparative experiments in this embodiment of the invention. Detailed Implementation

[0056] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0057] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0058] Reference Figure 1 This application provides a training-free multi-field substrate glass defect detection method based on a visual language model, comprising the following steps:

[0059] S1. Design bright-field and dark-field cue words, and build a text encoder based on the visual language model. Input the bright-field and dark-field cue words into the text encoder respectively to obtain the bright-field text encoding features and the dark-field text encoding features.

[0060] S2. Input the bright field and dark field images into a dual-branch network based on a visual language model to obtain multi-level bright field and dark field image features; obtain multi-modal and multi-level defect features containing Gaussian noise based on the bright field text encoding features, dark field text encoding features, and multi-level bright field and dark field image features.

[0061] S3. Denoising of multimodal, multi-level defect features is performed using structural sparsity and low-rank priors to obtain the final substrate glass defect image. Structural sparsity emphasizes "a small number of salient features with structure" (such as edges and corners), while low-rank priors emphasize "redundancy and similarity of the overall image data" (such as repetitive textures and smooth regions).

[0062] In some embodiments, S1 specifically includes the following steps:

[0063] S11. For substrate glass inspection, describe the bright field and dark field images respectively, for example: A bright photo of a perfect glass; A bright image of an abnormal glass; A dark photo of a perfect glass; Dark image of an abnormal glass.

[0064] Then, based on the description, the bright field and dark field cue words are obtained;

[0065] S12. Select the Transformer network as the visual language model, and then build a text encoder based on the Transformer network;

[0066] Specifically, the Transformer network consists of 12 encoding layers (CLIPEncoderLayer), each of which comprises a self-attention module and a multilayer perceptron. The text sequence (and bright / dark field cues) (e.g., 'A bright / dark field image of a [state] glass') is first converted into a word vector matrix through a sub-word segmentation algorithm, and then superimposed with learnable positional embeddings with sinusoidal positional encoding characteristics to form a 384-dimensional contextual representation containing the [category CLS] flag. Each CLIPEncoderLayer adopts a standardized Transformer architecture, whose core components include: 1) a multi-head self-attention module (configured with 8 attention heads), which achieves context-aware semantic association modeling through a learnable QKV (Q is query, K is key, V is value) projection matrix; 2) a residual connection mechanism implemented after each sub-layer, which, together with Layer Normalization (LN), forms a stable gradient propagation path; 3) a feedforward network consisting of two cascaded fully connected layers (dimensionality expansion ratio of 4:1), using the GeLU activation function to enhance non-linear representation capabilities. The word vector matrix (i.e., the [CLS] flag) is encoded by CLIPEncoderLayer and then output through Layer Normalization.

[0067] S13. Input the bright field and dark field cue words into the text encoder respectively. The text encoder extracts the text features within the bright field and dark field cue words to obtain the bright field text encoding features and the dark field text encoding features.

[0068] In some embodiments, S2 specifically includes the following steps:

[0069] S21. Establish a dual-branch network based on a visual language model;

[0070] S22. Composite lighting is applied to the substrate glass using transmitted bright field and transmitted dark field light sources to obtain bright field and dark field images.

[0071] S23, the bright-field image and the dark-field image are respectively input into the dual-branch network to obtain multi-level bright-field image features. (i=6,12,18,24) and multi-layered dark field image features (i=6,12,18,24); where R Represents the set of real numbers; H, W, C These represent the length, width, and number of channels of the feature map, respectively. By extracting image features from multiple layers, more information can be fused across layers. The receptive field of low-level networks is smaller and more suitable for extracting subtle defects, while the receptive field of high-level networks is larger and more suitable for extracting large-scale defects.

[0072] S24. Features of multi-level brightfield images A weighted summation is performed, and then matrix multiplication is carried out with the bright-field text-coded features to obtain the cosine similarity between the multi-level bright-field image features and the bright-field text-coded features. ;

[0073] S25. Features of multi-layered dark-field images A weighted sum is performed, and then matrix multiplication is carried out with the dark field text encoding features to obtain the cosine similarity between the multi-level dark field image features and the dark field text encoding features. ;

[0074] S26. Cosine similarity Similarity to cosine By stacking along the feature dimensions, noisy multimodal multilevel features are obtained. ;

[0075] S27. Noisy multimodal and multilevel features Scaling the spatial dimension yields multimodal, multi-level defect features containing Gaussian noise. .

[0076] In some embodiments, the dual-branch network includes two image encoders (i.e. Figure 1 The self-attention recovery mechanism in the image encoder is used, and each image encoder contains 24 layers.

[0077] Specifically, the image encoding path in the original CLIP model is constructed based on ViT (an image recognition model based on the Transformer architecture), which contains a self-attention module. Traditional self-attention modules use a query matrix. AND key matrix The attention matrix is ​​generated by the cosine distance and then multiplied with the value matrix V to obtain VI. However, this loss of semantic association is very detrimental to dense prediction tasks (such as anomaly segmentation); and the original CLIP (Contrastive Language-Image Pre-training) model learns a global label, which is usually used to capture overall information suitable for classification tasks. This loss of semantic association is very detrimental to dense prediction tasks (such as anomaly segmentation). Therefore, this invention designs an autocorrelation recovery module to replace the self-attention module mentioned above, making it more suitable for dense prediction tasks. The proposed autocorrelation recovery module is used to enhance the attention within image patches. Specifically, we use cosine similarity (i.e., cosine similarity) Similarity to cosine This determines the autocorrelation between the query and the key, ultimately yielding the bright-field or dark-field image encoding features. Qualitative and quantitative results demonstrate that this novel mechanism recovers semantic associations, providing a better foundation for subsequent attention map fusion. The calculation process of the autocorrelation recovery module is expressed by the following formula:

[0078] ;

[0079] in, Indicates linear projection. This indicates the number of heads in the bullish attention. Representing the query matrix , Represents the key matrix ( ), Represents the value matrix ( ); Indicates the i-th attention head in the key matrix The projection on This indicates that the i-th attention head is in the query matrix. Projection on This represents the projection of the i-th attention head onto the value matrix v. This represents the projection matrix of the i-th attention head query matrix; This represents the projection matrix of the i-th attention head key matrix; This represents the query matrix for the i-th attention head. This represents the key matrix of the i-th attention head. This represents the value matrix of the i-th attention head; This represents the weight of multi-head attention.

[0080] In some embodiments, S3 specifically includes the following steps:

[0081] S31. Based on structural sparsity and low-rank priors, characteristics of multimodal and multi-level defects. Decomposition was performed to obtain multimodal and multi-level defect features. The factorization of is as follows:

[0082] ;

[0083] Where N represents Gaussian noise; X represents the sparse multi-scale features, and the multi-scale features... That is, the spatial dimension of multi-scale feature X and the multi-modal, multi-level defect features. The spatial dimensions are the same;

[0084] Structural sparsity refers to the application of organized sparsity constraints across multiple layers of feature maps in deep neural networks, thereby improving the compression ratio and generalization ability of detection models. The detection model used in this invention includes a text encoder and a two-branch network.

[0085] The sparsity constraint requires that feature maps across multiple layers exhibit a zero-value distribution as much as possible. Features are only allowed to remain non-zero when important information shared across layers is activated, in order to avoid unnecessary redundancy.

[0086] Low-rank prior refers to a low-rank matrix represented by the feature map, meaning that most information can be effectively expressed through a small number of principal components. This prior helps to utilize the redundancy between data to reconstruct missing information, thereby enhancing the stability and representational power of the detection model.

[0087] S32. Based on multimodal and multi-level defect characteristics The decomposition forms the first optimization objective, which estimates the optimal value using the maximum a posteriori probability optimization method. ,in This represents the predicted multi-scale features; the specific calculation formula is as follows:

[0088] ;

[0089] in, For data fidelity term; P(X) represents the prior term, which is used to capture specific characteristics of the feature. This refers to finding the value of the multi-scale feature X that maximizes the corresponding function. The corresponding function in the calculation formula is The purpose of data fidelity is to ensure that the solution is consistent with the original data or the degradation process.

[0090] S33. Based on the monotonically increasing property of the logarithmic function, the first optimization objective is transformed into a second optimization objective, which is to minimize the negative logarithm. The specific expression is as follows:

[0091] ;

[0092] in, This indicates the search for the value of the multi-scale feature X that minimizes the corresponding function on the multi-scale feature X. The expression for the second optimization objective contains the corresponding function as follows: ;

[0093] S34, due to prior terms The second optimization objective is used to capture specific characteristics such as sparsity and low rank. These characteristics are represented by structural sparsity priors and low rank priors. Therefore, the second optimization objective can be transformed based on structural sparsity and low rank priors to obtain the optimization task. The specific expression of the optimization task is as follows:

[0094] ;

[0095] in, It is a balancing factor; Denotes the square of the F-norm. This means taking the 2-norm of each column of the matrix and then summing the 1-norms of these norms. This indicates the tensor expansion operation. Indicates taking the nuclear norm;

[0096] S35. Decouple the optimization task into a sparse problem and a low-rank term problem. A sparse problem refers to a situation in the spatial domain where the number of zero or near-zero elements in the defect is much greater than the number of non-zero elements. A low-rank term problem refers to a situation in the spatial domain where the background is structurally highly similar and is of low rank.

[0097] The expression for the sparsity problem is:

[0098] ;

[0099] The expression for the low-rank term problem is:

[0100] ;

[0101] Where H represents the actual defect diagram; This represents the final predicted defect diagram for the substrate glass. This represents finding the value of the actual defect map H that minimizes the corresponding function on the actual defect map H. The corresponding function in the expression of the low-rank term problem is... ;

[0102] S36. Solve the sparse problem using the Singular Value Decomposition (VST) algorithm, and then use the Vector Soft Thresholding (SVD) algorithm to calculate the low-rank term problem based on the calculation results of the sparse problem, thus obtaining the final substrate glass defect prediction map. The specific calculation results for the sparse problem are as follows:

[0103] ;

[0104] in, Representing multimodal and multilevel defect characteristics The OK; This represents the L2 norm.

[0105] In some embodiments, the training-free multi-field substrate glass defect detection method further includes, after step S3:

[0106] S4. Composite lighting is achieved using transmitted bright-field and transmitted dark-field light sources to obtain multi-viewpoint image data of the substrate glass, and a dataset is constructed for comparison with other defect detection methods. To evaluate the performance of the proposed method, this invention follows previous work on zero-shot anomaly segmentation, using four metrics—pAUROC (average area under the receiver characteristic curve), F1-MAX (maximum F1 score), AP (average accuracy), and PRO (regional overlap rate)—to verify the superiority of the proposed method. See Table 1 for details. Figure 2 :

[0107] Table 1: Performance Comparison of Different Methods. Higher values ​​for pAUROC, F1-MAX, AP, and PRO indicate better detection performance. Compared to traditional methods (such as Gaussian mixture models, edge detection, and quantile interval methods), this invention not only more accurately highlights defect regions but also demonstrates stronger anti-interference capabilities. For CLIP-based methods, results show that retraining can significantly improve performance. Among them, APRIL-GAN, AnomalyCLIP, and AdaCLIP outperform WinCLIP and CLIP-AD. However, these retraining-based methods still face challenges in accurate segmentation due to their reliance on prior information. For example, APRIL-GAN highlights regions that are too small, while AdaCLIP often generates regions that are too large. As shown in Table 1, this invention performs excellently across all four metrics: average pAUROC reaches 99.7, F1-MAX is 51.1, AP is 45.8, and PRO is 99.6. Compared to the best-performing traditional method, Percentile Range, TFMAS achieves improvements of 1.5, 11.5, 21.5, and 1.1 in pAUROC, F1-MAX, AP, and PRO, respectively. Compared to retraining methods such as APRIL-GAN, AnomalyCLIP, and AdaCLIP, this invention also demonstrates superior performance: improvements of 1.8, 0.7, and 0.2 in pAUROC, 23.9, 28.9, and 3.7 in F1-MAX, 24.5, 32.9, and 8.0 in AP, and 93.6, 12.9, and 0.6 in PRO.

[0108] Table 1: Experimental data comparing the performance of different methods;

[0109]

[0110] In another aspect, the present invention provides a training-free multi-field substrate glass defect detection system based on a visual language model, including a detection device that detects defects in the substrate glass according to a training-free multi-field substrate glass defect detection method.

[0111] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A training-free multi-field substrate glass defect detection method based on a visual language model, characterized in that, Includes the following steps: S1. Design bright-field and dark-field cue words, and build a text encoder based on the visual language model. Input the bright-field and dark-field cue words into the text encoder respectively to obtain the bright-field text encoding features and the dark-field text encoding features. S2. Input the bright field and dark field images into a dual-branch network based on a visual language model to obtain multi-level bright field and dark field image features; obtain multi-modal and multi-level defect features containing Gaussian noise based on the bright field text encoding features, dark field text encoding features, and multi-level bright field and dark field image features. S3. Denoising of multimodal and multilevel defect features is performed using structural sparsity and low-rank priors to obtain the final substrate glass defect image. S2 specifically includes the following steps: S21. Establish a dual-branch network based on a visual language model; S22. Composite lighting is applied to the substrate glass using transmitted bright field and transmitted dark field light sources to obtain bright field and dark field images. S23, the bright-field image and the dark-field image are respectively input into the dual-branch network to obtain multi-level bright-field image features. and multi-layered dark field image features ; S24. Features of multi-level brightfield images A weighted summation is performed, and then matrix multiplication is carried out with the bright-field text-coded features to obtain the cosine similarity between the multi-level bright-field image features and the bright-field text-coded features. ; S25. Features of multi-layered dark-field images A weighted sum is performed, and then matrix multiplication is carried out with the dark field text encoding features to obtain the cosine similarity between the multi-level dark field image features and the dark field text encoding features. ; S26. Cosine similarity Similarity with cosine By stacking along the feature dimensions, noisy multimodal multilevel features are obtained. ; S27. Noisy multimodal and multilevel features Scaling the spatial dimension yields multimodal, multi-level defect features containing Gaussian noise. ; S3 specifically includes the following steps: S31. Based on structural sparsity and low-rank priors, characteristics of multimodal and multi-level defects. Decomposition was performed to obtain multimodal and multi-level defect features. The factorization of is as follows: ； Where N represents Gaussian noise; X represents multi-scale features, and the spatial dimension of multi-scale feature X is multi-modal and multi-level defect features. The spatial dimensions are the same; Structural sparsity refers to the application of organized sparsity constraints on the feature maps of multiple layers in deep networks. Sparsity constraints mean that the feature maps of multiple layers need to present a distribution of zero values, and features are only allowed to remain non-zero when important information shared across layers is activated. Low-rank prior refers to the low-rank matrix represented by the feature map, that is, information with a proportion higher than a first set ratio can be expressed by principal components with a proportion lower than a second set ratio. S32. Based on multimodal and multi-level defect characteristics The decomposition forms the first optimization objective, which estimates the optimal value using the maximum a posteriori probability optimization method. ,in Represents the multi-scale features of the prediction; S33. Based on the monotonically increasing property of the logarithmic function, the first optimization objective is transformed into a second optimization objective, which is to minimize the negative logarithm. S34. Based on the structural sparsity and low-rank prior, the second optimization objective is transformed to obtain the optimization task; S35. Decouple the optimization task into a sparse problem and a low-rank term problem; a sparse problem refers to a situation in the spatial domain where there are more zero or near-zero elements than non-zero elements in the defect; a low-rank term problem refers to a situation in the spatial domain where the background is structurally similar and is low-rank. S36. Solve the sparse problem using the Vector Soft Thresholding (VST) algorithm, and then use the Vector SVD algorithm to calculate the low-rank term problem based on the calculation results of the sparse problem, to obtain the final substrate glass defect prediction map. .

2. The method for detecting defects in multi-field substrate glass without training based on a visual language model according to claim 1, characterized in that, S1 specifically includes the following steps: S11. For substrate glass inspection, describe the bright field and dark field images respectively to obtain bright field and dark field prompt words; S12. Select the Transformer network as the visual language model, and then build a text encoder based on the Transformer network; S13. Input the bright field and dark field cue words into the text encoder respectively. The text encoder extracts the text features within the bright field and dark field cue words to obtain the bright field text encoding features and the dark field text encoding features.

3. The method for detecting defects in multi-field substrate glass without training based on a visual language model according to claim 1, characterized in that, The dual-branch network includes two image encoders, and each image encoder contains 24 layers.

4. The method for detecting defects in multi-field substrate glass without training based on a visual language model according to claim 1, characterized in that, The multi-scale features predicted in S32 The formula for calculation is: ； in, For data fidelity items; Indicates prior terms; This refers to finding the value of the multi-scale feature X that maximizes the corresponding function. The corresponding function in the calculation formula is .

5. The method for detecting defects in multi-field substrate glass without training based on a visual language model according to claim 4, characterized in that, The expression for the second optimization objective in S33 is as follows: ； in, This indicates the search for the value of the multi-scale feature X that minimizes the corresponding function on the multi-scale feature X. The expression for the second optimization objective contains the corresponding function as follows: .

6. The method for detecting defects in multi-field substrate glass without training based on a visual language model according to claim 5, characterized in that, The specific expression for the optimization task in S34 is as follows: ； in, It is a balancing factor; Denotes the square of the F-norm. This means taking the 2-norm of each column of the matrix and then summing the 1-norms of these norms. This indicates the tensor expansion operation. This indicates taking the nuclear norm.

7. The method for detecting defects in multi-field substrate glass without training based on a visual language model according to claim 6, characterized in that, The expression for the sparsity problem in S35 is: ； The expression for the low-rank term problem is: ； Where H represents the actual defect diagram; This represents the final predicted defect diagram for the substrate glass. This represents finding the value of the actual defect map H that minimizes the corresponding function on the actual defect map H. The corresponding function in the expression of the low-rank term problem is... ; The specific calculation results for the sparse problem are as follows: ； in, Representing multimodal, multi-level defect characteristics The OK; This represents the L2 norm.

8. A training-free multi-field substrate glass defect detection system based on a visual language model, characterized in that, The invention includes a testing device, which detects defects in the substrate glass according to the training-free multi-field substrate glass defect detection method according to any one of claims 1 to 7.

Images