Glass substrate defect real-time detection method and system based on double-branch features

Abstract

This application relates to a method and system for real-time defect detection of glass substrates based on bi-branch features. The method includes: acquiring a glass substrate image from an industrial control computer and preprocessing it to obtain a preprocessed image; transmitting the preprocessed image to a cloud server, constructing a feature matching model, and deploying the feature matching model on the cloud server; performing bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and training the feature matching model to obtain a real-time defect detection model for the glass substrate; and performing real-time defect detection of the glass substrate based on the real-time defect detection model. This method extracts coarse and fine features of the image through a feature matching model, which is more robust than previous methods using only VLM or CNN feature networks, thus effectively improving detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of image detection, and more specifically, to a method and system for real-time detection of defects in glass substrates based on bi-branch features. Background Technology

[0002] Glass substrates are key packaging materials for high-performance chips. During the production of glass substrates, extremely subtle and difficult-to-detect defects are inevitably generated, significantly impacting the substrate's production yield. Traditional manual visual inspection is not only inefficient (requiring dozens of inspectors per production line) but also susceptible to subjective experience and visual fatigue, making quantitative analysis impossible. Traditional image-based defect detection is greatly affected by complex backgrounds, minute or complex defect morphologies, and lighting conditions, leading to frequent misjudgments or missed detections. Feature extraction relies on manual design and experience, resulting in low development efficiency and a lack of flexibility. In large-scale data or high-resolution image processing, computational complexity is high, speed is slow, and it struggles to meet real-time detection requirements. Furthermore, its robustness is insufficient, lacking self-learning and optimization mechanisms, making it difficult to adapt to changes in environment or defect type. Traditional image-based defect detection is completely inadequate for industrial needs. Therefore, developing automated, high-precision, and real-time intelligent defect detection technology for glass substrates has become a core demand for industry development. Summary of the Invention

[0003] Therefore, it is necessary to provide a high-precision real-time detection method and system for glass substrate defects based on bi-branch features to address the aforementioned technical problems.

[0004] The real-time defect detection method for glass substrates based on bi-branch features includes:

[0005] S1: Obtain the glass substrate image from the industrial control computer and perform preprocessing to obtain the preprocessed image; S2: Transmit the preprocessed image to the cloud server, build a feature matching model, and deploy the feature matching model on the cloud server. S3: The feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and trains the feature matching model to obtain a real-time detection model for glass substrate defects. S4: Real-time detection of defects in glass substrates is performed based on the real-time defect detection model for glass substrates.

[0006] Furthermore, in step S1, the industrial control computer specifically refers to: The industrial control computer hardware is precisely selected based on the characteristics of the glass substrate; the hardware includes a high-speed linear scan camera, a high-resolution optical lens, a directional dark field illumination source, a polarization control component, and a high-performance image processing unit. High-speed line scan camera: The high-frequency line scan camera is equipped with a high-precision rotary encoder to collect the displacement signal of the conveyor roller in real time, so as to realize the microsecond-level synchronous triggering of camera exposure and light source flash; High-performance image processing unit: Utilizes CoaXPress or Camera Link HS high-speed image interfaces to ensure real-time transmission of massive image data; Backend deployment of a multi-core CPU and GPU / FPGA collaborative processing architecture supports online image stitching, real-time filtering, and defect classification; High-resolution optical lens: 8K or 16K high-resolution line scan camera, combined with a telecentric lens or a dedicated FA lens with high modulation transfer function and low distortion; by accurately calculating the field of view, pixel resolution and Nyquist sampling criterion, it ensures that the physical size of a single pixel meets the detection accuracy requirements, and realizes full-frame seamless and blind-zone-free scanning; Directional dark field illumination source: LED light source illuminates the glass surface at an incident angle of 30-60 degrees, so that the incident light in the normal area cannot enter the imaging light path due to refraction or transmission, while the surface defects reflect some light to the camera sensor due to the scattering effect, thus forming high-contrast bright spots in the image and significantly improving the visibility of defects. Polarization control components: A polarizer is set at the output end of the directional dark field illumination source, and an analyzer is configured at the front end of the high-speed linear array camera lens to form a cross-polarized optical path.

[0007] Furthermore, in step S2, transmitting the preprocessed image to the cloud server specifically involves: By selecting gRPC as the communication framework, the preprocessed image is transmitted to the cloud server. gRPC is based on the HTTP / 2 protocol. The multiplexing feature of HTTP / 2 allows multiple requests and responses to be sent in parallel on a single TCP connection, completely avoiding the head-of-line blocking problem of HTTP / 1.1. gRPC uses Protocol Buffers as its interface definition language and default serialization tool. TLS / SSL encryption is enabled on gRPC channels to encrypt data in transit. On the cloud server side, the gRPC server program is deployed on a virtual machine with a public IP address or a dedicated line connection and listens on a specific port.

[0008] Furthermore, in step S2, deploying the feature matching model on the cloud server specifically involves: NVIDIA TensorRT is used for model optimization and deployment. The trained PyTorch model is converted into the platform-independent ONNX intermediate representation format. Then, the optimized inference engine is built on the target deployment device using NVIDIA TensorRT. Finally, the engine is loaded on the remote server to achieve low latency and high throughput real-time inference service.

[0009] Furthermore, in step S2, the feature matching model is specifically as follows: The module consists of a feature encoder, a global matcher, a multi-scale convolutional modifier, and a loss function module. In the feature encoding module, the frozen DINOv2 model is used as a coarse feature encoder to provide coarse features with strong generalization, and a dedicated VGG19 network is used to extract precise fine features, forming a robust and localizable feature pyramid. The global matcher module consists of a Gaussian process matching encoder and a Transformer matching decoder. The latter models the multimodal distribution by predicting anchor point probabilities and outputs a coarse warp. The multi-scale convolutional modifier iteratively optimizes the coarse warp based on fine features, gradually improving matching accuracy. The loss function module employs classification and regression loss for coarse-match multimodal characteristics and generalized Charbonnier robust loss for fine-correction single-modal characteristics, achieving accurate training.

[0010] Further, in step S3, the feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse features. Specifically, it calculates the covariance matrix between the features of image A and image B using a cosine similarity kernel function to model spatial smoothness constraints; it first performs Fourier basis encoding on the normalized grid coordinates of image B to generate position embeddings, and then propagates the position information of image B to image A according to feature similarity through Gaussian process regression to obtain soft position encodings; the soft position encodings are concatenated with the features of image A and input into a multi-layer Transformer decoder, which aggregates global context information through a self-attention mechanism, outputs a classification probability distribution, and converts the weighted sum of discrete coordinates into a continuous initial flow field.

[0011] Further, in step S3, the feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain the fine features, specifically: In the refinement stage, a convolutional neural network is used to progressively refine the flow field. At each scale, the ConvRefiner module fuses four types of complementary information: the original features of image A, matching features sampled from image B based on the current flow field, displacement embeddings that deviate from the identity mapping in the current prediction, and feature correlation maps computed within a local window around the prediction location. The window radius of the local correlation decreases with scale, focusing on an increasingly smaller search range to eliminate matching ambiguities. Discriminative representations are extracted through multiple depthwise separable convolutional blocks to predict incremental corrections and confidence changes in the flow field. The incremental correction strategy allows the network to learn only the residuals, accelerating convergence and improving accuracy. The updated flow field is bilinearly upsampled to the next scale for further refinement.

[0012] Furthermore, in step S3, the real-time defect detection model for glass substrates is trained by training the feature matching model as follows: During the training phase, the ground truth flow field is generated through 3D geometric projection using depth maps and camera parameters as a supervision signal. The loss function adopts a multi-scale joint supervision strategy: coarse matching uses cross-entropy classification loss and confidence binary cross-entropy loss, while the refinement phase uses generalized Charbonnier regression loss to enhance robustness to outliers. To focus on optimizing hard examples, local loss constraints are introduced starting from the 8th scale, and the loss is calculated only in regions where the prediction error of the previous scale is less than the threshold, avoiding repeated optimization in simple regions. For high-precision scenarios, a super-resolution refinement mode is supported, which upsamples the input image to a high resolution, skips the expensive DINOv2 computation with coarse-scale prediction as initialization, and repeats the process only in the refinement phase to obtain denser matching results.

[0013] Further, in step S4, specifically: The real-time defect detection model for glass substrates uses two images as paired inputs: a template image and the current frame image. Internally, the model first performs multi-scale feature extraction on both input images. After feature extraction, it further performs cross-image correlation calculations on the template features and the current frame features. By calculating the similarity between features at different spatial locations in the feature space, the model can find potential corresponding positions for each pixel in the template image in the current frame. Based on the above cross-image association results, a coarse-to-fine inference strategy is adopted to generate dense pixel mapping output step by step: first, a coarse correspondence between the template image and the current frame is predicted at a lower spatial resolution; then, through step-by-step refinement and upsampling operations, the mapping result is restored to the target resolution, thereby obtaining a dense pixel mapping field covering the entire image; the mapping field is used to describe the corresponding position of each pixel in the template image in the current frame; while generating the dense pixel mapping, a corresponding matching confidence score is also output for each pixel mapping relationship. The dense matching results are parsed and structured to generate the location, confidence level, and category information of defects. The dense pixel mapping relationship and pixel-by-pixel matching confidence level serve as input data for the post-processing stage. During the defect location generation process, the pixel mapping relationship output by the model is used to migrate the pre-stored defect spatial annotation regions in the template image to the current frame. Specifically, the defect annotation regions in the template image can be represented as one or more defect region sets. By querying the pixel mapping relationship, the pixels within the template defect region can obtain their corresponding spatial locations in the current frame, thereby forming a set of candidate pixels related to defects in the current frame. Spatial aggregation is performed on the aforementioned candidate pixel set. By statistically analyzing their coordinate distribution in the current frame, the minimum bounding rectangle of the candidate pixel set is calculated, and this rectangle is used as the initial location box of the defect in the current frame. This processing method transforms the continuous dense pixel mapping results into a structured bounding box representation. During the confidence calculation, based on pixel-by-pixel matching confidence, region-level fusion is performed on the confidence information within the defect location box. In the final result generation stage, the defect location box, defect location confidence, and defect category and classification confidence are comprehensively processed to form a unified defect detection result; the location confidence and classification confidence are weighted and fused to generate a comprehensive confidence index that characterizes the overall reliability of the defect detection result. Finally, the defect detection results are output in the form of structured data, including defect category, defect location bounding box coordinates and corresponding confidence information, and are returned to the industrial control computer through the communication interface.

[0014] Furthermore, this invention provides a real-time defect detection system for glass substrates based on dual-branch features, comprising: Acquisition module: Acquires the glass substrate image from the industrial control computer and performs preprocessing to obtain the preprocessed image; Preprocessing module: Transmits the preprocessed image to the cloud server, constructs a feature matching model, and deploys the feature matching model on the cloud server; Model module: The feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and trains the feature matching model to obtain a real-time defect detection model for glass substrates; Detection module: Real-time detection of defects in glass substrates based on a real-time defect detection model for glass substrates.

[0015] This invention extracts both coarse and fine features from images using a feature matching model, which is more robust than previous methods that only used VLM or CNN feature networks. This effectively improves detection accuracy. Attached Figure Description

[0016] Figure 1 This is a flowchart of a real-time defect detection method for glass substrates based on bi-branch features in one embodiment; Figure 2 This is a block diagram of a real-time defect detection system for glass substrates based on bi-branch features in one embodiment; Figure 3 This is a schematic diagram of the real-time detection process in one embodiment; Figure 4 This is a schematic diagram of a dual-branch dense feature matching model architecture in one embodiment; Figure 5 This is a schematic diagram of the gRPC call flow in one embodiment; Detailed Implementation To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0017] Example 1: The real-time defect detection method for glass substrates based on bi-branch features includes: S1: Obtain the glass substrate image from the industrial control computer and perform preprocessing to obtain the preprocessed image; S2: Transmit the preprocessed image to the cloud server, build a feature matching model, and deploy the feature matching model on the cloud server. S3: The feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and trains the feature matching model to obtain a real-time detection model for glass substrate defects. S4: Real-time detection of defects in glass substrates is performed based on the real-time defect detection model for glass substrates.

[0018] This embodiment extracts both coarse and fine features from an image using a feature matching model, which is more robust than previous methods that only used VLM or CNN feature networks. This effectively improves detection accuracy.

[0019] Example 2: This embodiment further discloses information based on Embodiment 1: Furthermore, in step S1, the industrial control computer specifically refers to: The industrial control computer hardware is precisely selected based on the characteristics of the glass substrate; the hardware includes a high-speed linear scan camera, a high-resolution optical lens, a directional dark field illumination source, a polarization control component, and a high-performance image processing unit. High-speed line scan camera: The high-frequency line scan camera is equipped with a high-precision rotary encoder to collect the displacement signal of the conveyor roller in real time, so as to realize the microsecond-level synchronous triggering of camera exposure and light source flash; High-performance image processing unit: Utilizes CoaXPress or Camera Link HS high-speed image interfaces to ensure real-time transmission of massive image data; Backend deployment of a multi-core CPU and GPU / FPGA collaborative processing architecture supports online image stitching, real-time filtering, and defect classification; High-resolution optical lens: 8K or 16K high-resolution line scan camera, combined with a telecentric lens or a dedicated FA lens with high modulation transfer function and low distortion; by accurately calculating the field of view, pixel resolution and Nyquist sampling criterion, it ensures that the physical size of a single pixel meets the detection accuracy requirements, and realizes full-frame seamless and blind-zone-free scanning; Directional dark field illumination source: LED light source illuminates the glass surface at an incident angle of 30-60 degrees, so that the incident light in the normal area cannot enter the imaging light path due to refraction or transmission, while the surface defects reflect some light to the camera sensor due to the scattering effect, thus forming high-contrast bright spots in the image and significantly improving the visibility of defects. Polarization control components: A polarizer is set at the output end of the directional dark field illumination source, and an analyzer is configured at the front end of the high-speed linear array camera lens to form a cross-polarized optical path.

[0020] Furthermore, in step S2, transmitting the preprocessed image to the cloud server specifically involves: By selecting gRPC as the communication framework, the preprocessed image is transmitted to the cloud server. gRPC is based on the HTTP / 2 protocol. The multiplexing feature of HTTP / 2 allows multiple requests and responses to be sent in parallel on a single TCP connection, completely avoiding the head-of-line blocking problem of HTTP / 1.1. gRPC uses Protocol Buffers as its interface definition language and default serialization tool. TLS / SSL encryption is enabled on gRPC channels to encrypt data in transit. On the cloud server side, the gRPC server program is deployed on a virtual machine with a public IP address or a dedicated line connection and listens on a specific port.

[0021] Furthermore, in step S2, deploying the feature matching model on the cloud server specifically involves: NVIDIA TensorRT is used for model optimization and deployment. The trained PyTorch model is converted into the platform-independent ONNX intermediate representation format. Then, the optimized inference engine is built on the target deployment device using NVIDIA TensorRT. Finally, the engine is loaded on the remote server to achieve low latency and high throughput real-time inference service.

[0022] Furthermore, in step S2, the feature matching model is specifically as follows: The module consists of a feature encoder, a global matcher, a multi-scale convolutional modifier, and a loss function module. In the feature encoding module, the frozen DINOv2 model is used as a coarse feature encoder to provide coarse features with strong generalization, and a dedicated VGG19 network is used to extract precise fine features, forming a robust and localizable feature pyramid. The global matcher module consists of a Gaussian process matching encoder and a Transformer matching decoder. The latter models the multimodal distribution by predicting anchor point probabilities and outputs a coarse warp. The multi-scale convolutional modifier iteratively optimizes the coarse warp based on fine features, gradually improving matching accuracy. The loss function module employs classification and regression loss for coarse-match multimodal characteristics and generalized Charbonnier robust loss for fine-correction single-modal characteristics, achieving accurate training.

[0023] Further, in step S3, the feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse features. Specifically, it calculates the covariance matrix between the features of image A and image B using a cosine similarity kernel function to model spatial smoothness constraints; it first performs Fourier basis encoding on the normalized grid coordinates of image B to generate position embeddings, and then propagates the position information of image B to image A according to feature similarity through Gaussian process regression to obtain soft position encodings; the soft position encodings are concatenated with the features of image A and input into a multi-layer Transformer decoder, which aggregates global context information through a self-attention mechanism, outputs a classification probability distribution, and converts the weighted sum of discrete coordinates into a continuous initial flow field.

[0024] Further, in step S3, the feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain the fine features, specifically: In the refinement stage, a convolutional neural network is used to progressively refine the flow field. At each scale, the ConvRefiner module fuses four types of complementary information: the original features of image A, matching features sampled from image B based on the current flow field, displacement embeddings that deviate from the identity mapping in the current prediction, and feature correlation maps computed within a local window around the prediction location. The window radius of the local correlation decreases with scale, focusing on an increasingly smaller search range to eliminate matching ambiguities. Discriminative representations are extracted through multiple depthwise separable convolutional blocks to predict incremental corrections and confidence changes in the flow field. The incremental correction strategy allows the network to learn only the residuals, accelerating convergence and improving accuracy. The updated flow field is bilinearly upsampled to the next scale for further refinement.

[0025] Furthermore, in step S3, the real-time defect detection model for glass substrates is trained by training the feature matching model as follows: During the training phase, the ground truth flow field is generated through 3D geometric projection using depth maps and camera parameters as a supervision signal. The loss function adopts a multi-scale joint supervision strategy: coarse matching uses cross-entropy classification loss and confidence binary cross-entropy loss, while the refinement phase uses generalized Charbonnier regression loss to enhance robustness to outliers. To focus on optimizing hard examples, local loss constraints are introduced starting from the 8th scale, and the loss is calculated only in regions where the prediction error of the previous scale is less than the threshold, avoiding repeated optimization in simple regions. For high-precision scenarios, a super-resolution refinement mode is supported, which upsamples the input image to a high resolution, skips the expensive DINOv2 computation with coarse-scale prediction as initialization, and repeats the process only in the refinement phase to obtain denser matching results.

[0026] Further, in step S4, specifically: The real-time defect detection model for glass substrates uses two images as paired inputs: a template image and the current frame image. Internally, the model first performs multi-scale feature extraction on both input images. After feature extraction, it further performs cross-image correlation calculations on the template features and the current frame features. By calculating the similarity between features at different spatial locations in the feature space, the model can find potential corresponding positions for each pixel in the template image in the current frame. Based on the above cross-image association results, a coarse-to-fine inference strategy is adopted to generate dense pixel mapping output step by step: first, a coarse correspondence between the template image and the current frame is predicted at a lower spatial resolution; then, through step-by-step refinement and upsampling operations, the mapping result is restored to the target resolution, thereby obtaining a dense pixel mapping field covering the entire image; the mapping field is used to describe the corresponding position of each pixel in the template image in the current frame; while generating the dense pixel mapping, a corresponding matching confidence score is also output for each pixel mapping relationship. The dense matching results are parsed and structured to generate the location, confidence level, and category information of defects. The dense pixel mapping relationship and pixel-by-pixel matching confidence level serve as input data for the post-processing stage. During the defect location generation process, the pixel mapping relationship output by the model is used to migrate the pre-stored defect spatial annotation regions in the template image to the current frame. Specifically, the defect annotation regions in the template image can be represented as one or more defect region sets. By querying the pixel mapping relationship, the pixels within the template defect region can obtain their corresponding spatial locations in the current frame, thereby forming a set of candidate pixels related to defects in the current frame. Spatial aggregation is performed on the aforementioned candidate pixel set. By statistically analyzing their coordinate distribution in the current frame, the minimum bounding rectangle of the candidate pixel set is calculated, and this rectangle is used as the initial location box of the defect in the current frame. This processing method transforms the continuous dense pixel mapping results into a structured bounding box representation. During the confidence calculation, based on pixel-by-pixel matching confidence, region-level fusion is performed on the confidence information within the defect location box. In the final result generation stage, the defect location box, defect location confidence, and defect category and classification confidence are comprehensively processed to form a unified defect detection result; the location confidence and classification confidence are weighted and fused to generate a comprehensive confidence index that characterizes the overall reliability of the defect detection result. Finally, the defect detection results are output in the form of structured data, including defect category, defect location bounding box coordinates and corresponding confidence information, and are returned to the industrial control computer through the communication interface.

[0027] This embodiment extracts both coarse and fine features from an image using a feature matching model, which is more robust than previous methods that only used VLM or CNN feature networks. This effectively improves detection accuracy.

[0028] Example 3: Furthermore, this embodiment provides a real-time defect detection system for glass substrates based on dual-branch features, including: Acquisition module: Acquires the glass substrate image from the industrial control computer and performs preprocessing to obtain the preprocessed image; Preprocessing module: Transmits the preprocessed image to the cloud server, constructs a feature matching model, and deploys the feature matching model on the cloud server; Model module: The feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and trains the feature matching model to obtain a real-time defect detection model for glass substrates; Detection module: Real-time detection of defects in glass substrates based on a real-time defect detection model for glass substrates.

[0029] This embodiment extracts both coarse and fine features from an image using a feature matching model, which is more robust than previous methods that only used VLM or CNN feature networks. This effectively improves detection accuracy.

[0030] Example 4: This embodiment further discloses the following based on Embodiment 1: The overall process of this embodiment is as follows: Figure 3As shown in the figure, the feature matching model utilizes an encoder-decoder architecture to achieve pixel-level dense matching. The specific process consists of three stages: First, a hybrid feature encoder simultaneously extracts multi-scale local features and global semantic features of image pairs; second, an initial coarse match is established at the coarsest scale using a Gaussian process module, and the feature similarity is converted into a dense flow field using a Transformer decoder; finally, a multi-scale cascaded refinement strategy is adopted, upsampling the flow field from coarse to fine, fusing local correlation and displacement information at each scale for convolutional refinement, and finally outputting a high-resolution dense matching field and confidence map. Traditional image processing methods rely on manual feature extraction and parameter adjustment, which have poor adaptability to complex backgrounds, small defects, and changes in illumination, and lack robustness. We use a one-shot feature matching deep learning algorithm model for detection, which only requires calibrating a rectangular template and feeding it into a pre-trained feature matching model for probabilistic anchor point prediction, thereby achieving automated real-time detection of small defects on glass substrates. Specifically, the structural design revolves around the core logic of "coarse-fine combination," integrating pre-trained basic model features, dedicated network components, and optimized loss functions. The overall architecture can be broken down into four main modules: feature encoder module, global matcher module, warp corrector module, and loss function module. Each module has a clear division of labor and deep collaboration. In the feature encoding module, a frozen DINOv2 model is used as a coarse feature encoder to provide highly generalizable coarse features, which are then combined with a dedicated VGG19 network to extract precise fine features, forming a robust and localizable feature pyramid. The global matcher consists of a Gaussian process matching encoder and a Transformer matching decoder. The latter models multimodal distribution by predicting anchor point probabilities and outputs a coarse warp. The multi-scale convolutional corrector iteratively optimizes the coarse warp based on fine features, gradually improving matching accuracy. The loss function module uses classification regression loss for the multimodal characteristics of coarse matching and generalized Charbonnier robust loss for the single-modal characteristics of fine correction, achieving accurate training. The four modules each have their own focus and work together in a deep synergy: the feature encoder lays the foundation for "robust and accurate" features, the global matcher completes the initial correspondence estimation, the warp corrector achieves iterative improvement in accuracy, and the loss function module adapts to the distribution characteristics of different stages to ensure training effect. Together, they enable the feature matching model to maintain excellent matching performance even in extreme scenarios.

[0031] First, the camera captures images of objects on the conveyor belt and transmits them to the industrial PC. The industrial PC encodes the images and sends the encoded input to the inference server via the gRPC protocol. The inference server uses a one-shot feature matching large model equipped with multiple GPUs (GPU0-GPU7) to perform inference calculations and finally sends the inference results back to the industrial PC, thus realizing real-time online defect detection.

[0032] Example 5: This embodiment further discloses information based on Embodiment 1: The architecture design of the dual-branch image feature processing system used in this embodiment is as follows: Figure 4 As shown.

[0033] The hybrid encoder used in this embodiment employs a dual-branch design to accommodate feature representations at different levels. The feature branch uses a VGG19-BN network to extract a multi-scale feature pyramid, capturing local texture information downsampled by 1x, 2x, 4x, and 8x sequentially from shallow to deep layers, with the feature dimension gradually increasing from 64 to 512, preserving geometric details such as edges and corners. The Transformer branch uses a frozen DINOv2 ViT-L / 14 model to extract global features downsampled by 16x, with a dimension of 1024. DINOv2's semantic priors, obtained through self-supervised learning, can handle challenging scenarios such as changes in illumination and viewpoint, providing highly discriminative feature representations for the coarse matching stage. The two branches complement and fuse at different scales, forming a complete feature description system from local to global.

[0034] The core of the coarse matching stage is the Gaussian process module, which calculates the covariance matrix between the features of image A and image B using a cosine similarity kernel function, modeling spatial smoothness constraints. Specifically, the normalized grid coordinates of image B are first encoded using Fourier basis to generate a position embedding. Then, Gaussian process regression propagates the position information of image B to image A according to feature similarity, resulting in a soft position encoding. This mechanism avoids hard nearest neighbor matching and can handle regions with repetitive and weak textures. The position encoding is concatenated with the features of image A and input into a 5-layer Transformer decoder. A self-attention mechanism aggregates global context information and outputs a 64×64 grid classification probability distribution, transforming the weighted summation of discrete coordinates into a continuous initial flow field.

[0035] In the refinement stage, a convolutional neural network is used to progressively refine the flow field. At each scale, the ConvRefiner module fuses four types of complementary information: the original features of image A, matching features sampled from image B based on the current flow field, displacement embeddings of the current prediction deviating from the identity mapping, and feature correlation maps computed within a local window around the predicted location. The window radius of the local correlation decreases with scale (from 7×7 to 2×2), focusing on an increasingly smaller search range to eliminate matching ambiguities. These four types of features are concatenated and used to extract discriminative representations through eight depthwise separable convolutional blocks to predict incremental corrections and confidence changes in the flow field. The incremental correction strategy allows the network to learn only the residuals, accelerating convergence and improving accuracy. The updated flow field is bilinearly upsampled to the next scale for further refinement, gradually increasing the resolution from 40×40 to 560×560.

[0036] During the training phase, the ground truth flow field is generated through 3D geometric projection using depth maps and camera parameters as a supervision signal. The loss function employs a multi-scale joint supervision strategy: coarse matching uses cross-entropy classification loss and confidence-based binary cross-entropy loss, while the refinement phase uses generalized Charbonnier regression loss to enhance robustness against outliers. To focus on optimizing difficult examples, local loss constraints are introduced starting from scale 8, calculating the loss only in regions where the prediction error at the previous scale is less than a threshold, avoiding repeated optimization in simple regions. For high-precision scenarios, a super-resolution refinement mode is supported, upsampling the input image to 864×864 resolution, skipping the expensive DINOv2 computation with coarse-scale prediction as initialization, and repeating this only in the refinement phase to obtain denser matching results. gRPC, as... Figure 5 The diagram illustrates how HTTP / 2 and Protocol Buffers can be used to achieve efficient and reliable communication. Example 6: This embodiment further discloses information based on Embodiment 1: High-precision acquisition of front-end images and intelligent edge preprocessing: A stable, reliable, high-quality image acquisition system was constructed, and one-shot annotation of defects on the glass substrate was performed. Preliminary, intelligent preprocessing was then conducted on an industrial control computer at the edge. This stage is fundamental to the entire system, and its quality directly determines the accuracy of subsequent analysis. In terms of hardware configuration, precise selection is required based on the characteristics of the glass substrate (such as transparency, reflectivity, size, and movement speed). The hardware includes a high-speed linear scan camera, a high-resolution optical lens, a directional dark-field illumination source, a polarization control component, a motion synchronization encoder, and a high-performance image processing unit. The core selection criteria are as follows: 1. Due to the high transparency of glass substrates, conventional backlighting or diffused forward illumination is insufficient to effectively highlight surface micro-defects (such as scratches, micro-cracks, and particles). A low-angle dark-field illumination structure is employed: LED light sources illuminate the glass surface at an angle of 30°–60°. This prevents incident light from entering the imaging optical path in normal areas due to refraction or transmission, while scattering effects at surface defects reflect some light to the camera sensor, creating high-contrast bright spots in the image and significantly improving defect visibility.

[0037] 2. To address the strong specular reflectivity of glass surfaces, stray ambient light and direct reflection from the light source can easily generate high-brightness interference, masking actual defects. A polarizer is placed at the light source output, and an analyzer is positioned at the front of the camera lens, forming a cross-polarized optical path. This design effectively suppresses specular reflection components while preserving the unpolarized components of the defect-scattered light, thereby significantly reducing background spot noise without losing the defect signal.

[0038] 3. For the large-size characteristics of glass substrates (typically 1500 mm × 1850 mm and above) and the need for micron-level defect detection (e.g., ≥10 μm), an 8K or 16K high-resolution linear scan camera is preferred, paired with a telecentric lens or a dedicated FA lens with high MTF (modulation transfer function) and low distortion (<0.1%). By accurately calculating the field of view (FOV), pixel resolution, and Nyquist sampling criterion, the physical size of each pixel is ensured to meet the detection accuracy requirements (e.g., ≤5 μm / pixel), achieving full-frame, stitch-free, and blind-zone-free scanning.

[0039] 4. For high-speed operation on the production line (conveyor speed 0.5–2 m / s), a high-frequency linear array camera (line frequency ≥100 kHz) is used, along with a high-precision rotary encoder to acquire the conveyor roller displacement signal in real time, achieving microsecond-level synchronous triggering of camera exposure and light source flash. Simultaneously, a high-speed image interface such as CoaXPress or Camera Link HS is selected to ensure the real-time transmission of massive amounts of image data; the backend deploys a multi-core CPU and GPU / FPGA collaborative processing architecture to support online image stitching, real-time filtering, and defect classification.

[0040] Furthermore, the entire imaging module is encapsulated in a closed, light-shielding dark box to isolate it from external ambient light interference, and is equipped with temperature control and vibration damping mechanisms to ensure the stability and repeatability of the system during long-term operation in industrial settings.

[0041] Through a multi-dimensional hardware collaborative design combining dark-field illumination, polarization suppression, high-speed line scanning, and precise synchronization, the imaging challenges of high-speed online inspection of transparent, highly reflective materials are overcome, providing a highly reliable and high-detection-rate hardware solution for surface defect detection on glass substrates. At the industrial control computer, after receiving the raw image data, the image size is normalized to prepare for subsequent transmission.

[0042] (2) High-efficiency and reliable cloud-edge data transmission link based on gRPC Preprocessed image data needs to be transmitted quickly and reliably to the cloud server, while the computation results from the cloud also need to be sent back promptly. Choosing gRPC as the communication framework is the core technical means to achieve low latency, high throughput, and real-time performance. gRPC is a high-performance, open-source remote procedure call framework developed by Google. Based on the HTTP / 2 protocol, it brings many significant advantages over traditional REST APIs, which are specifically demonstrated in this scenario. First, HTTP / 2's multiplexing feature allows multiple requests and responses to be sent in parallel over a single TCP connection, completely avoiding the head-of-line blocking problem of HTTP / 1.1. This means that the industrial control computer can continuously send inference requests for multiple images without waiting for the return of the previous request, greatly improving transmission efficiency and making it very suitable for continuous video streaming processing. Second, gRPC uses Protocol Buffers as its interface definition language and default serialization tool. In practice, a .proto file needs to be defined first. For example, a service named ImageAnalysis can be defined, containing an RPC method called AnalyzeImage. This method receives an ImageRequest message containing fields such as image_data (binary image data) and box_coordinates (label box coordinates); it returns an Inference Result message containing fields such as defect_type (defect type), confidence (confidence level), and location (defect location). Protobuf's binary serialization efficiency is far superior to text formats like JSON, resulting in smaller data packets and faster serialization / deserialization speeds, further reducing network latency and bandwidth consumption. At the deployment level, the industrial control computer acts as a gRPC client, establishing a long-lived connection with the cloud server (gRPC server). A specific implementation detail is the connection health check and retry mechanism. Due to potential network fluctuations in industrial environments, the client code needs to implement disconnection reconnection and request retry logic to ensure that temporary network failures do not cause system interruptions. Furthermore, to ensure data security, TLS / SSL encryption is enabled on the gRPC channel to encrypt data during transmission, ensuring the confidentiality, integrity, and trustworthiness of data transmitted between the client and server.The specific implementation includes: the server loads an X.509 digital certificate and its private key issued or self-signed by a trusted Certificate Authority (CA), and configures a TLS listener; the client loads the corresponding CA root certificate to verify the server's identity, thereby establishing a one-way TLS secure connection; for higher security levels, the system further supports bidirectional TLS (mTLS), where the client also provides a CA-issued client certificate, and the server verifies the client's identity by verifying the certificate. All certificates contain a Subject Alternative Name (SAN) field that matches the actual communication address to ensure successful hostname matching verification; both communicating parties establish an encrypted channel using the standard SSL / TLS handshake process within the gRPC framework based on the HTTP / 2 protocol, prohibiting insecure plaintext transmission or configurations that skip certificate verification. This scheme effectively prevents man-in-the-middle attacks, data eavesdropping, and tampering, and is suitable for high-security industrial control data interaction scenarios. In the cloud, the gRPC server program needs to be deployed on a virtual machine with a public IP address or dedicated line connection and listens on a specific port. It is responsible for receiving requests from industrial control computers in multiple production bases across the country and even globally, and distributing them to the backend computing modules. This gRPC-based communication solution builds an efficient, stable, and secure data highway for the entire system.

[0043] (3) Deployment of high-performance feature matching models in the cloud Once the data arrives in the cloud, the core computational tasks are completed by the feature matching model deployed in the cloud. The powerful computing capabilities of the cloud environment, especially the parallel computing capabilities of GPUs, make it possible to run complex and accurate deep learning models, which are the "brain" of intelligent detection. The so-called "dense feature matching model" uses a detector-free dense feature matching algorithm based on a dual-branch Transformer and CNN architecture. The cloud computing module uses NVIDIA TensorRT for model optimization and deployment. The trained PyTorch model is first converted to the platform-independent ONNX intermediate representation format. Then, on the target deployment device, NVIDIA TensorRT is used to build an optimized inference engine (.engine file). This engine automatically performs operator fusion, memory reuse, and precision quantization (such as FP16 / INT8) for specific GPU architectures. Finally, the engine is loaded on a remote server to achieve low-latency, high-throughput real-time inference services. The entire process ensures efficient model migration from training to deployment and fully leverages hardware acceleration capabilities to meet the stringent requirements of production lines for high throughput and low latency. TensorRT is a high-performance deep learning inference optimization library launched by NVIDIA. It can optimize trained models by quantization, layer fusion, and automatic kernel tuning, significantly improving inference speed and reducing resource consumption.

[0044] (4) Real-time feedback of test results and integrated control of production line After the cloud model completes the calculation, it will immediately return the structured results to the industrial control computer on the edge via the gRPC link.

[0045] In cloud processing, a deep learning model is first used to jointly infer between the template image and the current frame image to obtain dense matching results for defect detection. The model takes two images as paired inputs and performs multi-scale feature extraction on each image. The feature extraction module combines convolutional and global modeling structures to map the original image into multi-level feature representations, thus preserving both local texture and overall structural information at different spatial scales. To ensure consistency and comparability between the template image and the current frame image in the feature space, they share the same set of feature extraction parameters, resulting in multi-scale feature mapping within a unified feature space. After feature extraction, the model further performs cross-image association calculations between the template and current frame features. By calculating the similarity between features at different spatial locations in the feature space, the model can find potential corresponding positions for each pixel in the template image in the current frame. This association calculation combines local contextual information with global consistency constraints, enabling the model to maintain stable and reliable matching relationships even under complex conditions such as viewpoint changes, lighting differences, or local occlusion.

[0046] Based on the aforementioned cross-image association results, the model employs a coarse-to-fine inference strategy to progressively generate dense pixel mapping outputs. Specifically, the model first predicts a coarse correspondence between the template image and the current frame at a lower spatial resolution. Then, through progressive refinement and upsampling operations, the mapping results are restored to the target resolution, resulting in a dense pixel mapping field covering the entire image. This mapping field describes the corresponding position of each pixel in the template image within the current frame. Simultaneously generating the dense pixel mapping, the model also outputs a corresponding matching confidence score for each pixel mapping relationship. This confidence score characterizes the reliability of the pixel match under feature space and contextual constraints, and its value reflects the model's trust in the matching result. By simultaneously outputting pixel mapping results and pixel-by-pixel confidence information, the model provides the foundation for subsequent robust selection, position generation, and result fusion. After model inference is complete, the dense pixel mapping field and pixel-by-pixel matching confidence scores are organized into a structured output vector and stored in GPU memory. This vector is then directly passed to the post-processing module in tensor form, avoiding frequent copying of intermediate results between different computing devices.

[0047] After model inference is completed, the post-processing stage begins. The dense matching results output by the model are parsed and structured to generate the location, confidence level, and category information of the defects. The dense pixel mapping relationship and pixel-by-pixel matching confidence level serve as input data for the post-processing stage. During defect location generation, the pixel mapping relationship output by the model is used to migrate the pre-stored defect spatial annotation regions in the template image to the current frame. Specifically, the defect annotation regions in the template image can be represented as one or more sets of defect regions. By querying the pixel mapping relationship, pixels within the template defect regions can obtain their corresponding spatial locations in the current frame, thus forming a set of candidate pixels related to the defects in the current frame.

[0048] Subsequently, the candidate pixel set is spatially aggregated. By statistically analyzing its coordinate distribution in the current frame, the minimum bounding rectangle of the candidate pixel set is calculated, and this rectangle is used as the initial location box of the defect in the current frame. This processing method transforms the continuous dense pixel mapping results into a structured bounding box representation, facilitating the subsequent output of defect results and integration with the business system. During the confidence calculation, the pixel-by-pixel matching confidence score output by the model is used as a basis, and the confidence information within the defect location box is fused at the regional level. Specifically, the confidence distribution of pixels within the location box is statistically analyzed, and the regional confidence score is calculated using the maximum value, average value, or weighted average value to reflect the overall reliability of the defect location prediction result in the spatial dimension. Simultaneously, the robust geometric estimation process introduced during the construction of pixel mapping relationships can output a corresponding consistency metric or inlier ratio. This consistency metric is fused with the regional confidence score to obtain the final defect location confidence score, thereby effectively reducing the probability of false detection caused by local mismatches. During the defect category determination process, based on the determined defect location bounding box, the corresponding image region in the current frame is cropped to generate a local image of the defect candidate region. This local image is input into the defect classification model for inference to obtain the defect category and its corresponding classification confidence score. The defect classification model can be a lightweight convolutional neural network or a feature-based classification model, and its output is managed and fused with the confidence score obtained during the localization phase within a unified framework.

[0049] In the final result generation stage, the defect location bounding box, defect location confidence, and defect category and classification confidence are comprehensively processed to form a unified defect detection result. Furthermore, based on actual application requirements, the location confidence and classification confidence can be weighted and fused to generate a comprehensive confidence index characterizing the overall reliability of the defect detection result. Finally, the defect detection result is output in structured data format, including defect category, defect location bounding box coordinates, and corresponding confidence information, and is returned to the industrial control computer via a communication interface.

[0050] This is not simply a "pass / fail" signal, but a rich set of information that needs to be parsed, displayed, and ultimately integrated into the production control system on the industrial control computer to form a closed-loop detection system. The local industrial control computer first obtains a Protocol Buffer interface definition file (.proto file) that is strictly consistent with the cloud-based inference service. This file explicitly defines the service interface for Remote Procedure Call (RPC), the request message structure (such as DetectRequest containing the byte stream of the image to be detected), and the response message structure (such as DetectResponse containing the defect detection results). The response message structure declares key business data in structured fields, including defect type (type, string type), confidence (confidence, 32-bit floating-point number), bounding box coordinates (x1, y1, x2, y2, normalized or pixel-unit floating-point numbers), and physical dimensions (such as scratch length_mm, floating-point number in millimeters). It also supports using the repeated modifier to represent a list of multiple defect detection results. Subsequently, the local client uses the Protocol Buffer compiler (protoc) in conjunction with the corresponding language's gRPC plugin (such as grpc_python_plugin) to automatically generate two sets of local code based on the .proto file: (1) Message class code (such as detect_response_pb2.py) contains local language classes (such as Python's DetectResponse and Defect classes) that correspond one-to-one with the definitions in .proto, and are used to carry the deserialized data; (2) gRPC client stub code (such as detect_pb2_grpc.py) encapsulates the details of underlying HTTP / 2 communication, serialization / deserialization and TLS encryption (if enabled), and exposes synchronous or asynchronous remote method call interfaces to the outside world.

[0051] During runtime, the local application initiates a call to the cloud-based gRPC server through this stub, transmitting a serialized request message. After completing model inference, the server fills the detection results into the DetectResponse message instance and serializes it into a compact binary format through the gRPC framework, returning it to the client via an established TLS encrypted channel (authenticated with an X.509 certificate). Upon receiving the binary stream, the client's gRPC runtime library automatically triggers the deserialization process, accurately restoring the byte stream into an object instance (such as a DetectResponse object) in local memory based on the pre-generated message class structure. Each field is mapped to the native data types str, float, list, etc., of the local language according to the type and name defined in .proto. The application does not need to handle any underlying network protocols or binary parsing logic; it only needs to use standard object-oriented field access syntax, such as response.defects[0].type, defect.confidence, and defect.length_mm, to efficiently and securely extract the required detection information. Then, real-time visualization is performed using libraries in OpenCV. One specific implementation uses a technique similar to video subtitle overlay, directly overlaying the result information onto the real-time video stream. The software interface continuously displays the video from the camera. When the inspection result of a glass substrate is returned, the software marks the corresponding position of the substrate in the video stream with different colored graphics based on the result data. For example, qualified substrates are marked with a green semi-transparent rectangle and the highlighted text "PASS"; unqualified substrates are marked with a red rectangle and the defect type is indicated next to it with text such as "SCRATCH - 99.5%". At the same time, the software generates a log entry, recording the timestamp, product ID, inspection result, defect image snapshot, etc., and stores it in a local database or synchronizes it to a cloud big data platform for subsequent quality traceability and production analysis. Furthermore, the value of this system is ultimately reflected in its deep integration with the production execution system. The display software is not just a "monitor" but also a "controller". It communicates with the lower-level programmable logic controller through the digital I / O ports (such as GPIO), serial ports, or industrial Ethernet protocols (such as Profinet, EtherCAT) provided by the industrial control computer. When a defective product is detected, the software immediately sends a switching signal to the PLC. Upon receiving this signal, the PLC triggers a series of interlocking actions: for example, controlling the solenoid valve or cylinder at the end of the conveyor belt to activate a push rod, accurately removing the defective product to the defective product area; or, when multiple defective products appear consecutively, the system can automatically trigger an audible and visual alarm to remind staff to intervene and inspect the equipment.In this way, a complete automated closed loop is achieved from "visual perception" to "intelligent analysis" and then to "precise execution," seamlessly embedding artificial intelligence detection capabilities into the core processes of modern intelligent production lines, significantly improving the level of automation in production and product quality.

Claims

1. A method for real-time detection of defects on a glass substrate based on dual-branch features, characterized in that, include: S1: Acquire an image of the glass substrate and perform preprocessing to obtain a preprocessed image; S2: Construct a feature matching model; S3: The feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and trains the feature matching model to obtain a real-time detection model for glass substrate defects. S4: Real-time detection of defects in glass substrates is performed based on the real-time defect detection model for glass substrates.

2. The real-time defect detection method for glass substrates based on bi-branch features according to claim 1, characterized in that, In step S2, the feature matching model is specifically as follows: The module consists of a feature encoder, a global matcher, a multi-scale convolutional modifier, and a loss function module. In the feature encoding module, the frozen DINOv2 model is used as a coarse feature encoder to provide coarse features with strong generalization, and a dedicated VGG19 network is used to extract precise fine features, forming a robust and localizable feature pyramid. The global matcher module consists of a Gaussian process matching encoder and a Transformer matching decoder. The latter models the multimodal distribution by predicting anchor point probabilities and outputs a coarse warp. The multi-scale convolutional modifier iteratively optimizes the coarse warp based on fine features; The loss function module uses classification regression loss for coarse-matched multimodal characteristics and generalized Charbonnier robust loss for fine-corrected single-modal characteristics.

3. The real-time defect detection method for glass substrates based on bi-branch features according to claim 1, characterized in that, In step S3, the feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse features. Specifically, the complete and defect-free glass substrate image is taken as image A, and the preprocessed image is taken as image B. The covariance matrix between the features of image A and image B is calculated using the cosine similarity kernel function to model spatial smoothness constraints. First, Fourier basis encoding is performed on the normalized grid coordinates of image B to generate position embeddings. Then, Gaussian process regression is used to propagate the position information of image B to image A according to feature similarity to obtain soft position encoding. The soft position encoding is concatenated with the features of image A and input into a multi-layer Transformer decoder. Global context information is aggregated through a self-attention mechanism to output a classification probability distribution, and the weighted summation of discrete coordinates is converted into a continuous initial flow field.

4. The real-time defect detection method for glass substrates based on bi-branch features according to claim 1, characterized in that, In step S3, the feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain the fine features, specifically: Image A is a complete, defect-free glass substrate image, and image B is a preprocessed image. In the refinement stage, a convolutional neural network is used to progressively correct the flow field. At each scale, the ConvRefiner module fuses four types of complementary information: the original features of image A, matching features sampled from image B based on the current flow field, displacement embeddings that deviate from the identity mapping in the current prediction, and feature correlation maps computed within a local window around the prediction location. The window radius of the local correlation decreases with scale, focusing on an increasingly smaller search range to eliminate matching ambiguities. Discriminative representations are extracted through multiple depthwise separable convolutional blocks to incrementally correct the flow field and change the confidence level. The incremental correction strategy allows the network to learn only the residuals, accelerating convergence and improving accuracy. The updated flow field is bilinearly upsampled to the next scale for further refinement.

5. The real-time defect detection method for glass substrates based on bi-branch features according to claim 1, characterized in that, In step S3, the feature matching model is trained to obtain the real-time defect detection model for the glass substrate, specifically as follows: During the training phase, the ground truth flow field is generated through 3D geometric projection using depth maps and camera parameters as a supervision signal. The loss function adopts a multi-scale joint supervision strategy: coarse matching uses cross-entropy classification loss and confidence binary cross-entropy loss, while the refinement phase uses generalized Charbonnier regression loss to enhance robustness to outliers. To focus on optimizing hard examples, local loss constraints are introduced starting from the 8th scale, and the loss is calculated only in regions where the prediction error of the previous scale is less than the threshold. For high-precision scenarios, the input image is upsampled to high resolution, and the coarse-scale prediction is used as initialization and repeated only in the refinement phase to obtain denser matching results.

6. The real-time defect detection method for glass substrates based on bi-branch features according to claim 1, characterized in that, In step S4, specifically: The real-time defect detection model for glass substrates uses two images as paired inputs: a template image and the current frame image. Internally, the model first performs multi-scale feature extraction on both input images. After feature extraction is completed, cross-image association calculation is further performed on the template features and the current frame features. By calculating the similarity relationship between features at different spatial locations in the feature space, the model can find potential corresponding locations for each pixel position in the template image in the current frame. Based on the above cross-image association results, a coarse-to-fine inference strategy is adopted to generate dense pixel mapping output step by step: first, a coarse correspondence between the template image and the current frame is predicted at a lower spatial resolution, and then the mapping result is restored to the target resolution through progressive refinement and upsampling operations, thereby obtaining a dense pixel mapping field covering the entire image; the mapping field is used to describe the corresponding position of each pixel in the template image in the current frame; while generating dense pixel mapping, a corresponding matching confidence score is also output for each pixel mapping relationship; The dense matching results are parsed and structured to generate information on the location, confidence level, and category of defects; Dense pixel mapping relationships and pixel-by-pixel matching confidence are used as input data for the post-processing stage; During the defect location generation process, the pixel mapping relationship output by the model is used to migrate the pre-stored defect spatial annotation region in the template image to the current frame; Specifically, the defect annotation area in the template image can be represented as a set of one or more defect areas. By querying the pixel mapping relationship, the pixels in the template defect area can obtain the corresponding spatial position in the current frame, thereby forming a set of candidate pixels related to the defect in the current frame. Spatial aggregation processing is performed on the above candidate pixel set. By statistically analyzing their coordinate distribution in the current frame, the minimum bounding rectangle of the candidate pixel set is calculated, and this rectangle is used as the initial location box of the defect in the current frame. Through this processing method, the continuous dense pixel mapping result is transformed into a structured bounding box representation. In the confidence calculation process, the confidence information within the defect location box is fused at the region level based on pixel-by-pixel matching confidence. In the final result generation stage, the defect location box, defect location confidence, and defect category and classification confidence are comprehensively processed to form a unified defect detection result. The location confidence and classification confidence are weighted and fused to generate a comprehensive confidence index that characterizes the overall reliability of the defect detection results; Finally, the defect detection results are output in the form of structured data, including defect category, defect location bounding box coordinates and corresponding confidence information, and are returned to the industrial control computer through the communication interface.

7. A real-time defect detection system for glass substrates based on dual-branch features, characterized in that, include: Image acquisition module and cloud server; The cloud server deploys a preprocessing submodule, a model submodule, and a detection submodule. Image acquisition module: Acquires the glass substrate image from the industrial control computer and performs preprocessing to obtain the preprocessed image; Preprocessing submodule: Transmits the preprocessed image to the cloud server, constructs a feature matching model, and deploys the feature matching model on the cloud server; Model submodule: The feature matching model performs bi-branch image feature matching based on the preprocessed image to obtain coarse and fine features, and trains the feature matching model to obtain a real-time defect detection model for glass substrates. Detection submodule: Performs real-time detection of defects in glass substrates based on the real-time defect detection model for glass substrates.

8. The real-time defect detection system for glass substrates based on dual-branch features according to claim 7, characterized in that, The image acquisition module includes a high-speed linear array camera, a high-resolution optical lens, a directional dark field illumination source, a polarization control component, and a high-performance image processing unit. High-speed line scan camera: The high-frequency line scan camera is equipped with a high-precision rotary encoder to acquire the displacement signal of the conveyor roller in real time; High-performance image processing unit: uses CoaXPress or Camera Link HS high-speed image interface; back-end deployment of multi-core CPU and GPU / FPGA collaborative processing architecture; High-resolution optical lenses: Linear scan cameras; Directional dark field lighting source: LED light source illuminates the glass surface at an angle of incidence of 30 degrees to 60 degrees; Polarization control components: A polarizer is set at the output end of the directional dark field illumination source, and an analyzer is configured at the front end of the high-speed linear array camera lens to form a cross-polarized optical path.

9. The real-time defect detection system for glass substrates based on dual-branch features according to claim 7, characterized in that, In step S2, transmitting the preprocessed image to the cloud server specifically involves: gRPC was chosen as the communication framework to transmit the preprocessed image to the cloud server. gRPC is based on the HTTP / 2 protocol. HTTP / 2's multiplexing feature allows multiple requests and responses to be sent in parallel over a single TCP connection. gRPC uses Protocol Buffers as its interface definition language and default serialization tool. TLS / SSL encryption is enabled on gRPC channels to encrypt data during transmission. On the cloud server side, the gRPC server program is deployed on a virtual machine with a public IP address or a dedicated line connection and listens on a specific port.

10. The real-time defect detection system for glass substrates based on dual-branch features according to claim 7, characterized in that, In step S2, deploying the feature matching model on the cloud server specifically involves: NVIDIA TensorRT is used for model optimization and deployment. The trained PyTorch model is converted into a platform-independent ONNX intermediate representation format, and then the optimized inference engine is built on the target deployment device using NVIDIA TensorRT.

Images