Glass edge defect detection method and system based on machine vision

Abstract

The invention provides a glass edge defect detection method and system based on machine vision, and relates to the technical field of image analysis. The method effectively solves the core problems of insufficient detection precision and poor reliability caused by reflection of light on the surface of glass, stain interference and low defect contrast in the prior art, and comprises the following steps: obtaining defect characteristics of a glass calibration plate containing a standard defect sample, verifying the defect characteristics, and obtaining defect verification characteristics; the defect verification features are used for representing high-confidence-coefficient defects subjected to physical rule cross validation; reconstructing the defect verification features to obtain defect quantization information; calculating the defect quantification information to obtain defect prediction data; and integrating the defect prediction data and the defect quantification information to obtain defect detection data.

Description

Technical Field

[0001] This application relates to the field of image analysis technology, and in particular to a method and system for detecting glass edge defects based on machine vision. Background Technology

[0002] With the rapid development of high-end manufacturing, glass, due to its excellent physical and optical properties, is increasingly widely used in fields such as smartphone displays, automotive glass, photovoltaic panels, and precision optical instruments. As the most vulnerable part of its structure, the glass edge is highly susceptible to various defects such as cracks, chips, edge breakage, and bubbles during processing, transportation, and use. These minute defects not only affect the product's appearance but can also become stress concentration points, potentially leading to the entire glass breaking during subsequent use and causing serious safety accidents. Therefore, efficient and accurate defect detection of glass edges is a crucial step in ensuring product quality and production safety.

[0003] In existing technologies, traditional glass defect detection mainly relies on manual visual inspection. This method is highly subjective, inefficient, costly, and prone to missed detections and false detections over long periods of operation. To overcome the drawbacks of manual inspection, detection methods based on single-modal machine vision are widely used. For example, high-resolution line scan cameras are used to acquire images of the glass edges, and then image processing algorithms are used to identify defects. However, such methods have significant limitations: they are easily affected by glass surface reflections, stains, water stains, etc., resulting in a high false alarm rate; their detection capability is insufficient for transparent or semi-transparent internal defects and tiny cracks with extremely low contrast to the background; and they can only provide two-dimensional information, failing to obtain key three-dimensional dimensions such as the depth and volume of defects, resulting in an incomplete assessment. To improve detection accuracy, some technical solutions have begun to introduce multimodal information, such as combining visible light and infrared thermal imaging. However, most existing solutions remain at the level of simple image registration, pixel-level fusion, or feature-level stitching, failing to delve into the physical relationships underlying different modal information. For example, visible light images reflect geometric shapes, while infrared images reflect thermal conductivity characteristics; there is an inherent connection between the two determined by physical laws. Current technologies fail to effectively utilize this connection, resulting in low efficiency in utilizing multimodal information, limited ability to identify false features, and an inability to fundamentally solve the problem of 3D reconstruction of transparent objects. Furthermore, current technologies generally remain at the level of "defect detection," lacking the ability to predict the future development trend of defects and failing to provide data support for product remaining life assessment and preventative maintenance. Summary of the Invention

[0004] This application provides a machine vision-based method and system for detecting glass edge defects. It can effectively solve the core problems of insufficient detection accuracy and poor reliability caused by glass surface reflection, stain interference and low defect contrast, while ensuring high detection efficiency and automation. It also breaks through the technical bottleneck of traditional methods being unable to accurately quantify defects in three dimensions and predict their evolution trend.

[0005] To achieve the above objectives, this application adopts the following technical solution: In the first aspect, a machine vision-based method for detecting glass edge defects is provided, including: acquiring the defect features of a glass calibration plate containing standard defect samples, wherein the standard defect samples refer to defects of the type of cracks, notches, chipping, and bubbles; The defect features are verified to obtain defect verification features, which are used to characterize high-confidence defects that have been cross-verified by physical laws; the verification is used to verify whether the defect features conform to the spatial morphological characteristics and temporal evolution laws of real defects. The defect verification features are reconstructed to obtain defect quantification information. Reconstruction refers to converting the verified features into corresponding three-dimensional defect quantification information. The quantification information is used to characterize the three-dimensional geometric parameters, thermodynamic characteristic parameters, and defect type identifier of the defect. The defect quantification information is calculated to obtain defect prediction data. The defect prediction data and defect quantification information are integrated to obtain defect detection data.

[0006] Preferably, an image of a glass calibration plate containing standard defect samples is acquired. Based on the image of the glass calibration plate containing standard defect samples, a visible light image and an infrared thermal image of the edge region of the glass to be tested are extracted. The visible light image is a high-resolution visible light image of the glass edge region, and the thermal image sequence is a dynamic infrared thermal image of the glass edge region under controlled thermal excitation. The visible light image is processed by an edge-aware network to obtain geometric candidate features of the glass edge in the visible light image. The infrared thermal image is processed by a physical information neural network embedding the heat conduction equation to obtain thermodynamic candidate features. The geometric candidate features and the thermodynamic candidate features are used as defect features.

[0007] Preferably, multimodal image acquisition is performed on the glass calibration plate containing standard defect samples to calibrate the spatial transformation matrix and time synchronization parameters between the industrial camera and the infrared camera. For the glass calibration plate containing standard defect samples, the calibrated spatial transformation matrix and time synchronization parameters are used to synchronously acquire visible light images and infrared thermal imaging images under controlled thermal excitation.

[0008] Preferably, for each geometric candidate feature, a first morphological feature in the infrared thermal imaging image is predicted based on a pre-calibrated association model. The first morphological feature is used to characterize the stress field morphology that the geometric candidate feature should present in the infrared thermal imaging image. Then, the first morphological feature is searched and matched in the thermodynamic candidate features. The co-confidence score of the matching degree between each thermodynamic candidate feature and the corresponding first morphological feature is calculated. The first morphological feature is filtered based on the co-confidence score to obtain the second morphological feature. For each thermodynamic candidate feature, an enhanced search is performed in the visible light image at the corresponding position to obtain the third morphological feature. The enhanced search is a local region search centered on the thermodynamic feature position using an edge detection algorithm. The second morphological feature and the third morphological feature are integrated to obtain the defect verification feature.

[0009] Preferably, for the defect verification features, three-dimensional reconstruction is performed to obtain several defects and the defect quantification information corresponding to each defect. The three-dimensional reconstruction adopts the multi-view photometric stereo method and uses the stress field gradient inverted from the infrared thermal imaging image as prior knowledge to guide and constrain the solution process of the three-dimensional morphology.

[0010] Preferably, a defect agent is constructed based on the defect and the defect quantification information corresponding to each defect. Based on the defect agent in a simulation environment, the stress intensity factor is calculated using finite element analysis. Based on the stress intensity factor, the expansion path, rate, and remaining safe lifetime of the defect agent are predicted. The defect and the defect quantification information corresponding to each defect, the expansion path, rate, and remaining safe lifetime of the defect agent are integrated to obtain defect prediction data.

[0011] Preferably, the defect quantification information and defect prediction data are structurally integrated to generate a comprehensive inspection report that includes defect classification, location coordinates, three-dimensional dimensions, confidence score, evolution trend prediction and remaining safe life, and this comprehensive inspection report is output as the final defect inspection data.

[0012] Based on the aforementioned technical approach, by utilizing a pre-calibrated physical correlation model, the information from the two modalities undergoes in-depth cross-validation and mutual verification, significantly improving detection accuracy and effectively suppressing false alarms caused by factors such as surface stains and uneven illumination. Simultaneously, by embedding physical laws into the neural network model and 3D reconstruction algorithm, the detection challenges posed by the transparency of glass are resolved, achieving precise 3D quantification of defects. Furthermore, this elevates detection from post-event discovery to pre-event prediction, providing technical support for the full lifecycle quality control and safety assessment of glass products.

[0013] Secondly, a machine vision-based glass edge defect detection system is provided, the system comprising: Acquisition module: Used to acquire the defect characteristics of a glass calibration plate containing standard defect samples. Standard defect samples refer to defects such as cracks, notches, chipped edges, and bubbles. Verification module: Used to verify defect features and obtain defect verification features. Defect verification features are used to characterize high-confidence defects that have been cross-validated by physical laws. Reconstruction module: Used to reconstruct defect verification features to obtain defect quantification information; The calculation module is used to perform calculations on the defect quantification information to obtain defect prediction data. Output module: Used to integrate defect prediction data and defect quantification information to obtain defect detection data.

[0014] The solution provided in the second aspect above is used to implement the method provided in the first aspect above, and its specific implementation will not be described in detail here. The technical effects corresponding to any implementation method of the solution provided in the second aspect above can be found in the technical effects corresponding to any implementation method of the first aspect above, and will not be described in detail here.

[0015] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the defect detection device provided in the embodiments of this application; Figure 2 A schematic flowchart of a glass edge defect detection method based on machine vision provided in an embodiment of this application; Figure 3 This is a schematic diagram of a machine vision-based glass edge defect detection system provided in an embodiment of this application. Detailed Implementation

[0018] In the embodiments of this application, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different. The technical features described by "first" and "second" have no sequential or size order.

[0019] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0020] In the embodiments of this application, at least one can also be described as one or more, and multiple can be two, three, four or more, and this application does not impose any restrictions.

[0021] Furthermore, the network architecture and scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0022] The solutions provided by the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0023] The solution provided in this application can be applied to Figure 1 The diagram shows the structure of the defect detection device.

[0024] For example, Figure 1 The defect detection device shown may include: a detection controller 101 and a robotic arm 103. The detection controller 101 can be an industrial computer, an embedded controller, or a computing unit containing a central processing unit and a graphics processing unit, used to execute the algorithm model and logic control in the detection method described in this application.

[0025] For example, Figure 1 The detection controller 101 shown can be configured to: control the movement trajectory of the robotic arm 103 to drive the industrial camera 102 to scan the glass surface to be tested; control the industrial camera 102 to acquire images; receive and process image data from the industrial camera 102; run a defect detection algorithm; and finally generate and output the detection results.

[0026] Optionally, the detection controller 101 may use a field-programmable gate array or an application-specific integrated circuit as a hardware acceleration unit to accelerate computationally intensive tasks such as image preprocessing and feature extraction in order to meet the real-time requirements of high-speed production lines.

[0027] Optionally, the detection controller 101 supports communication with the servo driver of the robotic arm 103 via industrial Ethernet to send motion commands; it also supports connection with the industrial camera 102 via a universal serial bus or GigEVision interface to achieve high-speed image data transmission.

[0028] The detection controller 101 sends a preset motion path command to the robotic arm 103 and simultaneously sends a trigger signal to the industrial camera 102 to achieve accurate image acquisition when the robotic arm moves or reaches a designated detection point.

[0029] Specifically, when the inspection begins, the inspection controller 101 first drives the robotic arm 103 to move to the initial inspection position, and then controls the industrial camera 102 to take pictures of the starting area of ​​the glass. Subsequently, the inspection controller 101 continuously controls the robotic arm 103 to move according to a preset scanning strategy, such as an "S" shaped path, and triggers the industrial camera 102 to acquire images at each predetermined shooting point until the entire surface of the glass to be tested is covered.

[0030] Optional, Figure 1 The defect detection device shown may also include a light source and a human-machine interface. The light source provides stable and uniform illumination for the industrial camera 102 to eliminate ambient light interference. The human-machine interface displays real-time detection images, defect details, statistics, and alarm information, and allows the operator to configure detection parameters.

[0031] like Figure 2 As shown in the embodiments of this application, a glass edge defect detection method based on machine vision may include: S201: Obtain the defect characteristics of a glass calibration plate containing standard defect samples.

[0032] The standard defect sample refers to defects such as cracks, notches, chipped edges, and bubbles.

[0033] Optionally, a pre-defined calibration procedure can be used to obtain the defect features of a glass calibration plate containing standard defect samples. This procedure includes: first, applying standardized, controllable thermal excitation to the glass calibration plate under controlled conditions; then, simultaneously acquiring visible light and infrared thermal imaging image sequences of the calibration plate during the thermal excitation process using an industrial camera and an infrared camera with precisely calibrated spatial and temporal relationships; finally, extracting features corresponding to known defects from the two modal images using algorithms, and establishing a quantitative mapping relationship between them, i.e., an optical-thermodynamic correlation model.

[0034] Among them, the optical-thermodynamic correlation model refers to a knowledge base established through a calibration process to quantitatively describe the mapping relationship between the geometric morphological characteristics of glass defects and their thermodynamic response modes under controlled thermal excitation.

[0035] Specifically, the input parameters of the optical-thermodynamic correlation model can be defect geometric features from visible light images, such as defect type, size, orientation, and location. Its output is the corresponding predicted thermodynamic features, such as expected hotspot morphology, peak temperature difference, and temperature evolution curve over time. Conversely, the model can also take anomalous thermodynamic signals as input and output inferences and search guidance for potential geometric morphologies. Its core purpose is to provide a physical basis for the mutual conversion and verification of features from two different modes, thereby achieving cross-validation of defects.

[0036] In some embodiments, an image of a glass calibration plate containing standard defect samples is acquired. Based on the image of the glass calibration plate containing standard defect samples, a visible light image and an infrared thermal image of the edge region of the glass to be tested are extracted. The visible light image is a high-resolution visible light image of the glass edge region, and the thermal image sequence is a dynamic infrared thermal image of the glass edge region under controlled thermal excitation. The visible light image is processed by an edge-aware network to obtain geometric candidate features of the glass edge in the visible light image. The infrared thermal image is processed by a physical information neural network embedding the heat conduction equation to obtain thermodynamic candidate features. The geometric candidate features and the thermodynamic candidate features are used as defect features.

[0037] Among them, the physical information neural network refers to a special computational model that embeds physical laws as constraints into the training process of a deep neural network. In this embodiment, it is specifically used to process dynamic infrared thermal imaging image sequences. Its core lies in constructing a composite loss function, which includes the data loss term of the traditional neural network, namely the mean square error between the temperature field predicted by the network and the actual infrared image observation value, and introduces a physical loss term. This physical loss term uses automatic differentiation technology to transform the partial differential equation describing the heat conduction process into a constraint on the network output, forcing the network to learn the temperature field evolution law that must follow the real physical laws.

[0038] Specifically, the physical information neural network is a function approximator used to learn a continuous temperature field function T(x, y, t), where (x, y) represents the spatial coordinates of the glass edge region, and t represents time. The network input consists of these spatiotemporal coordinates, and the output is the predicted temperature value at the corresponding point. During training, the data loss term is calculated as the mean square error between the predicted temperature T_pred and the observed temperature T_obs at the pixel points (x_i, y_i, t_i) actually observed by the infrared camera. Simultaneously, the physical loss term uses automatic differentiation to solve for the first-order partial derivative of the network output temperature field T(x, y, t) with respect to time t and the second-order partial derivatives with respect to space x and y, respectively. These derivatives are then substituted into the Fourier partial differential equation of heat conduction describing heat diffusion in the glass to obtain a physical residual. This physical loss term represents the mean square value of this physical residual at a large number of randomly sampled "collatexes" throughout the entire spatiotemporal domain. By simultaneously minimizing data loss and physical loss, the network is forced not only to fit the temperature on a limited number of observation data points, but also to follow thermodynamic laws throughout the entire continuous spatiotemporal domain, ultimately outputting a smooth, continuous and physically consistent temperature field. This allows for the precise extraction of thermodynamic candidate features such as temperature gradients that reflect abnormal heat conduction inside defects.

[0039] Specifically, the industrial camera and infrared camera of the calibrated detection device simultaneously acquire images of the glass to be tested placed at the inspection station. High-resolution visible light images are input into a pre-trained edge perception network, such as an improved U-Net or HED network, which is specifically designed to detect pixel-level edge information and outputs a set of geometric candidate features, each containing location coordinates, orientation, length, and preliminary geometric confidence. Simultaneously, a sequence of dynamic infrared thermal imaging images is input into a physical information neural network embedded with the heat conduction equation. During training, the physical laws of heat conduction are used as part of the loss function, ensuring that the learned features conform to physical laws. This network solves an inverse problem to deduce the stress field distribution on the glass surface and outputs a set of thermodynamic candidate features, each containing the location, morphology, peak temperature difference, and time evolution curve of the abnormal region.

[0040] Controlled thermal excitation refers to applying a brief, standardized thermal shock to the edge of a glass using an excitation source whose energy, duration, and area of ​​action can be precisely controlled. The aim is to generate a transient, non-uniform temperature field within the glass, causing hidden defects to manifest as weak temperature difference signals that can be captured by an infrared camera due to changes in the local heat conduction path.

[0041] Thermodynamic candidate features refer to thermodynamic response patterns that characterize potential defects, extracted from infrared thermal imaging image sequences by physical information neural networks. For example, cracks may lead to linear abnormally high or low temperature zones, bubbles may lead to point-like hot spots with different heat capacities than the surrounding material, and edge chipping may lead to irregular thermal diffusion fronts. These features not only contain spatial location and morphology but also dynamic information that changes over time.

[0042] Geometric candidate features refer to geometric discontinuities detected by edge-aware networks in high-resolution visible light images that may correspond to real defects. These include, but are not limited to, linear suspected cracks, arc-shaped suspected notches, and irregular suspected chipped edge contours. These features are extracted based on low-level visual information such as image grayscale, gradient, and texture, and have not yet been physically verified, therefore there is a certain possibility of false positives.

[0043] For example, multimodal image acquisition is performed on a glass calibration plate containing standard defect samples to calibrate the spatial transformation matrix and time synchronization parameters between the industrial camera and the infrared camera. For the glass calibration plate containing standard defect samples, the calibrated spatial transformation matrix and time synchronization parameters are used to synchronously acquire visible light images and infrared thermal imaging images under controlled thermal excitation.

[0044] Specifically, the calibration process consists of three sub-steps: Geometric calibration: Using a glass calibration plate with a high-contrast checkerboard pattern, multiple sets of images with different poses are simultaneously captured by an industrial camera and an infrared camera. Using algorithms such as Zhang's calibration method, the intrinsic parameters (focal length, distortion, etc.) and extrinsic parameters (relative rotation and translation between the cameras) of the two cameras are calculated. Finally, the spatial transformation matrix for converting the infrared image coordinate system to the visible light image coordinate system is calculated.

[0045] Time synchronization calibration: A hardware trigger signal generator is used to send trigger signals to two cameras simultaneously. The exposure delay of the two cameras is verified and calibrated by analyzing the timestamps output by the cameras using a high-speed oscilloscope or by analyzing the timestamps output by the cameras, ensuring that their time synchronization parameters are accurate to the microsecond level.

[0046] Optical-Thermodynamic Correlation Model Calibration: Standardized controlled thermal excitation is applied to the glass calibration plate, and its visible light and infrared image sequences are acquired simultaneously. For each standard defect on the calibration plate with known geometric parameters, its geometric features in the visible light image and its thermodynamic features in the infrared image sequence are extracted. These paired feature data are stored to construct a query database, which constitutes the optical-thermodynamic correlation model upon which subsequent physical co-verification is based.

[0047] S202: Verify the defect features to obtain defect verification features.

[0048] Among them, defect verification features are used to characterize high-confidence defects that have been cross-validated by physical laws.

[0049] The above verification is used to verify whether the defect characteristics conform to the spatial morphological characteristics and temporal evolution of real defects.

[0050] In some embodiments, for each geometric candidate feature, a first morphological feature in the infrared thermal imaging image is predicted according to a pre-calibrated association model. The first morphological feature is used to characterize the stress field morphology that the geometric candidate feature should present in the infrared thermal imaging image. The first morphological feature is searched and matched among thermodynamic candidate features. The co-confidence score of the matching degree between each thermodynamic candidate feature and the corresponding first morphological feature is calculated. The first morphological feature is filtered based on the co-confidence score to obtain a second morphological feature. For each thermodynamic candidate feature, an enhanced search is performed in the visible light image at the corresponding location to obtain a third morphological feature. The enhanced search is a local region search centered on the thermodynamic feature location using a more sensitive edge detection algorithm. The second morphological feature and the third morphological feature are integrated to obtain the defect verification feature.

[0051] The first morphological feature refers to a theoretical thermodynamic response pattern derived from physical laws. It is not a feature directly observed from the image, but rather a prediction of the specific stress field distribution pattern that the geometric defect should exhibit in the infrared thermal imaging image sequence under controlled thermal excitation, based on a geometric candidate feature, such as the shape, size, and location of a suspected crack line, by calling the optical-thermodynamic correlation model calibrated in S201. For example, a linear high-temperature anomaly region whose temperature decays with time according to a specific curve.

[0052] The second morphological feature refers to the defect feature that has been positively verified and confirmed as having high confidence. Specifically, it refers to those features whose collaborative confidence score exceeds a preset threshold after matching the first morphological feature with the actual extracted thermodynamic candidate features. This means that the predicted thermal performance of a geometric shape seen in a visible light image is highly consistent with the actual observed thermal performance.

[0053] The third type of feature refers to newly discovered geometric features through reverse verification. When a thermodynamic candidate feature, such as an isolated hot spot, cannot find a corresponding geometric candidate feature in forward verification, the system considers it to be a "hidden" defect that is difficult to detect in visible light images by conventional algorithms, such as a very fine crack or internal bubble. In this case, the system will use the location of the thermodynamic feature as the center to perform a more refined search in the corresponding visible light image region, thereby discovering new geometric discontinuities.

[0054] Specifically, higher-sensitivity edge detection algorithms refer to those employing operators such as the Canny operator or the LoG (Laplacian of Gaussian) operator, capable of precisely capturing subtle edges. The Canny operator is particularly suitable due to its non-maximum suppression and double-threshold hysteresis capabilities, effectively connecting discontinuous, weak edges. The search region of this algorithm is a locally adaptive window centered on the centroid of the thermodynamic feature, typically set to 1.5 to 2 times the equivalent diameter of the thermodynamic feature to ensure complete coverage of potential geometric defects. Regarding threshold settings, to achieve high sensitivity, the system does not use a globally fixed threshold. Instead, it dynamically calculates a lower double threshold based on the image gradient magnitude distribution within the local search region. For example, the high threshold is set to the 70th percentile of the local gradient magnitude, and the low threshold to the 30th percentile. This allows for the detection of subtle edges with extremely low contrast that might be overlooked in conventional processing, such as hidden cracks or scratches.

[0055] Specifically, the process of predicting the first morphological feature of geometric candidate features in infrared thermal imaging images includes: First, according to the type of geometric candidate feature, query the corresponding physical parameters in the association model, such as the change of thermal conductivity coefficient and stress concentration coefficient; then, combining the geometric dimensions of the feature and the parameters of the controlled thermal excitation, calculate the theoretical temperature field distribution sequence over time using a simplified thermal conduction finite element model or empirical formula, and this sequence constitutes the first morphological feature.

[0056] The process of searching and matching the first morphological feature among the thermodynamic candidate features then involves comparing the predicted first morphological feature, such as a dynamic temperature distribution template, with all the actually extracted thermodynamic candidate features one by one.

[0057] Specifically, the matching process compares not only the similarity of spatial morphology but also the similarity of temporal evolution curves. A collaborative confidence score is obtained by calculating a comprehensive similarity index.

[0058] The formula is as follows:

[0059] Among them, the Sco collaborative confidence score ranges from (0, 1). The higher the score, the higher the matching degree, and the greater the possibility that the feature is a real defect. These are the spatial morphology weight and the temporal evolution weight, used to adjust the importance of the two dimensions in the final score. These two weights are non-negative and typically satisfy the following conditions: .

[0060] For example, for crack-like defects, their temporal evolution characteristics may be more discriminative than their spatial shape, therefore, a setting can be made. .

[0061] The key to allocating the two weights lies in flexibly adjusting them based on the physical characteristics of the defects and the quality of the data.

[0062] Specifically, the allocation rules prioritize defects based on their type. For example, for crack-type defects, their temporal evolution characteristics are more discriminative than their spatial morphology, thus requiring a higher weight for temporal evolution. Conversely, for defects like pores or bubbles, their spatial morphology characteristics are more critical, necessitating a higher weight for spatial morphology. Simultaneously, the rules consider the quality of the input data. When the signal-to-noise ratio of infrared thermal imaging data is low or the sampling rate is insufficient, the system reduces the weight of unreliable temporal evolution and relies more heavily on relatively stable visible light spatial morphology characteristics. Through this dual dynamic adjustment mechanism based on defect type and data quality, and by consistently ensuring that the sum of the two weights equals 1, the system can flexibly adapt to various complex operating conditions, thereby maximizing the accuracy and robustness of the collaborative confidence score.

[0063] It is a spatial morphological similarity score. It is the temporal evolution similarity score.

[0064] in,

[0065] It is the set of pixels representing the first morphological feature predicted based on geometric candidate features. It is a set of pixels extracted from infrared thermal imaging images as candidate thermodynamic features. The Hausdorff distance is the bidirectional distance between two sets of points, which measures the maximum degree of mismatch between two shapes. The smaller the distance, the more similar the shapes. This is a spatial normalization constant, which, for example, can be set to the estimated length of the defect or the number of pixels along the image diagonal, to normalize the distance values ​​so that the input to the exponential function is within a reasonable range. This exponential function ensures that when the Hausdorff distance is 0 (perfect match), As distance increases Approaching 0.

[0066]

[0067] It is a curve showing the temperature change over time at the center point of the predicted first morphological feature. It is the curve showing the change in temperature at the center point of the observed thermodynamic candidate feature over time. It is the dynamic time-warped distance between two curves. It calculates the similarity between two curves by finding an optimal time-aligned path, is insensitive to small time delays in the heat conduction process, and is more robust than the simple Euclidean distance. It is a time normalization constant, which can be set to the peak temperature difference or maximum temperature change rate of the prediction curve to normalize the DTW distance value.

[0068] Similarly, the exponential function ensures that S2=1 when the DTW distance is 0 (perfect match).

[0069] Through this comprehensive formula, the system can perform dual physical verification on a geometric candidate feature: it not only looks like a defect in space but also behaves like a defect in temporal behavior. Only features that pass both tests simultaneously can obtain a high co-confidence score and thus be confirmed as high-confidence defect verification features.

[0070] Specifically, the search and matching process can employ feature-based similarity measurement algorithms, such as Hausdorff distance to measure the similarity of spatial shapes, and dynamic time warping to measure the similarity of time series curves. Ultimately, the collaborative confidence score is a weighted sum of spatial and temporal similarities.

[0071] Integrating the second and third morphological features to obtain defect verification features involves merging the feature set that passed forward verification with the new feature set discovered through reverse verification, removing any potential overlap. This integration process creates a more complete defect feature set, which includes not only defects that can be detected by conventional algorithms and have been physically verified, but also hidden defects that are difficult for conventional algorithms to detect and are deduced from physical anomalies, thereby maximizing the recall and reliability of the detection. Each of the final output defect verification features includes its verification method and the final collaborative confidence score.

[0072] S203: Reconstruct the defect verification features to obtain defect quantification information.

[0073] Reconstruction refers to converting the verified features into corresponding three-dimensional defect quantification information.

[0074] The aforementioned quantitative information is used to characterize the three-dimensional geometric parameters, thermodynamic characteristic parameters, and defect type identifiers of the defects.

[0075] In some embodiments, for defect verification features, three-dimensional reconstruction is performed to obtain several defects and the defect quantification information corresponding to each defect. The three-dimensional reconstruction adopts the multi-view photometric stereo method and uses the stress field gradient inverted from the infrared thermal imaging image as prior knowledge to guide and constrain the solution process of the three-dimensional morphology.

[0076] Specifically, multi-view image acquisition involves controlling a robotic arm to move an industrial camera around the edge of the glass under test and capturing high-resolution images from at least three different preset viewpoints. At each viewpoint, a ring light source composed of multiple LEDs is controlled to illuminate the glass edge sequentially from different directions, such as left, right, top, and bottom, thereby obtaining a sequence of images under multiple illumination conditions at each viewpoint.

[0077] Preliminary calculation of surface normals: For each viewing angle, based on the fundamental principle of photometric stereochemistry—that the brightness of a point on the object's surface is proportional to the dot product of the light source direction and the surface normal vector at that point—the surface normal vector field of the visible region under that viewing angle is calculated. Due to the transparency of glass, this step may produce ambiguity or errors in flat surface areas, but at geometrically discontinuous edge defects, relatively reliable normal information can still be calculated due to light scattering and reflection.

[0078] Stress field gradient prior knowledge extraction: Simultaneously, the infrared thermal imaging image sequence of the same region is processed. The temperature field distribution of the region is inverted using a physical information neural network, and then the gradient vector field of the temperature field is calculated using numerical differentiation. This gradient field intuitively shows the direction and rate of heat conduction, exhibiting drastic and regular changes at defects, which directly reflects the influence of the defect's geometry on the physical field.

[0079] Solving 3D topography under physical constraints: fusing the surface normal fields initially calculated from multiple perspectives. During the fusion process, the infrared stress field gradient is introduced as strong prior knowledge for constraint.

[0080] Specifically, when constructing the objective function, in addition to including a term that minimizes the normal consistency error, a physical constraint term is added. The purpose of this physical constraint term is to penalize geometric solutions that do not conform to the stress field gradient distribution.

[0081] For example, if the infrared gradient field shows a linear high-gradient region, the optimization process will be guided to generate a 3D morphology with sharp grooves, i.e., cracks, rather than a gentle convexity. By solving this optimization problem constrained by physical laws, a unique and more realistic, unambiguous 3D point cloud or mesh model can be obtained.

[0082] Defect quantification information extraction: From the final generated 3D model, the region corresponding to each defect verification feature is accurately segmented, and its 3D dimensions are calculated, including but not limited to: depth, width, length, volume, and projected area in the edge direction. These data constitute the defect quantification information.

[0083] Alternatively, besides the multi-view photometric stereo method, this scheme can also use structured light measurement as the basis for 3D reconstruction. For example, a sinusoidal fringe pattern can be projected onto the edge of the glass, and 3D information can be obtained by analyzing the deformation of the pattern. However, due to the transparency and specular reflection properties of glass, the effect of a single structured light method is limited. Therefore, the stress field gradient obtained from infrared thermal imaging inversion is also needed as prior knowledge to constrain and correct phase unwrapping errors and measurement noise generated by the structured light algorithm on transparent materials, thereby achieving high-precision 3D reconstruction.

[0084] S204: Perform calculations on the defect quantification information to obtain defect prediction data.

[0085] In some embodiments, a defect agent is constructed based on the defect and the defect quantification information corresponding to each defect. Based on the defect agent in a simulation environment, the stress intensity factor is calculated using finite element analysis. Based on the stress intensity factor, the expansion path, rate, and remaining safe lifetime of the defect agent are predicted. The defect and the defect quantification information corresponding to each defect, the expansion path, rate, and remaining safe lifetime of the defect agent are integrated to obtain defect prediction data.

[0086] Specifically, the computation process is a physical simulation flow based on digital twins and fracture mechanics. First, the system takes the precise three-dimensional quantification information of each defect obtained in S203 as input, and instantiates a corresponding defect agent in a virtual glass model that has the same material properties and boundary conditions as the real glass under test, such as stress conditions and fixing methods.

[0087] The system then applies the anticipated external loads to the virtual model containing the defective agent and runs a finite element analysis solver. The solver calculates the stress field distribution at the tip of the defective agent and extracts the key mechanical parameter—the stress intensity factor. Finally, based on fracture mechanics criteria, such as the Paris-Erdogan fatigue crack propagation law or the Griffith brittle fracture criterion, the system uses the calculated stress intensity factor to predict the evolution of the defect during future service.

[0088] The defect agent is a digital twin model containing all the key geometric and physical information of a real defect. It is not merely a three-dimensional geometric object, but a behavioral entity within a physical simulation environment. It encapsulates the defect's geometric, positional, and material interface properties. By constructing the defect agent, abstract defect data can be transformed into an object capable of physical analysis and dynamic simulation.

[0089] The stress intensity factor is a core physical quantity in fracture mechanics, used to quantitatively describe the intensity of the stress field near the crack tip. Since the crack tip is theoretically a singularity where stress tends to infinity, the traditional concept of stress becomes invalid. The stress intensity factor, however, bypasses this singularity, characterizing the magnitude of the "force" driving crack propagation. When the stress intensity factor K reaches or exceeds the material's critical value, the crack will propagate unstably, leading to material fracture. Therefore, it is a key criterion for assessing structural safety and predicting crack life.

[0090] Specifically, the process of calculating the stress intensity factor using finite element analysis is as follows: In the virtual glass model, cracks are precisely implanted based on the geometric information of the defective intelligent agent; The mesh at the crack tip region is extremely refined to capture drastic stress changes; Apply loads and constraints that correspond to the actual working conditions; Run FEA to solve for the stress and displacement fields of the entire model; Post-processing techniques, such as the J-integral method or displacement extrapolation method, are used to accurately extract the values ​​of stress intensity factors K_I (opening type), K_II (slipping type), and K_III (tearing type) from the calculation results at the crack tip.

[0091] Specifically, the defects and their corresponding quantitative information, the expansion path and rate of the defect agent, and the remaining safe lifetime are integrated to generate a structured, decision-oriented comprehensive data package from the defect prediction data.

[0092] It integrates the following information, indexed by each defect: Static quantification information: defect ID, type, 3D dimensions, and location coordinates; Dynamic prediction information: propagation path: a vector sequence or probability contour map showing the most likely direction of crack propagation; propagation rate: the rate at which crack length increases with time or number of cycles under a specific load (e.g., micrometers per 10,000 cycles); Remaining safe lifetime: the predicted time or number of cycles from the current moment until the crack reaches the critical size (i.e., fracture) based on the current defect size and propagation rate.

[0093] S205: Integrate defect prediction data and defect quantification information to obtain defect detection data.

[0094] In some embodiments, defect quantification information and defect prediction data are structurally integrated to generate a comprehensive inspection report that includes defect classification, location coordinates, three-dimensional dimensions, confidence score, evolution trend prediction, and remaining safe lifetime. This comprehensive inspection report is then output as the final defect inspection data.

[0095] Specifically, the integration process first correlates the static quantitative information of each defect with dynamic predictive information, such as the expansion path and remaining lifespan, to construct a unique digital twin file spanning its entire lifecycle. Subsequently, the system's built-in decision engine assesses the risk of each file based on preset quality standards, ultimately making an intelligent judgment of "qualified," "reworked," or "scrapped" for the entire piece of glass. Finally, all files and judgment results are integrated to generate a comprehensive inspection report with rich graphics and text. This report not only visually displays the current state of the defect but also profoundly reveals its future evolution trend and potential risks, thus constituting the final defect inspection data with decision-making guidance value. As the final output, it can be pushed in real-time to the human-machine interface for operator review, uploaded to the manufacturing execution system for quality traceability, or even directly drive the sorting mechanism to perform automated rejection operations.

[0096] like Figure 3 The diagram illustrates the structure of a machine vision-based glass edge defect detection system. This system may include: Acquisition module 301: used to acquire the defect characteristics of a glass calibration plate containing standard defect samples, where standard defect samples refer to defects such as cracks, notches, chipped edges, and bubbles; Verification module 302: used to verify defect features and obtain defect verification features. Defect verification features are used to characterize high-confidence defects that have been cross-verified by physical laws. Reconstruction module 303: Used to reconstruct defect verification features to obtain defect quantification information; Calculation module 304: Used to perform calculations on defect quantification information to obtain defect prediction data; Output module 305: Used to integrate defect prediction data and defect quantification information to obtain defect detection data.

[0097] This application embodiment can divide the machine vision-based glass edge defect detection system into functional modules according to the above method embodiment. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0098] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary embodiment couples a storage medium to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in a network device. Alternatively, the processor and storage medium can exist as discrete components in the network device. In the above embodiments, implementation can be entirely or partially achieved through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product. A computer program product includes one or more computer programs or instructions. When a computer program or instruction is loaded and executed on a computer, all or part of the processes or functions of the embodiments of this application are performed. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable module. The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, a computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video disc (DVD); or a semiconductor medium, such as a solid-state drive (SSD). The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0099] Since the vehicle control device in the embodiments of the present invention can be applied to the above method, the technical effects it can achieve can also be referred to the above method embodiments, and the embodiments of the present invention will not be repeated here. The above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims. The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor, thereby enabling the processor to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a network device. Of course, the processor and storage medium can also exist as discrete components in the network device. In the above embodiments, implementation can be entirely or partially achieved through software, hardware, firmware, or any combination thereof. When implemented in software, it can be entirely or partially implemented in the form of a computer program product. A computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, all or part of the processes or functions of the embodiments of this application are performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable modules. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, a computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.The usable medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD). The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting glass edge defects based on machine vision, characterized in that, The method includes: Obtain the defect characteristics of a glass calibration plate containing standard defect samples, wherein the standard defect samples refer to defects of the type such as cracks, notches, chipping, and bubbles; The defect features are verified to obtain defect verification features, which are used to characterize high-confidence defects that have been cross-verified by physical laws; the verification is used to verify whether the defect features conform to the spatial morphological features and temporal evolution laws of real defects. The defect verification features are reconstructed to obtain defect quantification information; the reconstruction refers to converting the verified features into corresponding three-dimensional defect quantification information; the quantification information is used to characterize the three-dimensional geometric parameters, thermodynamic characteristic parameters and defect type identifier of the defect; The defect quantification information is processed to obtain defect prediction data; The defect prediction data and the defect quantification information are integrated to obtain defect detection data.

2. The method according to claim 1, characterized in that, The acquisition of defect features from a glass calibration plate containing standard defect samples includes: Acquire an image of the glass calibration plate containing the standard defect samples; Based on the image of the glass calibration plate containing standard defect samples, a visible light image and an infrared thermal image of the edge region of the glass to be tested are extracted. The visible light image is a high-resolution visible light image of the edge region of the glass, and the thermal image sequence is a dynamic infrared thermal image of the edge region of the glass under controlled thermal excitation. The visible light image is processed by an edge-aware network to obtain geometric candidate features of the glass edge in the visible light image; The infrared thermal imaging image is processed by a physical information neural network embedding the heat conduction equation to obtain thermodynamic candidate features; The geometric candidate features and the thermodynamic candidate features are used as the defect features.

3. The method according to claim 2, characterized in that, The extraction of visible light and infrared thermal images of the edge region of the glass under test based on the image of the glass calibration plate containing standard defect samples includes: Multimodal image acquisition is performed on the glass calibration plate containing standard defect samples to calibrate the spatial transformation matrix and time synchronization parameters between the industrial camera and the infrared camera; For the glass calibration plate containing standard defect samples, a calibrated spatial transformation matrix and time synchronization parameters are used to synchronously acquire visible light images and infrared thermal imaging images under controlled thermal excitation.

4. The method according to claim 2, characterized in that, The verification of the defect features to obtain defect verification features includes: For each of the geometric candidate features, a first morphological feature of the geometric candidate feature in the infrared thermal imaging image is predicted according to a pre-calibrated association model. The first morphological feature is used to characterize the stress field morphology that the geometric candidate feature should present in the infrared thermal imaging image. Search and match the first morphological feature among the thermodynamic candidate features, and calculate the collaborative confidence score of the matching degree between each thermodynamic candidate feature and the corresponding first morphological feature. The first morphological feature is filtered based on the collaborative confidence score to obtain the second morphological feature; For each thermodynamic candidate feature, an enhanced search is performed in the visible light image at the corresponding location to obtain the third morphological feature. The enhanced search is a local region search centered on the thermodynamic feature location using an edge detection algorithm. The second morphological feature and the third morphological feature are integrated to obtain the defect verification feature.

5. The method according to claim 1, characterized in that, The reconstruction of the defect verification features yields several defects and corresponding defect quantification information for each defect, including: For the defect verification features, a three-dimensional reconstruction is performed to obtain several defects and the defect quantification information corresponding to each defect. The three-dimensional reconstruction adopts the multi-view photometric stereo method and uses the stress field gradient retrieved from the infrared thermal imaging image as prior knowledge to guide and constrain the solution process of the three-dimensional morphology.

6. The method according to claim 1, characterized in that, The step of performing calculations on the defects and the defect quantification information corresponding to each defect to obtain defect prediction data includes: Based on the aforementioned defects and the defect quantification information corresponding to each defect, a defect intelligent agent is constructed; Based on the aforementioned defective intelligent agent in a simulated environment, the stress intensity factor is calculated using finite element analysis. Based on the stress intensity factor, the expansion path, rate, and remaining safe lifetime of the defective agent are predicted. The defect and its corresponding defect quantification information, the expansion path and rate of the defect agent, and the remaining safe lifetime are integrated to obtain defect prediction data.

7. The method according to claim 1, characterized in that, The defect detection data obtained by integrating the defect prediction data and the defect quantification information includes: The defect quantification information and the defect prediction data are structured and integrated to generate a comprehensive inspection report that includes defect classification, location coordinates, three-dimensional dimensions, confidence score, evolution trend prediction, and remaining safe life. This comprehensive inspection report is then output as the final defect inspection data.

8. A machine vision-based glass edge defect detection system, the system comprising: Acquisition module: used to acquire the defect characteristics of a glass calibration plate containing standard defect samples, wherein the standard defect samples refer to defects such as cracks, notches, chipped edges, and bubbles; Verification module: used to verify the defect features to obtain defect verification features, which are used to characterize high-confidence defects that have been cross-verified by physical laws; Reconstruction module: used to reconstruct the defect verification features to obtain defect quantification information; The calculation module is used to perform calculations on the defect quantification information to obtain defect prediction data. Output module: used to integrate the defect prediction data and the defect quantification information to obtain defect detection data.

Images