Glass product surface concave-convex detection system and method based on artificial intelligence

Abstract

The invention discloses a glass product surface concave-convex detection system and method based on artificial intelligence, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a surface image of a glass product, and inputting the surface image into a constructed cross-material transfer learning network to generate an initial defect classification map; using a polarization interferometer to collect an interference fringe image according to the initial defect classification graph, establishing a double-branch structure convolutional neural network, inputting the interference fringe image into the double-branch structure convolutional neural network, and outputting a surface deformation field; a glass product is placed under a terahertz time-domain spectrometer, a concave-convex area is screened based on a surface deformation field, pixel points with height absolute values exceeding a defect threshold value in a surface deformation field graph are screened in the concave-convex area by setting the defect threshold value, and adjacent pixel points are clustered and merged to generate the surface deformation field image. Capturing a time domain signal of the concave-convex area by using a photoelectric detector; according to the method, the limitation of scarcity of glass samples is broken through by constructing the cross-material transfer learning network, and the generalization ability is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based glass product surface concave-convex detection system and method. Background Art

[0002] Surface bump detection technology for glass products is one of the core research directions in the field of modern industrial quality control. Its development has evolved from traditional manual detection to automated optical detection. The rapid development of artificial intelligence technology, especially the application of convolutional neural networks and transfer learning, has provided new solutions for defect classification and feature extraction, and promoted the further improvement of detection accuracy and automation.

[0003] However, there are still many problems with existing methods for detecting bumps on glass surfaces. First, traditional machine vision methods and single optical detection technologies usually require a large amount of training data support for specific materials, while glass defect samples are often scarce due to production process restrictions, resulting in limited generalization capabilities and difficulty in adapting to diverse defect patterns. Second, traditional technologies focus on two-dimensional recognition of glass surface defects and lack the ability to comprehensively analyze multi-scale features such as macro bumps and micro cracks. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a glass product surface concave-convex detection method based on artificial intelligence to solve the problems of scarcity of glass defect samples and lack of comprehensive analysis capabilities for multi-scale features such as macro concave-convex and micro cracks.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for detecting concave-convex surfaces of glass products based on artificial intelligence, which comprises: Collect surface images of glass products and input them into the constructed cross-material transfer learning network to generate an initial defect classification map; Collect interference fringe images using a polarization interferometer according to the initial defect classification map; A dual-branch convolutional neural network is established, the interference fringe image is input, and the surface deformation field is output; The glass product is placed under a terahertz time-domain spectrometer, and the concave-convex area is screened based on the surface deformation field. The concave-convex area is screened by setting a defect threshold and the pixel points whose absolute height value exceeds the defect threshold in the surface deformation field map are screened, and the adjacent pixel points are generated by clustering and merging, and the time domain signal of the concave-convex area is captured by a photodetector; The time domain signal is input into a pre-established time domain signal processing dedicated network to generate time domain features, frequency domain signals and frequency domain features, and the time domain features and frequency domain features are fused to obtain the defect signal. The defect signal and frequency domain signal are input into the constructed multi-scale defect association model to generate the final defect distribution map.

[0007] As a preferred solution of the artificial intelligence-based glass product surface concave-convex detection method of the present invention, wherein: generating the initial defect classification map refers to constructing a cross-material transfer learning network; ResNet-50, a cross-material transfer learning network, is used to extract multiple features from a training set of metal and ceramic defect images with defect annotations to generate metal feature vectors and ceramic feature vectors, which represent the defect modes of metal and ceramic, respectively. At the same time, features are extracted from the surface image of the glass to be tested to obtain a glass feature vector. The metal and ceramic defect images are training data with defect annotations. Through training, the cross-material transfer learning network learns cross-material defect knowledge; The metal feature vector and the ceramic feature vector are spliced ​​to generate a joint feature vector, and then the joint feature vector is spliced ​​with the glass feature vector to generate a cross-material fusion feature; The fully connected layer and SENet attention mechanism in the cross-material transfer learning network are used to process cross-material fusion features and generate glass defect features. The defect classifier of the cross-material transfer learning network processes the glass defect features to generate an initial defect classification map.

[0008] As a preferred solution of the artificial intelligence-based glass product surface concave-convex detection method of the present invention, wherein: generating glass defect features refers to dimensional conversion of cross-material fusion features through the fully connected layer of the cross-material transfer learning network; The SENet attention mechanism is used to perform weighted optimization on cross-material fusion features to generate glass defect features.

[0009] As a preferred solution of the method for detecting concave-convex surfaces of glass products based on artificial intelligence of the present invention, wherein: collecting interference fringe images using a polarization interferometer according to the initial defect classification map specifically includes the following steps; The glass is placed on a polarization interferometer, the coordinates of the defect area are extracted according to the initial defect classification map, and the interference fringe image is located and collected based on the coordinates of the defect area.

[0010] As a preferred solution of the glass product surface concave-convex detection method based on artificial intelligence of the present invention, wherein: the dual-branch structure convolutional neural network includes an encoder, a physical constraint branch and a decoder, and the physical constraint branch includes an optical path difference calculation unit; By inputting the interference fringe image into the encoder, feature extraction is performed to obtain spatial features; The physical constraint branch obtains phase information by performing Fourier transform on the spatial features, generates the optical path difference through the optical path difference calculation unit according to the phase information, expands the optical path difference to the same tensor as the spatial features, splices it with the spatial features, and then generates the surface deformation field through upsampling.

[0011] As a preferred solution of the glass product surface concave-convex detection method based on artificial intelligence of the present invention, wherein: the time domain signal is input into a pre-established time domain signal processing dedicated network to generate a frequency domain signal, and the time domain signal and the frequency domain signal are fused to obtain a defect signal, which specifically includes the following steps; The time domain signal processing dedicated network is composed of a time domain processing unit, a frequency domain processing unit, a fusion processing unit and a sequence processing unit; The time domain signal is processed by a time domain processing unit to obtain a time domain feature; The frequency domain processing unit processes the time domain features through short-time Fourier transform to obtain frequency domain signals and frequency domain features, and enhances the frequency domain features using the attention mechanism to obtain enhanced frequency domain features; The time domain features and the enhanced frequency domain features are fused through the fusion processing unit to generate fusion features, and the fusion features are processed by the sequence processing unit to output defect signals.

[0012] As a preferred solution of the glass product surface concave-convex detection method based on artificial intelligence of the present invention, wherein: the generating of the final defect distribution map specifically includes the following steps; The defect signal and frequency domain signal are input into the constructed multi-scale defect association model, and the final defect distribution map is generated through time domain analysis, frequency domain analysis and defect quantification.

[0013] In a second aspect, the present invention provides a glass product surface concave-convex detection system based on artificial intelligence, comprising: The initial defect module collects surface images of glass products and inputs them into the constructed cross-material transfer learning network to generate an initial defect classification map; A data acquisition module collects interference fringe images using a polarization interferometer according to the initial defect classification diagram; The surface deformation module establishes a double-branch convolutional neural network, inputs the interference fringe image, and outputs the surface deformation field; The time domain capture module places the glass product under the terahertz time domain spectrometer, and screens the concave-convex area based on the surface deformation field. The concave-convex area screens the pixel points whose absolute height value exceeds the defect threshold in the surface deformation field map by setting the defect threshold, and generates by clustering and merging the adjacent pixel points, and uses a photodetector to capture the time domain signal of the concave-convex area; The final defect module inputs the time domain signal into a pre-established time domain signal processing dedicated network to generate time domain features, frequency domain signals and frequency domain features, and fuses the time domain features with the frequency domain features to obtain the defect signal. The defect signal and the frequency domain signal are input into the constructed multi-scale defect association model to generate the final defect distribution map.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the artificial intelligence-based glass product surface convexity and concaveness detection method as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the artificial intelligence-based glass product surface convexity and concaveness detection method as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: by constructing a cross-material transfer learning network, the rich defect samples of metals and ceramics (such as scratches and cracks) are used to make up for the lack of glass data, and the key features (such as texture) are highlighted through the attention mechanism, and a classification map containing the probabilities of concave, convex, and defect-free is generated, which is used to quickly screen the defective areas on the glass surface. It breaks through the limitation of the scarcity of glass samples, enhances the generalization ability, especially improves the sensitivity to tiny defects, reduces the data collection cost and supports multi-material detection scenarios. In addition, through the time domain signal processing network and the multi-scale defect joint model, the time domain features and frequency domain features are fused to generate a defect distribution map, which is convenient for rapid decision-making and defect classification, and improves production efficiency and product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a flow chart of the method for detecting unevenness on the surface of a glass product based on artificial intelligence in Example 1.

[0019] Figure 2 This is a diagram of the network structure of cross-material transfer learning in Example 1.

[0020] Figure 3 Schematic diagram of the dual-branch convolutional neural network in Example 1.

[0021] Figure 4 This is a schematic diagram of the system modules in Example 1. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0025] Example 1, reference Figure 1~Figure 4 This embodiment provides a method for detecting concave-convex surfaces of glass products based on artificial intelligence, comprising the following steps: S1. Collect surface images of glass products and input them into the constructed cross-material transfer learning network to generate an initial defect classification map.

[0026] The specific steps include: Use a high-resolution industrial camera (such as a 20-megapixel CCD camera) to shoot the surface of the glass product to be tested from multiple angles to collect real-time surface images. The image resolution is standardized and fixed to 1024×1024 pixels to ensure that all potential defect areas on the glass surface are covered. When shooting, the camera lens maintains an angle of 30° to 60° with the glass surface to capture defect features such as pits, bumps or scratches under different lighting conditions. Then, the collected images are preprocessed to optimize the quality. First, MATLAB is used to convert the color image into a grayscale image to reduce the computational complexity brought by the color channel while retaining the texture and contrast information of the defect. Next, the grayscale image is filtered by applying the Laplace variance. The specific steps of the screening are: the high-frequency details (such as edges and textures) of each pixel are calculated by the Laplace operator, and the variance value of the entire image is counted. The higher the variance value, the clearer the image; otherwise, it may be blurred or interfered. Set an empirical threshold (for example, 150) to retain only images with a variance higher than this value to ensure that the data input into the AI ​​model are all high-quality data. The clear grayscale images that have passed the screening are denoised using the Gaussian blur algorithm: first, the clear grayscale images are smoothed, and then the interference of environmental noise (such as dust reflection and sensor noise) on defect detection is eliminated. After denoising, the pixel values ​​of the glass surface image are normalized (i.e., in the range of 0 to 1).

[0027] S1.1. Now we start to build a cross-material transfer learning network. First, prepare the data. Because it is a cross-material transfer learning network, we need to collect image data of metal, ceramic and glass. Among them, metal and ceramic are image data with defects, such as scratches and pits on the metal surface, cracks on the ceramic surface, etc., and are marked with defect types (concave, convex, no defects).

[0028] In order to utilize the image data of metals and ceramics, ResNet-50 is used to extract features. The image data of metals and ceramics are processed by layers of convolution and pooling. Finally, the feature vector of each image data is generated by global average pooling to obtain metal feature vectors and ceramic feature vectors, which represent the defect modes of the two materials respectively.

[0029] Start to build a cross-material transfer learning network structure. First, concatenate the metal feature vector and the ceramic feature vector into a joint feature vector, use a fully connected layer to convert the joint feature vector to the glass scene, add the SENet attention mechanism, and enhance the focus on defect-related features. For example, if the texture of the scratch is important, it will give the scratch texture feature a higher weight. The specific process is: perform global statistics on each feature channel (metal and ceramic), calculate the average value of all pixels in each channel, and obtain a value representing the global importance of the channel. For example, a high average value of the scratch texture channel may indicate that the scratch texture channel frequently activates defect areas in the image. The global information of the feature channel is analyzed through a two-layer fully connected neural network, and the weight is dynamically allocated. For example, the scratch texture channel may obtain a weight of 0.9, while the background channel only obtains a weight of 0.1. If during the training process, it is found that the scratch texture feature frequently appears when correctly classifying the glass pit defect (such as the gradient change of the pit edge is similar to the scratch texture), the SENet attention mechanism will adjust the weight to make the scratch texture feature channel dominate in subsequent calculations. A dual-path framework is established for the cross-material transfer learning network, that is, there are two branches. The first branch is the material identifier, which is a three-layer fully connected network with a specific structure: the first layer receives the input feature vector, outputs 512 neurons, and uses the ReLU activation function; the second layer receives 512 neurons, outputs 128 neurons, and uses the ReLU activation function; the third layer receives 128 neurons, outputs 3 neurons (corresponding to the probabilities of metal, ceramic, and glass), and uses the Softmax activation function to determine whether the input features come from metal, ceramic, or glass, to ensure that the input features are not biased towards a certain material; the second branch is the defect classifier, which is a four-layer convolutional network that converts the determined features into defect types (concave, convex, or defect-free) and outputs a defect probability map for each image.

[0030] After the cross-material transfer learning network structure is built, training begins. First, the training goal must be set. In layman's terms, the transfer learning network must learn two things: one is to make the joint feature vector applicable to all materials, and the other is to accurately classify defects. Since ResNet-50 is included in the cross-material transfer learning network, the glass image is directly input into the cross-material transfer learning network to obtain the glass features. The training content of the material discriminator is to input the joint feature vector and the glass features into the material discriminator. The material discriminator outputs three-category probabilities (corresponding to metal, ceramic, and glass, respectively) to determine the material source of the input features. The material discriminator is adjusted using cross entropy loss, where the true label is the material category corresponding to the feature (for example, 0 for metal, 1 for ceramic, and 2 for glass); the training content of the defect classifier is to predict the defect type (concave, convex, or defect-free) of the image based on the input glass defect features, and compare it with the manually labeled defect classification. The FocalLoss loss function is used to adjust the defect classifier. In order to make the cross-material transfer learning network make the joint feature vector applicable to all materials, an adversarial process is required. In layman's terms, the material identifier wants to distinguish materials, while the main network (fully connected layer + SENet) tries to make the joint feature vector difficult to distinguish, forming an adversarial process. Finally, the joint feature vector is applicable to all materials. Use the Adam optimizer to update the cross-material transfer learning network, and check the loss reduction each time until the requirements are met. After the cross-material transfer learning network is trained, it is verified with data containing metals, ceramics, and glass. When the verification effect meets the requirements, the establishment of the entire cross-material transfer learning network is completed.

[0031] In practice, the preprocessed glass surface image is input into the ResNet-50 of the cross-material transfer learning network to extract features and obtain the glass feature vector, which describes the texture and possible defect information of the glass surface. During the training phase, the cross-material transfer learning network uses ResNet-50 to extract multiple feature vectors from the training set of metal and ceramic defect images with defect annotations to generate metal feature vectors and ceramic feature vectors, representing the defect modes of metal and ceramic respectively (such as scratches and cracks). Through adversarial training, the network learns cross-material defect knowledge so that the joint feature vector generated by splicing the metal feature vector and the ceramic feature vector contains the common defect modes of metal and ceramic. Subsequently, the joint feature vector is spliced ​​with the glass feature vector to generate a cross-material fusion feature. The cross-material fusion feature is input into the fully connected layer, and the dimension is converted through matrix multiplication and the bias vector (initial value is 0) to obtain the intermediate feature of the glass.

[0032] The intermediate features of the glass are input into the SENet attention mechanism module. SENet compresses the intermediate features of the glass into a single value through global average pooling, and then generates attention weights (values ​​between 0 and 1) through two layers of fully connected networks (the first layer uses the ReLU activation function to reduce the dimension, and the second layer uses the Sigmoid activation function to increase the dimension). The attention weights are multiplied by the intermediate features of the glass channel by channel, and the optimized glass defect features are output, highlighting the features related to the defects (such as scratch textures and concave and convex shapes).

[0033] Preferably, the SENet attention mechanism optimizes the expressiveness of defect features, generates attention weights through global average pooling and two-layer fully connected networks, enhances the defect-related parts of the glass intermediate features (such as scratch textures, concave and convex shapes), amplifies features with high attention weights, suppresses information with low attention weights, and makes the optimized glass defect features more focused on the defect area. The SENet attention mechanism improves the sensitivity of defect detection, especially for small or complex defects (such as fine cracks), which can be more accurately identified and located, reducing misjudgments.

[0034] Use deconvolution to map the optimized glass defect features back to two-dimensional space, generate a glass defect feature map, and input it into the defect classifier. The defect classifier contains a four-layer convolutional network, which gradually reduces the dimension through the first three layers (the first and second layers extract the texture and shape of the defect, and the third layer fuses the channel information). The fourth layer outputs the probabilities of concave, convex, and defect-free through Softmax activation to generate the initial defect classification map. Use PyTorch to convert the highest probability category of each pixel in the probability map into a classification label to obtain a classification map, and use OpenCV 4.9.0 to visualize it. The convex area is marked in red, the concave area is marked in blue, and the defect-free area is marked in green, with the probability value (such as 0.9) marked. Finally, save the initial defect classification map.

[0035] Use PyTorch to convert the highest probability category of each pixel in the probability map into a classification label to obtain a classification map. Then use visualization software, such as OpenCV 4.9.0, to visualize the classification map, marking convex areas in red, concave areas in blue, and defect-free areas in green, and annotate the probability value in each area (for example, 0.9 indicates high confidence). Finally, save the generated initial defect classification map.

[0036] It is further explained that by introducing defect data of other materials such as metals and ceramics and using ResNet-50 to extract features, the cross-material transfer learning network can learn richer defect patterns (such as pits, scratches, cracks, etc.). In industrial production, it is difficult to obtain a large number of glass defect samples. The introduction of defect data of other materials reduces the cost of data collection and improves the generalization ability of the cross-material transfer learning network, which is suitable for defect detection of various materials.

[0037] S2. Collect interference fringe images using a polarization interferometer according to the initial defect classification map.

[0038] Specifically, the following steps are included: Place the glass product to be tested in a polarization interferometer. The interferometer used is equipped with a He-Ne laser (wavelength λ=632.8nm, output power 5mW), whose monochromatic light characteristics ensure the clarity and stability of the interference fringes. Fix the glass product to be tested on a precision moving platform, and control the movement through a servo motor, with high precision and programmable position adjustment.

[0039] Obtain the initial defect classification map, and use OpenCV 4.9.0 software to extract the coordinates of the potential defect area (for example, the center point of the red area [x1, y1], the center point of the blue area [x2, y2]). The coordinates of the defect area are converted into physical movement instructions for the precision moving platform through the control program, guiding the interferometer to focus on the defect area (such as convexity or concavity) instead of scanning the entire glass surface. Use a high-sensitivity CCD camera to collect interference fringe images at a frequency of 10 frames / second (size 2048×2048, pixel value 0-255, single-channel grayscale image). The field of view covers an area of ​​approximately 10cm×10cm on the glass surface. Each scan targets a defect area in the initial defect classification map, and collects approximately 5 interference fringe images to cover all marked defect areas. At the same time, the collected interference fringe images are compared with the wavelength =632.8nm and the estimated optical path difference (optical path difference refers to the path difference between the measurement light and the reference light when the laser is reflected on the glass surface (in nanometers). The optical path difference is directly related to the height change of the glass surface: if the surface is convex, the distance traveled by the light will become longer (the optical path difference is positive); if the surface is concave, the distance traveled by the light will become shorter (the optical path difference is negative), forming light and dark stripes. The optical path difference is an estimated value and is set to 0) is combined together by combining the grayscale value of the collected interference fringe image with the wavelength =632.8nm and the estimated optical path difference are superimposed to form a three-channel array. The first channel is the grayscale value of the interference fringe image, and the second channel is filled in at each pixel position of the interference fringe image. (632.8), the third channel is to fill in the initial value of the optical path difference (0) at each pixel position of the interference fringe image. For convenience, the three-channel array is named three-channel interference fringe image.

[0040] S3. Establish a convolutional neural network with a dual-branch structure, input the interference fringe image, and output the surface deformation field.

[0041] The specific steps include the following: First, a model based on a convolutional neural network (CNN) is built, including a dual-branch structure, combined with physical constraints (known formulas or laws based on optics and physics that are used to limit or guide prediction results, such as the optical path difference formula) to ensure that the deformation regression conforms to both image features and optical principles.

[0042] The first branch is the encoder (feature extraction branch), which is a 5-layer convolutional network. It is used to extract spatial features (such as fringe density and width) from the input three-channel interference fringe image, gradually compress the information, and output a 64-channel spatial feature map. This uses the local receptive field of each convolution layer to capture the spatial pattern of the fringe. The second branch is the physical constraint branch, which is based on the Fourier transform to calculate the phase , embedding the optical path difference formula The function is to use the physical properties of interference fringes (wavelength and Phase ) constrains the CNN to ensure that the regression deformation is consistent with the optical principle. A more popular explanation is to extract frequency information from the feature map output by the encoder and estimate the phase Combined with input , calculate the optical path difference ; It also includes a decoder (fusion and regression). The decoder has a 5-layer transposed convolutional layer, which is used to fuse the outputs of the encoder and the physical constraint branch, upsample to restore the spatial resolution, and generate a deformation field map. The specific steps for fusing the outputs of the encoder and the physical constraint branch are as follows: the spatial features output by the encoder are 64-channel feature maps (including stripe density, width, etc.), and the physical constraint parameters output by the physical constraint branch are optical path differences calculated based on Fourier transform. , in order to achieve fusion, the optical path difference It is expanded to a tensor with the same spatial dimension as the feature map, and then concatenated with the 64-channel feature map along the channel axis to generate fused features.

[0043] Preferably, 5 layers of convolution capture the spatial pattern of stripes (such as density and width) and generate a 64-channel spatial feature map, providing rich image information.

[0044] The training process is as follows: prepare a data set (use about 5,000 glass interference fringe images with known deformation fields for training). Use the mean square error to measure the difference between the predicted deformation field and the actual deformation, and its expression is: ; in, represents the mean square error, Represents the total number of pixels. Indicates The predicted deformation field height value of each pixel, Indicates The actual deformation field height value of each pixel.

[0045] There is also physical consistency loss, which is to check whether the predicted optical path difference conforms to the optical principle. and the optical path difference calculated from the fringe phase If they are inconsistent, the loss will be great. To ensure compliance with the laws of physics, the expression is: ; in, represents the loss of physical consistency, Indicates The predicted optical path difference of each pixel is Indicates The difference between the predicted optical path difference of each pixel and the theoretical optical path difference.

[0046] Then the total loss is calculated based on the mean square error and the physical error, and its expression is: ; Among them, L Represents the total loss, which can also be understood as the total score. represents the mean square error, Represents the weight coefficient, which is set to 0.1 here (the weight is set to 0.1 here because more attention is paid to deformation accuracy, but physical consistency is also important. The specific weight value can be customized according to one's own business needs and actual needs). Indicates a loss of physical consistency.

[0047] In getting the total loss Then use the Adam optimizer to optimize until the total loss No longer decreases significantly, reaching loss convergence indicates that the training is complete, or a training threshold can be set. The training is completed if it is less than the training threshold (the value of the training threshold is customized according to needs).

[0048] The three-channel interference fringe image is directly input into the dual-branch convolutional neural network. First, it passes through the encoder. The encoder extracts the spatial features of the three-channel interference fringe image (such as fringe density and width change) through 5 layers of convolution to generate a 64-channel spatial feature map. The physical constraint branch performs Fourier transform on this 64-channel spatial feature map to calculate the phase , combined with the input =632.8nm, estimated optical path difference , and the initial =0, optimize the parameters of the dual-branch convolutional neural network, and through physical constraints, let the dual-branch convolutional neural network optimize the generation process of the 64-channel spatial feature map, and finally improve the quality of the surface deformation field map. The decoder fuses the outputs of the encoder and the physical constraint branch, and generates a surface deformation field map through 5 layers of transposed convolution upsampling. Each pixel value represents the height change of the glass surface.

[0049] Preferably, 5 layers of convolution capture the spatial pattern of the stripes (such as density and width), generate a 64-channel spatial feature map, and provide rich image information; and through the physical constraint branch, it ensures that the physical meaning of the surface deformation field is consistent with the interference stripes.

[0050] For the generated surface deformation field map, the defect depth can be directly read, for example, the convex area value is 0.15μm, and the concave area value is -0.10μm. Combined with the initial defect classification map, if the pixel height value of the surface deformation field map is greater than 0 and corresponds to the red area of ​​the initial defect classification map, it is confirmed to be convex; if the pixel height value of the surface deformation field map is less than 0 and corresponds to the blue area of ​​the initial defect classification map, it is confirmed to be concave. Use Matplotlib to generate a two-dimensional deformation heat map (the color depth indicates the height) and a three-dimensional profile map, annotate the depth and type, and then save them.

[0051] It is further explained that the convolutional neural network with a dual-branch structure not only extracts features from the interference fringe image, but also constrains the prediction results through optical formulas to ensure that the deformation field conforms to both the image features and the physical laws.

[0052] S4. Place the glass product under a terahertz time-domain spectrometer, and screen the concave-convex area based on the surface deformation field. The concave-convex area is screened by setting a defect threshold to screen the pixel points whose absolute height value exceeds the defect threshold in the surface deformation field map, and is generated by clustering and merging adjacent pixel points, and a photodetector is used to capture the time domain signal of the concave-convex area.

[0053] Specifically, the following steps are included: Place the glass product to be tested in a terahertz time-domain spectrometer with a frequency band covering 0.1-3THz (for example, MenloSystems TERA K15). The frequency band of 0.1-3THz can penetrate glass and detect surface and sub-surface defects. Use image processing algorithms (Python+NumPy) based on the surface deformation field for analysis. First, set a defect threshold (for example, the concave and convex points are greater than 0.05μm, which is customized according to the usage scenario and requirements) to screen out significant concave and convex points. Significant concave and convex points are pixels in the surface deformation field map whose height exceeds the defect threshold. These points are considered potential concave and convex defects due to their significant height deviation. Use the NumPy.where function to obtain the pixel coordinates that meet the defect threshold (for example, [x1, y1], [x2,y2]), and then merge neighboring points through DBSCAN (radius 5 pixels) to generate a key scanning area (such as a small block of 10×10mm). Assuming that the size of the glass to be tested is 100 by 100 mm, the pixel coordinates are mapped to the physical space (1 pixel is approximately equal to 0.05 mm) to output the movement instructions of the scanning platform, guide the terahertz beam to focus on the defect area, and the terahertz pulse irradiates the glass surface at vertical incidence. The optical alignment system (for example, a parabolic mirror) focuses the spot (about 1 mm in diameter) to ensure energy concentration. The photodetector captures the reflected signal, which represents the time domain waveform of the terahertz pulse (the electric field intensity changes with time). The sampling rate is set to 1 GHz (1 billion samples per second), the time window is 1 ns (1000 ps), and 1000 time domain sampling points are collected (each point is 1 ps apart). The time domain waveform contains the main reflection peak (surface reflection) and the secondary peak (sub-surface defect scattering), reflecting the defect depth and material changes. The acquisition is repeated 5 times for each sampling point, and the average value is obtained using the time averaging method to reduce noise interference. Finally, the acquisition of the reflected time domain signal is completed, and each time domain waveform reflects the signal changes caused by the defect (such as peak delay and amplitude attenuation). The time domain signal is stored as an N×1000 matrix (N is the number of scanning points, such as 10 points, 1000 sampling points at each point, unit: millivolt) with position coordinates (x, y) and surface height values.

[0054] Preferably, in glass defect detection, accurate positioning of concave and convex points can quickly lock the problem area, improve detection efficiency and reliability, and avoid wasting time on irrelevant areas. Moreover, mapping the pixel coordinates of the surface deformation field map to the physical space is equivalent to directly indicating where the problem is, which is convenient for subsequent positioning, repair or recording, and improves the operability of quality control.

[0055] S5. Input the time domain signal into a pre-established time domain signal processing dedicated network to generate time domain features, frequency domain signals and frequency domain features, and fuse the time domain features and frequency domain features to obtain defect signals. Input the defect signals and frequency domain signals into the constructed multi-scale defect association model to generate the final defect distribution map.

[0056] The specific steps include: S5.1. Here, a dedicated network for time domain signal processing is established first.

[0057] First, let's introduce the components of the dedicated network for time domain signal processing, which consists of a time domain processing part, a frequency domain processing part, a feature fusion part, and a sequence processing part. The time domain processing part is a one-dimensional convolution layer, which is used to extract the time domain features of the time domain signal, such as pulse shape, main peak position, and secondary peak delay; the frequency domain processing part is divided into two parts: a short-time Fourier transform part and a pooling layer. The short-time Fourier transform part is used to generate frequency domain signals and extract frequency domain features. The pooling layer is used to reduce the dimension of the frequency domain signal. The entire frequency domain processing part is used to analyze the frequency characteristics of the time domain signal, such as absorption peaks and microstructure changes; the feature fusion part is composed of an attention mechanism and a splicing layer. The attention mechanism enhances the frequency domain features through self-attention, and the splicing layer is used to merge the time domain features and the frequency domain features; the sequence processing part is a long short-term memory layer, which is used to integrate the time domain features and the enhanced frequency domain features, capture timing dependencies, and output defect signals.

[0058] Now is the training process of the dedicated network for time domain signal processing. Prepare a training set, which contains 5,000 sets of time domain signals, known defect annotations (depth and type), real time domain signals (collected by terahertz spectrometer), and real frequency domain signals (generated by short-time Fourier transform of real time domain signals). The gap between the predicted signal and the real signal is measured by calculating the time domain error, and its expression is: ; in, Indicates the number of sampling points of the signal.

[0059] The frequency domain characteristics are accurate by calculating the frequency domain error, and its expression is: ; in, Represents the L2 norm squared of the difference vector.

[0060] The total error is obtained based on the time domain error and frequency domain error, and its expression is: ; in, Represents the weight, which is generally set to 0.2 to balance the contributions of the two parts. It can be customized according to your needs.

[0061] The optimization tool uses the Adam optimizer (learning rate 0.001), the training rounds are 50 rounds, and the loss converges to 0.03, indicating that the training of the dedicated network for time domain signal processing is completed.

[0062] Normalize the time series signal to [-1,1], convert it into a tensor (N×1×1000), and input it into a dedicated network for time domain signal processing. After entering the dedicated network for time domain signal processing, it first enters the one-dimensional convolution layer. The one-dimensional convolution layer captures the pulse shape (such as the main peak width and the secondary peak interval) through the convolution kernel, generates time domain features (N×32×996), and finds the location of the peak in the time domain features through peak search, such as the main peak and the secondary peak, setting the minimum height to 0.1 and the interval to at least 10 points. Then enter the frequency domain processing part, generate the frequency domain signal (N×64×30) through short-time Fourier transform, and then reduce the dimension through the pooling layer, reducing the frequency domain signal from N×64×30 to N×32×15. The frequency domain signal then enters the self-attention mechanism. In layman's terms, it scores each part of the frequency domain signal to determine which frequencies are more important to the defect (such as a 0.5THz crack signal). Specifically, the frequency domain signal is divided into three parts: query, key, and value. The signal is compared with itself, and the weight of each point is calculated. The frequency domain signal is then adjusted with the weight of each point to highlight the key area. After processing, the data size remains unchanged, but the important frequencies are amplified and the unimportant frequencies are weakened, emphasizing the frequency information related to the defect. Then enter the concatenation layer, first use average pooling on the time domain features (N×32×996), divide the 996 points into small segments, take the average value of each segment, shrink it to 15 points, and become N×32×15, matching the size of the frequency domain features. Then, the time domain features and frequency domain features are concatenated together to obtain N×64×15 (32+32=64). Through the fully connected layer, each point of 64×15 is condensed into 128 key values ​​to become a fused feature (N×128), which integrates the time and frequency information. Finally, the fused features are calculated through the memory units of the long short-term memory layer to obtain the defect signal.

[0063] It is further explained that thanks to the joint analysis of time domain and frequency domain, coupled with the attention mechanism, it is possible to simultaneously detect the unevenness and microcracks on the glass surface, reduce missed detections and misjudgments, and is suitable for high-precision glass quality inspection.

[0064] S5.2 Now we are going to build a multi-scale defect correlation model, which consists of time domain analysis, frequency domain analysis and defect quantification. Among them, time domain analysis uses the peak search method to analyze the peaks in the defect signal and locate macro defects. Frequency domain analysis extracts the main frequency mode from the frequency domain signal through principal component analysis to analyze micro defects. Defect quantification combines the defect signal and the surface height value to determine the specific characteristics of the defect.

[0065] The training process is as follows: defect signals and frequency domain signals are generated from 5000 sets of time domain signals and known defect annotations (depth and type) of the training time domain signal processing dedicated network as the training set of the multi-scale defect association model.

[0066] For time domain analysis, the peak search rules are set as follows: minimum height 0.1 (to avoid misjudgment of noise), minimum interval 10 points (to avoid repeated detection). For frequency domain analysis, principal component analysis is applied to retain 90% of the main modes of the frequency domain signal and extract the absorption peak features (such as 0.5THz crack signal). For defect quantification, the rules are formulated: positive defect signals and positive surface height values ​​of the surface deformation field map indicate convexity, and negative defect signals and negative surface height values ​​of the surface deformation field map indicate concavity. Based on the training set, the parameters of peak search and principal component analysis are optimized to make the predicted depth error <0.01μm and the frequency peak position error <0.01GHz. Then, partial data (such as 1000 groups) are used for testing to confirm that the multi-scale defect association model can correctly identify macroscopic convexity (such as 0.15μm) and microscopic cracks (such as 0.5THz features). When the accuracy reaches 95%, the training of the multi-scale defect association model is completed.

[0067] Next, we will apply the multi-scale defect correlation model. First, we input the defect signal and frequency domain signal into the multi-scale defect correlation model. The time domain analysis uses the peak search method to scan the defect signal, set the minimum height to 0.1, and the interval to 10 points to find the main peak (surface reflection) and secondary peak (subsurface scattering). Calculate the delay time between the main peak and the secondary peak. For example, 0.5ps means that the signal is 0.5 picoseconds late. Then convert the delay into depth, and the expression is: ; in, Indicates depth, The speed of light is 3×10 8 m / s, Indicates that the glass refractive index is 1.5.

[0068] Frequency domain analysis extracts 90% of the main frequency modes from the frequency domain signal through principal component analysis, checks the absorption peaks in the main frequency modes, for example, cracks may have stronger signals at 0.5 THz, and then records them. Defect quantification generates defect distribution by combining the contents of time domain analysis and frequency domain analysis. Specifically, the depth Combined with the surface height value verification and classification, if the defect signal is positive (indicating a convex tendency) and the surface height value is also positive (such as 0.15μm), the defect is confirmed to be surface convexity, and the depth is calculated. (e.g. 0.05μm), or take the average of the two (depending on the requirements). If the defect signal is negative (indicating a tendency to sink) and the surface height value is also negative (e.g. -0.10μm), the defect is confirmed to be subsurface concavity with a depth of h (e.g. -0.05μm). Use the position coordinates (x, y) to map the defect information of each point onto a 100×100mm glass plane.

[0069] Using the heat map drawing method, the defect depth is represented by color (dark color represents high, light color represents low), and a two-dimensional map is generated to intuitively display the defect distribution. Using the three-dimensional profile drawing method, the position coordinates (x, y) and depth values ​​are drawn into a three-dimensional map, and the type and value of each defect (such as "convexity 0.15μm") are marked. Finally, the task of detecting the concave and convex surface of the glass product is completed.

[0070] Preferably, by combining time domain analysis and frequency domain analysis, macroscopic defects and microscopic defects of glass are located respectively, and combined with surface height value verification, an accurate defect distribution map is finally generated. Such a combined method can greatly improve the reliability of glass product quality control, while also reducing the risk of missed detection by a single method.

[0071] This embodiment also provides a glass product surface concave-convex detection system based on artificial intelligence, comprising: The initial defect module collects surface images of glass products and inputs them into the constructed cross-material transfer learning network to generate an initial defect classification map; A data acquisition module collects interference fringe images using a polarization interferometer according to the initial defect classification diagram; The surface deformation module establishes a double-branch convolutional neural network, inputs the interference fringe image, and outputs the surface deformation field; The time domain capture module places the glass product under the terahertz time domain spectrometer, and screens the concave-convex area based on the surface deformation field. The concave-convex area screens the pixel points whose absolute height value exceeds the defect threshold in the surface deformation field map by setting the defect threshold, and generates by clustering and merging the adjacent pixel points, and uses a photodetector to capture the time domain signal of the concave-convex area; The final defect module inputs the time domain signal into a pre-established time domain signal processing dedicated network to generate time domain features, frequency domain signals and frequency domain features, and fuses the time domain features with the frequency domain features to obtain the defect signal. The defect signal and the frequency domain signal are input into the constructed multi-scale defect association model to generate the final defect distribution map.

[0072] This embodiment also provides a computer device, which is suitable for the case of a glass product surface convexity and concave detection method based on artificial intelligence, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the glass product surface convexity and concave detection method based on artificial intelligence proposed in the above embodiment.

[0073] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0074] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for detecting unevenness on the surface of a glass product based on artificial intelligence as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0075] In summary, the present invention builds a cross-material transfer learning network, uses rich defect samples of metals and ceramics (such as scratches and cracks) to make up for the lack of glass data, highlights key features (such as texture) through the attention mechanism, and generates a classification map containing the probabilities of concave, convex, and defect-free, which is used to quickly screen defect areas on the glass surface. It breaks through the limitation of scarce glass samples, enhances the generalization ability, especially improves the sensitivity to tiny defects, reduces data collection costs and supports multi-material detection scenarios. In addition, through the time domain signal processing network and the multi-scale defect joint model, the time domain features and frequency domain features are fused to generate a defect distribution map, which is convenient for rapid decision-making and defect classification, and improves production efficiency and product quality.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting concave-convex surfaces of glass products based on artificial intelligence, characterized in that: include, Collect surface images of glass products and input them into the constructed cross-material transfer learning network to generate an initial defect classification map; Collect interference fringe images using a polarization interferometer according to the initial defect classification map; A dual-branch convolutional neural network is established, the interference fringe image is input, and the surface deformation field is output; The glass product is placed under a terahertz time-domain spectrometer, and the concave-convex area is screened based on the surface deformation field. The concave-convex area is screened by setting a defect threshold and the pixel points whose absolute height value exceeds the defect threshold in the surface deformation field map are screened, and the adjacent pixel points are generated by clustering and merging, and the time domain signal of the concave-convex area is captured by a photodetector; The time domain signal is input into a pre-established time domain signal processing dedicated network to generate time domain features, frequency domain signals and frequency domain features, and the time domain features and frequency features are fused to obtain the defect signal. The defect signal and frequency domain signal are input into the constructed multi-scale defect association model to generate the final defect distribution map.

2. The method for detecting unevenness on the surface of a glass product based on artificial intelligence according to claim 1, characterized in that: Generating the initial defect classification map refers to constructing a cross-material transfer learning network; ResNet-50, a cross-material transfer learning network, is used to extract multiple features from a training set of metal and ceramic defect images with defect annotations to generate metal feature vectors and ceramic feature vectors, which represent the defect modes of metal and ceramic, respectively. At the same time, features are extracted from the surface image of the glass to be tested to obtain a glass feature vector. The metal and ceramic defect images are training data with defect annotations. Through training, the cross-material transfer learning network learns cross-material defect knowledge; The metal feature vector and the ceramic feature vector are spliced ​​to generate a joint feature vector, and then the joint feature vector is spliced ​​with the glass feature vector to generate a cross-material fusion feature; The fully connected layer and SENet attention mechanism in the cross-material transfer learning network are used to process cross-material fusion features and generate glass defect features. The defect classifier of the cross-material transfer learning network processes the glass defect features to generate an initial defect classification map.

3. The method for detecting unevenness on the surface of a glass product based on artificial intelligence according to claim 2, characterized in that: Generating glass defect features refers to performing dimensional transformation on cross-material fusion features through a fully connected layer of a cross-material transfer learning network; The SENet attention mechanism is used to perform weighted optimization on cross-material fusion features to generate glass defect features.

4. The method for detecting unevenness on the surface of a glass product based on artificial intelligence as claimed in claim 3, characterized in that: According to the initial defect classification diagram, a polarization interferometer is used to collect an interference fringe image, which specifically includes the following steps: The glass is placed on a polarization interferometer, the coordinates of the defect area are extracted according to the initial defect classification map, and the interference fringe image is located and collected based on the coordinates of the defect area.

5. The method for detecting unevenness on the surface of a glass product based on artificial intelligence according to claim 4, characterized in that: The dual-branch structure convolutional neural network includes an encoder, a physical constraint branch and a decoder, and the physical constraint branch includes an optical path difference calculation unit; By inputting the interference fringe image into the encoder, feature extraction is performed to obtain spatial features; The physical constraint branch obtains phase information by performing Fourier transform on the spatial features, generates the optical path difference through the optical path difference calculation unit according to the phase information, expands the optical path difference to the same tensor as the spatial features, splices it with the spatial features, and then generates the surface deformation field through upsampling.

6. The method for detecting unevenness on the surface of a glass product based on artificial intelligence according to claim 5, characterized in that: The step of inputting the time domain signal into a pre-established dedicated network for time domain signal processing to generate a frequency domain signal, and fusing the time domain signal and the frequency domain signal to obtain a defect signal specifically includes the following steps: The time domain signal processing dedicated network is composed of a time domain processing unit, a frequency domain processing unit, a fusion processing unit and a sequence processing unit; The time domain signal is processed by a time domain processing unit to obtain a time domain feature; The frequency domain processing unit processes the time domain features through short-time Fourier transform to obtain frequency domain signals and frequency domain features, and enhances the frequency features using the attention mechanism to obtain enhanced frequency features; The time domain features and the enhanced frequency features are fused through a fusion processing unit to generate fusion features, and the fusion features are processed by a sequence processing unit to output defect signals.

7. The method for detecting unevenness on the surface of a glass product based on artificial intelligence according to claim 6, characterized in that: The generating of the final defect distribution map specifically comprises the following steps: The defect signal and frequency domain signal are input into the constructed multi-scale defect association model, and the final defect distribution map is generated through time domain analysis, frequency domain analysis and defect quantification.

8. A glass product surface concave-convex detection system based on artificial intelligence, which implements the glass product surface concave-convex detection method based on artificial intelligence according to any one of claims 1 to 7, characterized in that: include, The initial defect module collects surface images of glass products and inputs them into the constructed cross-material transfer learning network to generate an initial defect classification map; A data acquisition module collects interference fringe images using a polarization interferometer according to the initial defect classification diagram; The surface deformation module establishes a double-branch convolutional neural network, inputs the interference fringe image, and outputs the surface deformation field; The time domain capture module places the glass product under the terahertz time domain spectrometer, and screens the concave-convex area based on the surface deformation field. The concave-convex area screens the pixel points whose absolute height value exceeds the defect threshold in the surface deformation field map by setting the defect threshold, and generates by clustering and merging the adjacent pixel points, and uses a photodetector to capture the time domain signal of the concave-convex area; The final defect module inputs the time domain signal into a pre-established time domain signal processing dedicated network to generate time domain features, frequency domain signals and frequency domain features, and fuses the time domain features with the frequency domain features to obtain the defect signal. The defect signal and the frequency domain signal are input into the constructed multi-scale defect association model to generate the final defect distribution map.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for detecting unevenness on the surface of a glass product based on artificial intelligence according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting unevenness on the surface of a glass product based on artificial intelligence according to any one of claims 1 to 7 are implemented.

Images