A Method and System for Detecting Tempered Glass Based on a Two-Branch Reverse Residual Network Model
By combining a lightweight bi-branch inverse residual network model with ultrasonic testing technology, the problems of high computational requirements and insufficient accuracy in existing glass testing are solved, thus achieving efficient and accurate tempered glass testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHAANXI NORMAL UNIV
- Filing Date
- 2023-06-16
- Publication Date
- 2026-05-26
AI Technical Summary
In existing glass inspection technologies, high-performance network models require sophisticated computing equipment and have long inference times, while small-parameter network models have insufficient accuracy, resulting in inaccurate glass inspection results.
A lightweight bi-branch inverse residual network model is adopted, which combines a dual-path structure, an inverse residual structure, and an attention mechanism. By detecting the changes in residual stress inside the glass through ultrasonic waves, feature extraction and classification are performed using time-frequency domain images. This lightweight bi-branch inverse residual network model is used for tempered glass inspection.
While reducing the number of model parameters and computational load, the accuracy and efficiency of tempered glass inspection are improved, enabling non-destructive and continuous inspection and ensuring the safety and efficiency of glass production.
Smart Images
Figure CN116678537B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image recognition technology and relates to a method and system for detecting tempered glass based on a bi-branch inverse residual network model. Background Technology
[0002] In our daily lives, glass windows, glass doors, and glass walls are ubiquitous, mostly made of glass. Because glass is a transparent surface, quality control during the production process is particularly important. For example, document 202310172528.6 describes a glass inspection device and method. This method involves cameras at different positions automatically scanning the glass from different angles as it moves at a constant speed on a conveyor belt. The scan results are transmitted to a computer, and operators then judge whether the four edges of the glass are damaged based on the scanned images. However, this method primarily uses infrared technology to irradiate surface defects or damage to acquire images. This technology mainly identifies relatively serious or obvious surface defects and is prone to missed or false detections, making accurate results uncertain.
[0003] With the advancements in deep learning theory and computing devices, image recognition, as a crucial application of computer vision, has become closely intertwined with people's lives. In industrial manufacturing, it's used to analyze images of finished products. It can identify defects and deviations from quality standards more accurately and quickly than human inspectors. Similarly, in the medical field, it can assist doctors in diagnosis by recognizing X-rays, MRI scans, and other medical images, enabling early detection and treatment.
[0004] Deep learning has evolved from LeNet to AlexNet, and later to VGG, GoogleNet, ResNet, DenseNet, and so on. With the continuous development of these iconic models, the accuracy of image recognition has increased, and their applications have become more and more widespread.
[0005] However, the following problems still exist in the image processing of glass detection technology: (1) High-performance network models have large parameters, high requirements for computing equipment, and long inference time; (2) Networks with small model parameters are easy to deploy, but the accuracy is insufficient.
[0006] Therefore, the technology of applying deep learning network models to tempered glass inspection is still in the exploratory stage and there are still shortcomings in its application. Summary of the Invention
[0007] To address the shortcomings of existing glass production quality inspection technologies, this invention provides a tempered glass inspection method and corresponding system based on a lightweight bi-branch inverse residual network model, which has advantages such as high accuracy, small parameter quantity, and rapid detection capability.
[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0009] A method for inspecting tempered glass products based on a lightweight bi-branch inverse residual network model includes the following steps:
[0010] 1) Construct a lightweight bi-branch inverse residual network model;
[0011] 2) The change of residual stress in the glass is measured by comparing the ultrasonic wave with the nonlinear coefficient, and the time-domain data of the residual stress in the tempered glass product is obtained.
[0012] 3) Perform short-time Fourier transform on the time-domain image of glass stress to obtain the time-frequency domain image, and establish a dataset;
[0013] 4) Enhance the image data in the dataset from step 3) to obtain the original matrix P;
[0014] 5) Input the original matrix P from step 4) into the lightweight two-branch inverse residual network model for training, extract the features of the image, and obtain the weight model file;
[0015] 6) Load the weight model file trained in step 5) into the prediction program, input the time-frequency domain image of the stress of the tempered glass to be inspected, and obtain the detection result of the tempered glass sample by model recognition and prediction according to the method in steps 4) to 5).
[0016] Further specifying, the lightweight bi-branch inverse residual network model described in step 1) includes a preprocessing unit, a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, and a classification unit connected in sequence; the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit each contain one or more bi-branch inverse residual modules;
[0017] The dual-branch inverse residual module includes two parallel branches: Branch 1 and Branch 2. The preprocessed data in Branch 1 undergoes 3x3 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting a main feature matrix. In Branch 2, the data undergoes 5x5 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting a large receptive field matrix. The main feature matrix and the large receptive field matrix are merged along the channel dimension to output a merged matrix. Simultaneously, the main feature matrix and the large receptive field matrix are summed to obtain a summation matrix. The merged matrix and the summation matrix are then concatenated by channel to complete the novel feature fusion.
[0018] Further specifying, step 2) specifically includes:
[0019] 2.1) The ultrasonic transmitter and the ultrasonic receiver are mounted in pairs on multiple metal rails with adjustable spacing. The parallel metal rails form an ultrasonic transducer matrix with a certain number and spacing. The ultrasonic transmitter and the ultrasonic receiver are controlled by corresponding servo motors and can move independently along the X, Y and Z coordinates.
[0020] 2.2) Determine whether the ultrasonic transmitter and receiver are loaded onto the test point on the tempered glass. If not, readjust their positions; if yes, proceed to the next step.
[0021] 2.3) The ultrasonic image processing unit generates a periodic pulsed sinusoidal signal, which is input to the transmitting device to emit pulsed ultrasonic waves, which are then received by the receiving device after passing through the glass;
[0022] 2.4) The received ultrasonic signal is input into the system. Based on the fact that the relative nonlinear coefficient value is proportional to the stress of the stress-free standard tempered glass, the relative nonlinear coefficient corresponding to the tempered glass is obtained.
[0023] 2.5) The ultrasonic image processing unit correlates the nonlinear coefficient obtained in step (2.4) with its corresponding stress, plots the curve of the nonlinear coefficient with stress, obtains the image of the change of residual stress in the tempered glass, and then obtains the time domain data of the residual stress in the tempered glass product.
[0024] Further specifying, step 4) specifically includes:
[0025] The dataset obtained in step 3) is divided into training set, validation set and test set in a ratio of 8:1:1. The obtained data is then cropped and standardized according to the reset size and random size, and then augmented.
[0026] Further specifying, step 5) specifically includes:
[0027] 5.1) Input the original matrix P into the preprocessing unit, and after denoising by 5*5 convolution and 3*3 pooling, obtain the feature matrix P0;
[0028] 5.2) Input the feature matrix P0 into the first feature extraction unit, and perform feature extraction through a two-branch inverse residual module to obtain the inverse residual feature P1;
[0029] 5.3) Input the inverse residual feature P1 into the second feature extraction unit, and perform feature extraction through two cascaded double-branch inverse residual modules to obtain the inverse residual feature P2;
[0030] 5.4) Input the inverse residual feature P2 into the third feature extraction unit, and perform feature extraction through three cascaded double-branch inverse residual modules to obtain the inverse residual feature P3;
[0031] 5.5) Input the inverse residual feature P3 into the fourth feature extraction unit, and after feature extraction by two cascaded double-branch inverse residual modules, obtain the inverse residual feature P4;
[0032] 5.6) Input the inverse residual feature P4 into the adaptive average pooling layer, and then into the fully connected layer. Combine the extracted distribution features to form a one-dimensional feature vector, and then output the weight model file after weighted summation of the feature vector.
[0033] The characteristics of the output of the dual-branch inverse residual module in steps (5.2), (5.3), (5.4), and (5.5) above are as follows:
[0034]
[0035] Among them, P i ( x ) represents the feature output by the i-th layer. i =1, 2, 3, 4; I is the IremulbNet network, Mish is the activation function, F i The diagram illustrates the convolutional relationship of the intermediate hidden layers. P04 and P05 represent the results of concatenating and summing the dimensions of the feature matrix during feature extraction, respectively.
[0036] Further specifying, step 5.2) involves the following specific training process:
[0037] 5.2.1) Perform a 1*1 first convolution on the feature matrix P0 to obtain matrix P01;
[0038] 5.2.2) Matrix P01 is input to the first branch and the second branch respectively. In the first branch, after 3*3 depthwise convolution and 3*3 pooling, it is passed to the SE attention module or directly to the SE attention module for channel feature enhancement, and outputs the main feature matrix P02. In the second branch, after 5*5 depthwise convolution and 3*3 pooling, it is passed to the SE attention module or directly to the SE attention module for channel feature enhancement, and outputs the large receptive field matrix P03.
[0039] 5.2.3) Merge the main feature matrix P02 and the large receptive field matrix P03 extracted from the image in step 5.2.2) along the channel dimension to obtain the merged matrix P04. At the same time, sum P02 and P03 to obtain the summation matrix P05. Concatenate the merged matrix P04 and the summation matrix P05 along the channel, and perform a 1*1 second convolution pointwise convolution to reduce the dimension and obtain the dual-channel information matrix P07.
[0040] 5.2.4) Summing the dual-path information matrix P07 from step 5.2.3) with the feature matrix P0 after dimensional transformation, we obtain the inverse residual feature P1.
[0041] Further specifying, step 6) is followed by step 7), which specifically involves: judging the test results of the tempered glass sample. When the surface stress is less than 70 MPa, the glass stress condition is unqualified and it is a defective product; when the surface stress is 70~90 MPa, the glass stress condition meets the national standard requirements for semi-tempered glass; when the surface stress is greater than 90 MPa, the glass stress condition meets the national standard requirements for tempered glass and it is a qualified product.
[0042] The present invention also provides a tempered glass product inspection system based on a lightweight bi-branch inverse residual network model, which includes an ultrasonic testing device and an ultrasonic image processing unit.
[0043] An ultrasonic testing device is set on both sides of the glass to be tested. It includes an ultrasonic transmitter and an ultrasonic receiver. The ultrasonic transmitter emits pulsed ultrasonic signals, which pass through the glass and are received by the ultrasonic receiver. The received signals are then transmitted to the ultrasonic image processing unit.
[0044] The ultrasonic image processing unit is used to send periodic pulse sinusoidal signals to the ultrasonic transmitter of the ultrasonic testing device, and simultaneously receive ultrasonic signals from the ultrasonic receiver of the ultrasonic testing device. The unit processes the signals using the tempered glass product testing method based on the dual-branch inverse residual network model and outputs the testing results of the tempered glass.
[0045] Further specifying, the ultrasound image processing unit includes a central processing module, a signal preprocessing module, and a dual-branch inverted residual signal processing module;
[0046] Central processing module: Used to emit periodic pulse sinusoidal signals and receive ultrasonic signals, transmit the received ultrasonic signals to the signal preprocessing module, and receive the glass stress value transmitted by the dual-branch inverted residual signal processing module and output it.
[0047] Signal preprocessing module: Receives the ultrasound signal sent by the central processing module, performs short-time Fourier transform on the time-domain data to obtain a time-frequency domain image, extracts the feature information of the time-frequency domain image, and forwards the information to the dual-branch inverse residual signal processing module;
[0048] The dual-branch inverse residual signal processing module trains the enhanced image information in a dual-branch inverse residual network model, extracts the weight model file, loads the weight model file into the prediction program, compares it with the information of the glass stress to be detected, and transmits the glass stress value obtained from the comparison to the central processing module.
[0049] Further specifying, the lightweight bi-branch inverse residual network model includes a preprocessing unit, a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, and a classification unit connected in sequence; the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit are each composed of one or more bi-branch inverse residual modules;
[0050] The dual-branch inverse residual module includes two parallel branches: Branch 1 and Branch 2. The preprocessed data in Branch 1 undergoes 3x3 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting a main feature matrix. In Branch 2, the data undergoes 5x5 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting a large receptive field matrix. The main feature matrix and the large receptive field matrix are merged along the channel dimension to output a merged matrix. Simultaneously, the main feature matrix and the large receptive field matrix are summed to obtain a summation matrix. The merged matrix and the summation matrix are then concatenated by channel to complete the novel feature fusion.
[0051] Compared with the prior art, the beneficial effects of the present invention are:
[0052] 1. The present invention uses a lightweight bi-branch inverse residual network model to reconstruct an inverse residual structure for tempered glass quality inspection. It combines bi-path structure, inverse residual structure and attention mechanism, etc., to improve classification accuracy and improve the accuracy and efficiency of tempered glass inspection results while significantly reducing the number of model parameters and computation.
[0053] 2. The present invention for tempered glass quality inspection utilizes the output of the inverted residual structure to reduce the number of channels, and at the same time uses depthwise splitting convolution in the feature extraction layer. The operation of convolution with different sizes can increase the diversity of features and change the width of the network, effectively reducing the number of model parameters and computational load, and greatly improving the detection efficiency of tempered glass.
[0054] 3. In the feature extraction stage of the tempered glass quality inspection of this invention, feature fusion is used to splice and sum the feature information obtained from the two branches. Then, the two results are spliced in the channel dimension. The feature matrix output contains richer and more complex feature information, which effectively enhances the feature extraction capability and ensures the accuracy of the detection results.
[0055] 4) The present invention can improve the detection efficiency and accuracy of the above-mentioned tempered glass quality detection, while also ensuring non-destructive and continuous detection, thus ensuring the safety and efficiency of glass production. Attached Figure Description
[0056] Figure 1 This is a schematic diagram of the ultrasonic testing device.
[0057] Figure 2 This is a schematic diagram of the lightweight bi-branch inverse residual network IretbNet.
[0058] Figure 3 yes Figure 2 Schematic diagram of the dual-branch inverted residual module.
[0059] Figure 4 This is a flowchart of the online stress detection process for glass.
[0060] Figure 5A This is a time-domain image of glass stress.
[0061] Figure 5B This is a frequency domain image of glass stress.
[0062] Figure 5C This is a time-frequency domain image of glass stress.
[0063] Figure 6A This is a comparison chart of the parameter quantities of each network model in Experiment 1.
[0064] Figure 6B This is a comparison chart of the computational complexity of each network model in Experiment 1. Detailed Implementation
[0065] The invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0066] The experimental environment for this invention was a server running CentOS with a 2.6GHz Intel(R) Xeon(R) E5-2640 v3 processor, 125.65GB of memory, and a GeForce RTX 3090 graphics card with 24GB of video memory. The program environment consisted of Python version 3.9.1, PyTorch version 1.10.0, and CUDA version 11.3.
[0067] The technical idea of this invention is to establish a lightweight bi-branch inverse residual network model (hereinafter referred to as IretbNet) for image recognition and classification. The reconstructed inverse residual structure consists of a bi-path structure, an attention mechanism, and an inverse residual module, which can improve network performance and computational efficiency while significantly reducing the number of parameters and computational load.
[0068] The lightweight bi-branch inverse residual network model provided by this invention refers to... Figure 2It includes a preprocessing unit (preprocessing module), a first feature extraction unit (DI_Block×1), a second feature extraction unit (DI_Block×2), a third feature extraction unit (DI_Block×3), a fourth feature extraction unit (DI_Block×2), and a classification unit connected in sequence; the feature extraction unit is composed of one or more bi-branch inverse residual modules.
[0069] Furthermore, the dual-branch inverse residual module includes a 1*1 first convolutional layer, a dual-branch inverse residual module, and a 1*1 second convolutional layer connected in sequence.
[0070] Furthermore, the lightweight dual-branch inverse residual network model is characterized in that the dual-branch inverse residual module includes a parallel branch one and a branch two; the 1*1 first convolutional layer connects both branch one and branch two simultaneously; and the two branches are connected to a 1*1 second convolutional layer after feature fusion.
[0071] Furthermore, the lightweight dual-branch inverse residual network model is characterized in that branch one includes sequentially connected 3*3 depthwise separating convolutions, which pass through a 3*3 pooling layer when the stride is 2, and are directly passed through when the stride is 1, and then pass through the SE attention module; branch two includes sequentially connected 5*5 depthwise separating convolutions, which pass through a 3*3 pooling layer when the stride is 2, and are directly passed through when the stride is 1, and then pass through the SE attention module.
[0072] Furthermore, the lightweight bi-branch inverse residual network model is characterized in that the preprocessing unit sequentially includes a 5*5 convolutional layer and a 3*3 pooling layer.
[0073] Furthermore, the first, second, third, and fourth feature extraction units sequentially include one, two, three, and two novel feature fusion dual-branch inverted residual modules.
[0074] Furthermore, the classification unit sequentially includes an adaptive average pooling layer and a fully connected layer.
[0075] This invention designs a lightweight CNN model with a dual-branch structure, attention mechanism, and inverse residual structure as its core, namely the lightweight dual-branch inverse residual network model IretbNet. In the dual-path feature extraction stage, different operations are performed on two paths to increase feature diversity. A 5x5 convolutional layer is used to expand the receptive field. Simultaneously, an SE attention mechanism is used after convolution to better configure information according to channels. To reduce the number of model parameters, the inverse residual structure and depthwise separable convolution are introduced into the model design. In the classification stage, an adaptive average pooling layer is used to compress the size of the output feature map, preserving invariant feature information. A fully connected layer is then designed to combine distributed features and achieve classification.
[0076] Furthermore, the reconstruction of the dual-branch inverted residual module in this invention will be described in detail below from two aspects: dual-path feature extraction and feature fusion methods.
[0077] In the feature extraction stage, the feature information P0 obtained after a 1x1 dimensionality transformation is processed using a dual-path structure. Branch one first uses a 3x3 depthwise separating convolution to extract features. Then, if the stride is 2, 3x3 max pooling is used for downsampling, reducing computation while preserving the main features of the data. If the stride is 1, the features are directly passed. Next, the SE attention mechanism is used to assign weights to the resulting feature matrix. Branch two differs from branch one in that it uses a 5x5 depthwise separating convolution; the remaining processing steps are the same as in branch one.
[0078] Depthwise separable convolution plays a crucial role in reducing the number of parameters and computational cost of the model. It consists of depthwise convolution and pointwise convolution. Depthwise convolution is a Dk×Dk convolution with M channels, where each convolution kernel corresponds to one channel. Pointwise convolution is a 1×1 M-channel convolution.
[0079] For an input feature map of size Df×Df with M channels, and an output feature map of size Dg×Dg with N channels, the ratio of the number of parameters in the depthwise separable convolution to the number of parameters in the standard convolution is shown in Equation (1), and the ratio of its computational complexity is shown in Equation (2).
[0080] = + (1)
[0081] = + (2)
[0082] In the IretbNet network, dual-branch, dual-path feature extraction uses convolutional kernels of sizes 5*5 and 3*3, i.e. =5 and When the coefficient is 3, the number of parameters in DSC can be significantly reduced compared to the traditional convolution process, and the number of output channels of the dual-branch inverse residual module is also significantly reduced. Therefore, the IretbNet model of this invention uses a dual-branch inverse residual module, which greatly reduces the number of parameters in the model.
[0083] There are two common methods for feature fusion: one is the dimensional concatenation of feature matrices, represented by the Inception structure; the other is the addition of feature data of the same dimension in the residual structure and the inverse residual structure. The feature fusion method used in this invention combines the two methods, concatenating the dimensions of the processed results out1 and out2 from the two paths to obtain h1, and summing the results of the three to obtain h2. Finally, h1 and h2 are concatenated as the final result and input into the next convolutional layer.
[0084] The max pooling of this invention refers to the preservation of the maximum value within the receptive field after pooling, while the adaptive average pooling refers to the preservation of the average value of all data within the receptive field after pooling. These are well-known techniques in the field.
[0085] Specifically, the lightweight bi-branch inverse residual network model constructed above is applied to the inspection of tempered glass products to form a tempered glass product inspection system based on the lightweight bi-branch inverse residual network model.
[0086] This tempered glass product inspection system, based on a lightweight bi-branch inverse residual network model, mainly includes an ultrasonic testing device and an ultrasonic image processing unit.
[0087] Ultrasonic testing devices such as Figure 1 As shown, it specifically includes 1, an ultrasonic transmitter; 2, an ultrasonic receiver; 6, a metal guide rail; 7, a servo motor; and 8, tempered glass. An ultrasonic transmitter and receiver are mounted on metal guide rails 6 and moved to a set position by a servo motor. The ultrasonic transmitter emits a pulsed ultrasonic signal, which passes through the tempered glass and is received by the ultrasonic receiver, which then transmits the received signal to the ultrasonic image processing unit. Specifically, ultrasonic transmitters 1 and ultrasonic receivers 2 are mounted in pairs on multiple metal guide rails 6 with adjustable spacing. The parallel metal guide rails 6 form an ultrasonic transducer matrix with a certain number and spacing. The ultrasonic transmitter and receiver are controlled by corresponding servo motors 7 and can move independently along the X, Y, and Z coordinates. The servo motors 7 drive all ultrasonic transmitters 1 and ultrasonic receivers 2 to contact the tempered glass 8 being tested. At the same time, the ultrasonic receivers 2 receive the corresponding ultrasonic signals and input the received ultrasonic signals into the ultrasonic image processing unit 4. After processing, the nonlinear change value is obtained and compared with the stress-free standard tempered glass. A curve of the nonlinear coefficient changing with stress is plotted, and the stress distribution map of the tempered glass is plotted using the coordinates of each measurement point and the relative stress value, thus obtaining a time-domain image of the internal residual stress of the tempered glass product.
[0088] Optionally, the probe of the ultrasonic transmitting device 1 can be one or a combination of two or more of the following: a single crystal probe, a dual crystal probe, a water immersion probe, a composite probe, a high temperature probe, etc.
[0089] Optionally, a single ultrasonic transmitter 1 has a frequency of 1 to 5 MHz; and a single ultrasonic receiver 2 has a frequency of 2 to 10 MHz, which is twice the center frequency of the corresponding ultrasonic transmitter 1.
[0090] The ultrasonic image processing unit is used to send periodic pulse sinusoidal signals to the ultrasonic transmitter 1 of the ultrasonic testing device, and simultaneously receive ultrasonic signals from the ultrasonic receiver 2. The unit processes the signals using a lightweight double-branch inverse residual network model for tempered glass product testing, and outputs the test results of the tempered glass.
[0091] To further explain, the ultrasound image processing unit includes a central processing module, a signal preprocessing module, and a dual-branch inverted residual signal processing module;
[0092] Central processing module: Used to emit periodic pulse sinusoidal signals and receive ultrasonic signals, transmit the received ultrasonic signals to the signal preprocessing module, and receive the glass stress value transmitted by the dual-branch inverted residual signal processing module and output it.
[0093] Signal preprocessing module: Receives the ultrasound signal sent by the central processing module, performs short-time Fourier transform on the time-domain data to obtain a time-frequency domain image, extracts the feature information of the time-frequency domain image, and forwards the information to the dual-branch inverse residual signal processing module;
[0094] The dual-branch inverse residual signal processing module trains the enhanced image information in a dual-branch inverse residual network model, extracts the weight model file, loads the weight model file into the prediction program, compares it with the information of the glass stress to be detected, and transmits the glass stress value obtained from the comparison to the central processing module.
[0095] For further explanation, see [link to documentation]. Figure 2 The lightweight bi-branch inverse residual network model includes a preprocessing unit, a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, and a classification unit connected in sequence; the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit are each composed of one or more bi-branch inverse residual modules;
[0096] The dual-branch inverted residual module includes two parallel branches: Branch 1 and Branch 2; see [link / reference] Figure 3The preprocessed data undergoes 3x3 depthwise convolution in the first branch, followed by 3x3 pooling, and is then passed to the SE attention module or directly to it for channel feature enhancement, outputting the main feature matrix. In the second branch, the data undergoes 5x5 depthwise convolution, followed by 3x3 pooling, and is then passed to the SE attention module or directly to it for channel feature enhancement, outputting the large receptive field matrix. The main feature matrix and the large receptive field matrix are merged along the channel dimension to output the merged matrix. Simultaneously, the main feature matrix and the large receptive field matrix are summed to obtain the summation matrix. Finally, the merged matrix and the summation matrix are concatenated by channel to complete the novel feature fusion.
[0097] Example 1
[0098] The tempered glass product inspection method based on a lightweight bi-branch inverse residual network model includes the following steps:
[0099] 1) Construct a lightweight bi-branch inverse residual network model;
[0100] The lightweight bi-branch inverse residual network model, as described above, includes a preprocessing unit, a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, and a classification unit connected in sequence; the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit are each composed of one or more bi-branch inverse residual modules;
[0101] The dual-branch inverse residual module includes two parallel branches: Branch 1 and Branch 2. The preprocessed data in the first branch undergoes 3x3 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting the main feature matrix. In the second branch, the data undergoes 5x5 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting the large receptive field matrix. The main feature matrix and the large receptive field matrix are merged along the channel dimension to output a merged matrix. Simultaneously, the main feature matrix and the large receptive field matrix are summed to obtain a summation matrix. The merged matrix and the summation matrix are then concatenated by channel to complete the novel feature fusion, resulting in the following output features:
[0102]
[0103] Among them, P i ( x ) represents the feature output by the i-th layer. i =1, 2, ...; I is the IremulbNet network, Mish is the activation function, and F... iThe diagram illustrates the convolutional relationship of the intermediate hidden layers. P04 and P05 represent the results of concatenating and summing the dimensions of the feature matrix during feature extraction, respectively.
[0104] 2) The change in residual stress within the glass was measured using an ultrasonic contrast nonlinear coefficient method to obtain a time-domain image of the residual stress in the tempered glass product; see [link to relevant documentation]. Figure 4 And Figure 5, in which Figure 5A This is a time-domain image of glass stress. Figure 5B This is a frequency domain image of glass stress. Figure 5C The time-frequency domain image of glass stress is as follows:
[0105] 2.1) The ultrasonic transmitter 1 and the ultrasonic receiver 2 are mounted in pairs on multiple metal rails 6 with adjustable spacing. The parallel metal rails 6 form an ultrasonic transducer matrix with a certain number and spacing. The ultrasonic transmitter and the ultrasonic receiver are controlled by corresponding servo motors 7 and can move independently along the X, Y and Z coordinates.
[0106] 2.2) Determine whether the ultrasonic transmitter 1 and ultrasonic receiver 2 are applied to the test point on the tempered glass. If not, readjust the position; if yes, proceed to the next step.
[0107] 2.3) The periodic pulse sinusoidal signal generated by the ultrasonic image processing unit 4 is input to the transmitting device 1 to emit pulsed ultrasonic waves, which are received by the receiving device 2 after passing through the glass 8;
[0108] 2.4) The received ultrasonic signal is input into system 4. Based on the fact that the relative nonlinear coefficient value is proportional to the stress of the stress-free standard tempered glass, the relative nonlinear coefficient corresponding to the tempered glass is obtained.
[0109] 2.5) The ultrasonic image processing unit 4 correlates the nonlinear coefficient obtained in step (2.4) with its corresponding stress, plots the curve of the nonlinear coefficient with stress, obtains the image of the change of residual stress in the tempered glass, and then obtains the time domain data of the residual stress in the tempered glass product.
[0110] 3) Perform short-time Fourier transform (STFT) on the time-domain image of glass stress to obtain the time-frequency domain image, and establish a dataset;
[0111] 4) Enhance the image data in the dataset from step 3) to obtain the original matrix P; specifically:
[0112] The dataset obtained in step 3) is divided into training set, validation set and test set in a ratio of 8:1:1. The obtained data is then cropped and standardized according to the reset size and random size, and then augmented.
[0113] 5) Input the original matrix P from step 4) into the lightweight two-branch inverse residual network model for training, extract the features of the image, and obtain the weight model file;
[0114] 5.1) Input the original matrix P into the preprocessing unit, and after denoising by 5*5 convolution and 3*3 pooling, obtain the feature matrix P0;
[0115] 5.2) Input the feature matrix P0 into the first feature extraction unit (DI_Block×1), and perform feature extraction through a two-branch inverse residual module to obtain the inverse residual feature P1;
[0116] 5.2.1) Perform a 1*1 first convolution on the feature matrix P0 to obtain matrix P01;
[0117] 5.2.2) Matrix P01 is input to the first branch and the second branch respectively. In the first branch, after 3*3 depthwise convolution and 3*3 pooling, it is passed to the SE attention module or directly to the SE attention module for channel feature enhancement, and outputs the main feature matrix P02. In the second branch, after 5*5 depthwise convolution and 3*3 pooling, it is passed to the SE attention module or directly to the SE attention module for channel feature enhancement, and outputs the large receptive field matrix P03.
[0118] 5.2.3) Merge the main feature matrix P02 and the large receptive field matrix P03 extracted from the image in step 5.2.2) along the channel dimension to obtain the merged matrix P04. At the same time, sum P02 and P03 to obtain the summation matrix P05. Concatenate the merged matrix P04 and the summation matrix P05 along the channel, and perform a 1*1 second convolution pointwise convolution to reduce the dimension and obtain the dual-channel information matrix P07.
[0119] 5.2.4) Summing the dual-path information matrix P07 from step 5.2.3) with the feature matrix P0 after dimensional transformation, we obtain the inverse residual feature P1.
[0120] 5.3) Input the inverse residual feature P1 into the second feature extraction unit (DI_Block×2), and perform feature extraction through two cascaded double-branch inverse residual modules to obtain the inverse residual feature P2;
[0121] 5.4) Input the inverse residual feature P2 into the third feature extraction unit (DI_Block×3), and perform feature extraction through three cascaded double-branch inverse residual modules to obtain the inverse residual feature P3;
[0122] 5.5) Input the inverse residual feature P3 into the fourth feature extraction unit (DI_Block×2), and after feature extraction by two cascaded double-branch inverse residual modules, obtain the inverse residual feature P4;
[0123] 5.6) Input the inverse residual feature P4 into the adaptive average pooling layer, and then into the fully connected layer. Combine the extracted distribution features to form a one-dimensional feature vector, and output the weight model file after weighted summation of the feature vector.
[0124] 6) Load the weight model file trained in step 5) into the prediction program, input the time-frequency domain image of the stress of the tempered glass to be inspected, and obtain the detection result of the tempered glass sample by model recognition and prediction according to the method in steps 4) to 5).
[0125] To further explain, the characteristics of the output of the dual-branch inverted residual module in steps (5.2), (5.3), (5.4), and (5.5) above are as follows:
[0126]
[0127] Among them, P i ( x ) represents the feature output by the i-th layer. i =1, 2, 3, 4; I is the IremulbNet network, Mish is the activation function, F i The diagram illustrates the convolutional relationship of the intermediate hidden layers. P04 and P05 represent the results of concatenating and summing the dimensions of the feature matrix during feature extraction, respectively.
[0128] Example 2
[0129] Based on Example 1, this embodiment further includes step 7) after step 6). Step 7) specifically involves judging the test results of the tempered glass sample. When the surface stress is less than 70 MPa, the glass stress condition is unqualified and it is a defective product; when the surface stress is 70~90 MPa, the glass stress condition meets the national standard requirements for semi-tempered glass; when the surface stress is greater than 90 MPa, the glass stress condition meets the national standard requirements for tempered glass and it is a qualified product.
[0130] The advantages of the bi-branch inverse residual network model of the present invention are further compared with several existing common network models, and the results are as follows:
[0131] Experiment 1 Lightweight Model Analysis
[0132] The number of parameters and computational cost of the model are evaluated, and the parameters involved in the IretbNet model of this invention, such as convolution, activation, normalization and fully connected layers, are statistically analyzed.
[0133] Four network models were compared with the IretbNet of this invention in terms of parameter count, computational cost, and training time. The four network models are ResNet50, MobleNetV3, ShuffleNetV2, and RegNet.
[0134] During the evaluation process, the network input consisted of 3-channel 224*224 color images, and the performance analysis tool Profile was used to analyze the parameter count and model scaling of the four models. The results are shown in Figure 6.
[0135] from Figure 6A As can be seen from the model, the parameter count of the IretbNet model is 3.734 MB, while the classic ResNet50 model has 25.55 MB of parameters. Meanwhile, the lightweight MobleNetV3, ShuffleNetV2, and RegNet have parameter counts of 5.483 MB, 2.279 MB, and 2.685 MB, respectively. Compared to the ResNet50 model, IretbNet reduces the parameter count by 85.38%, and is comparable to RegNet in terms of parameter count.
[0136] exist Figure 6B In the test, using a 224*224 color image as input, IretbNet's computational cost was 579.311 MB, while ResNet50, MobleNetV3, ShuffleNetV2, and RegNet's computational costs were 4210.688 MB, 225.442 MB, 148.808 MB, and 203.384 MB, respectively. Compared to the ResNet50 model, IretbNet of this invention reduced the computational cost by 86.24%.
[0137] Experiment 2
[0138] In this experiment, the stress range of the glass was 30 MPa - 105 MPa. Based on the national tempered glass standard, the dataset consisted of 1855 images categorized into three classes: 600 images of substandard tempered glass (stress value less than 70 MPa), 540 images of semi-tempered glass (stress value greater than 70 MPa and less than 90 MPa), and 715 images of tempered glass (stress value greater than 90 MPa).
[0139] This experiment can monitor the production of tempered glass online in real time and promptly detect whether the produced glass is up to standard. The program divides the time-frequency domain image data of the glass into training, validation, and test sets in an 8:1:1 ratio. The acquired data is preprocessed by cropping and standardizing the data according to its original size and random size. Then, the IretbNet model provided in this invention is used to input the obtained image information into the network model to complete the classification. The model's parameters, processing time, and accuracy are recorded, and the results are shown in Table 1.
[0140] Meanwhile, four classic CNN models, namely ResNet50, MobleNetV3, ShuffleNetV2 and RegNet, were used to identify time-frequency domain images of glass with different stresses. The number of model parameters, computational cost and accuracy were recorded and compared. The results are shown in Table 1.
[0141] Table 1. Recognition performance of glass time-frequency domain images
[0142] Model accuracy Computational cost / MB Parameter count / MB ResNet50 0.973 4210.688 25.557 MobileNet V3 0.962 225.442 5.483 ShuffleNet V2 0.908 148.808 2.279 RegNet 0.903 203.384 2.685 IretbNet 0.978 579.311 3.734
[0143] Table 1 shows that the IretbNet model of this invention exhibits good performance even when the differences between data points are minimal. Its highest classification accuracy is 97.8%, followed by the classic model ResNet50. However, in terms of parameter count and computational cost, the computational cost of IretbNet is reduced by 86.24% and the number of parameters is reduced by 85.38% compared to ResNet50. In terms of classification accuracy, it outperforms the lightweight models MobileNet V3, ShuffleNet V2, and RegNet by 1.6%, 7.0%, and 7.5%, respectively.
[0144] The image classification model provided by this invention demonstrates that the reconstructed inverse residual structure and its feature fusion method have a significant impact on network performance. The designed depth and width parameters are reasonable, making the model both lightweight and efficient. The IretbNet model constructed and designed in this invention exhibits significant advantages in tempered glass detection, ensuring the continuity, non-destructive nature, accuracy, and efficiency of online detection.
[0145] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. A method for testing tempered glass based on a bi-branch inverse residual network model, characterized in that, Includes the following steps: 1) Construct a lightweight two-branch inverse residual network model; the lightweight two-branch inverse residual network model includes one or more two-branch inverse residual modules; the two-branch inverse residual modules include parallel branch one and branch two; the preprocessed data in the first branch is processed by 3*3 depthwise convolution, then by 3*3 pooling, and then passed to the SE attention module or directly to the SE attention module for channel feature enhancement, outputting the main feature matrix; in the second branch, it is processed by 5*5 depthwise convolution, then by 3*3 pooling, and then passed to the SE attention module or directly to the SE attention module for channel feature enhancement, outputting the large receptive field matrix; the main feature matrix and the large receptive field matrix are merged in the channel dimension, outputting the merged matrix; at the same time, the above main feature matrix and the large receptive field matrix are summed to obtain the summation matrix; the merged matrix and the summation matrix are concatenated by channel to complete the novel feature fusion; 2) The change of residual stress in the glass is measured by comparing the ultrasonic wave with the nonlinear coefficient, and the time-domain data of the residual stress in the tempered glass product is obtained. 3) Perform short-time Fourier transform on the time-domain image of glass stress to obtain the time-frequency domain image, and establish a dataset; 4) Enhance the image data in the dataset from step 3) to obtain the original matrix P; 5) Input the original matrix P from step 4) into a lightweight two-branch inverse residual network model for training, extract image features, and obtain the weight model file; specifically: 5.1) Input the original matrix P into the preprocessing unit, and after denoising by 5*5 convolution and 3*3 pooling, obtain the feature matrix P0; 5.2) Input the feature matrix P0 into the first feature extraction unit, and perform feature extraction through a two-branch inverse residual module to obtain the inverse residual feature P1; 5.3) Input the inverse residual feature P1 into the second feature extraction unit, and perform feature extraction through two cascaded double-branch inverse residual modules to obtain the inverse residual feature P2; 5.4) Input the inverse residual feature P2 into the third feature extraction unit, and perform feature extraction through three cascaded double-branch inverse residual modules to obtain the inverse residual feature P3; 5.5) Input the inverse residual feature P3 into the fourth feature extraction unit, and after feature extraction by two cascaded double-branch inverse residual modules, obtain the inverse residual feature P4; 5.6) Input the inverse residual feature P4 into the adaptive average pooling layer, and then into the fully connected layer. Combine the extracted distribution features to form a one-dimensional feature vector, and then output the weight model file after weighted summation of the feature vector. The characteristics of the output of the dual-branch inverse residual module in steps (5.2), (5.3), (5.4), and (5.5) above are as follows: Among them, P i ( x ) represents the feature output by the i-th layer. i =1, 2, 3, 4; I is the IremulbNet network, Mish is the activation function, F i The diagram illustrates the convolutional relationship of the intermediate hidden layers. P04 and P05 represent the results of concatenating and summing the dimensions of the feature matrix during feature extraction, respectively. 6) Load the weight model file trained in step 5) into the prediction program, input the time-frequency domain image of the stress of the tempered glass to be tested, and obtain the detection results of the tempered glass sample according to the method in steps 4) to 5).
2. The tempered glass testing method based on a bi-branch inverse residual network model according to claim 1, characterized in that, Step 1) The lightweight bi-branch inverse residual network model includes a preprocessing unit, a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, and a classification unit connected in sequence; the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit each contain one or more bi-branch inverse residual modules.
3. The tempered glass testing method based on a bi-branch inverse residual network model according to claim 1, characterized in that, Step 2) specifically refers to: 2.1) The ultrasonic transmitter and the ultrasonic receiver are mounted in pairs on multiple metal rails with adjustable spacing. The parallel metal rails form an ultrasonic transducer matrix with a certain number and spacing. The ultrasonic transmitter and the ultrasonic receiver are controlled by corresponding servo motors and can move independently along the X, Y and Z coordinates. 2.2) Determine whether the ultrasonic transmitter and receiver are loaded onto the test point on the tempered glass. If not, readjust their positions; if yes, proceed to the next step. 2.3) The periodic pulsed sinusoidal signal generated by the ultrasonic image processing unit is input to the ultrasonic transmitter to emit pulsed ultrasonic waves, which are received by the ultrasonic receiver after passing through the glass. 2.4) The received ultrasonic signal is input into the system. Based on the fact that the nonlinear coefficient is proportional to the stress of the stress-free standard tempered glass, the relative nonlinear coefficient of the tempered glass is obtained. 2.5) The ultrasonic image processing unit correlates the nonlinear coefficient obtained in step (2.4) with its corresponding stress, plots the curve of the nonlinear coefficient with stress, obtains the image of the change of residual stress in the tempered glass, and then obtains the time domain data of the residual stress in the tempered glass product.
4. The tempered glass testing method based on a bi-branch inverse residual network model according to claim 1, characterized in that, Step 4) specifically refers to: The dataset obtained in step 3) is divided into training set, validation set and test set in a ratio of 8:1:
1. The obtained data is then cropped and standardized according to the reset size and random size, and augmented to obtain the original matrix P.
5. The tempered glass testing method based on a bi-branch inverse residual network model according to claim 1, characterized in that, The specific training process in step 5.2) is as follows: 5.2.1) Perform a 1*1 first convolution on the feature matrix P0 to obtain matrix P01; 5.2.2) Matrix P01 is input to the first branch and the second branch respectively. In the first branch, after 3*3 depthwise convolution and 3*3 pooling, it is passed to the SE attention module or directly to the SE attention module for channel feature enhancement, and outputs the main feature matrix P02. In the second branch, after 5*5 depthwise convolution and 3*3 pooling, it is passed to the SE attention module or directly to the SE attention module for channel feature enhancement, and outputs the large receptive field matrix P03. 5.2.3) Merge the main feature matrix P02 and the large receptive field matrix P03 extracted from the image in step 5.2.2) along the channel dimension to obtain the merged matrix P04. At the same time, sum the main feature matrix P02 and the large receptive field matrix P03 to obtain the summation matrix P05. Concatenate the merged matrix P04 and the summation matrix P05 along the channel, and perform a 1*1 second convolution pointwise convolution to reduce the dimension and obtain the dual-channel information matrix P07. 5.2.4) Summing the dual-path information matrix P07 from step 5.2.3) with the feature matrix P01 after dimensional transformation, we obtain the inverse residual feature P1.
6. The tempered glass testing method based on a bi-branch inverse residual network model according to claim 1, characterized in that, Step 6) is followed by step 7), which specifically involves: judging the test results of the tempered glass sample. When the surface stress is less than 70 MPa, the glass stress condition is unqualified and it is a defective product; when the surface stress is 70~90 MPa, the glass stress condition meets the national standard requirements for semi-tempered glass. When the surface stress is greater than 90MPa, the stress condition of the glass meets the requirements of the national standard for tempered glass and is considered a qualified product.
7. A tempered glass inspection system based on a bi-branch inverse residual network model, characterized in that, Includes an ultrasonic testing device and an ultrasonic image processing unit; An ultrasonic testing device is set on both sides of the glass to be tested. It includes an ultrasonic transmitter and an ultrasonic receiver. The ultrasonic transmitter emits pulsed ultrasonic signals, which pass through the glass and are received by the ultrasonic receiver. The received signals are then transmitted to the ultrasonic image processing unit. An ultrasonic image processing unit is used to send periodic pulse sinusoidal signals to the ultrasonic transmitter of the ultrasonic testing device, and simultaneously receive ultrasonic signals from the ultrasonic receiver of the ultrasonic testing device. The unit processes the signals using the tempered glass testing method based on the bi-branch inverse residual network model as described in claim 1, and outputs the testing results of the tempered glass.
8. The tempered glass inspection system based on a bi-branch inverse residual network model according to claim 7, characterized in that, The ultrasound image processing unit includes a central processing module, a signal preprocessing module, and a dual-branch inverted residual signal processing module. Central processing module: Used to emit periodic pulse sinusoidal signals and receive ultrasonic signals, transmit the received ultrasonic signals to the signal preprocessing module, and receive the glass stress value transmitted by the dual-branch inverted residual signal processing module and output it. Signal preprocessing module: Receives the ultrasound signal sent by the central processing module, performs short-time Fourier transform on the time-domain data to obtain a time-frequency domain image, extracts the feature information of the time-frequency domain image, and forwards the information to the dual-branch inverse residual signal processing module; The dual-branch inverse residual signal processing module trains the enhanced image information in a dual-branch inverse residual network model, extracts the weight model file, loads the weight model file into the prediction program, compares it with the information of the glass stress to be detected, and transmits the glass stress value obtained from the comparison to the central processing module.
9. The tempered glass inspection system based on a bi-branch inverse residual network model according to claim 8, characterized in that, The lightweight bi-branch inverse residual network model includes a preprocessing unit, a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, and a classification unit connected in sequence; the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit are each composed of one or more bi-branch inverse residual modules; The dual-branch inverse residual module includes two parallel branches: Branch 1 and Branch 2. The preprocessed data in Branch 1 undergoes 3x3 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting a main feature matrix. In Branch 2, the data undergoes 5x5 depthwise convolution followed by 3x3 pooling before being passed to the SE attention module or directly to it for channel feature enhancement, outputting a large receptive field matrix. The main feature matrix and the large receptive field matrix are merged along the channel dimension to output a merged matrix. Simultaneously, the main feature matrix and the large receptive field matrix are summed to obtain a summation matrix. The merged matrix and the summation matrix are then concatenated by channel to complete the novel feature fusion.