Door and window glass defect feedback control method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device, and storage medium for feedback control of door and window glass defects, relating to the technical field of door and window glass defect detection technology. The method comprises the following steps: acquiring a first image after adjusting a polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles within the glass; acquiring a second image after adjusting the polarized light imaging detection device to enhance light reflected from the glass surface; calculating an adaptive weighting system by analyzing the local characteristics of the second image; performing weighted processing on the first image based on the adaptive weighting system, and subtracting the weighted first image from the second image to obtain a difference image; and extracting information about bubble defects within the glass by analyzing the difference image. The present invention overcomes interference caused by uneven reflection on the glass surface and improves the adaptability of image difference processing to suppress surface reflection background, thereby more effectively and accurately extracting image features of internal bubble defects.

Description

Technical Field

[0001] The present invention relates to the technical field of door and window glass defect detection, and in particular to a door and window glass defect feedback control method, device, electronic equipment and storage medium. Background Art

[0002] Customized, high-end doors and windows generally have higher quality requirements, especially for those with thick glass. Surface quality and internal defect control are crucial. To ensure product quality, production lines generally utilize optical inspection technology based on image sensors to achieve high-precision detection of glass defects and real-time feedback control. Existing technologies attempt to detect internal bubble defects using optical reflection principles. However, thick, transparent glass has two reflective surfaces, complicating the reflected light. The image captured by the image sensor inevitably mixes the reflected signals from the glass surface with those from internal bubbles, causing signal aliasing and severely impacting the accurate identification of internal bubble defects.

[0003] To reduce the interference of surface reflections, existing techniques have attempted to exploit the properties of polarized light. By controlling the polarization state of the incident light and the angle of the polarizer at the receiving end, it is possible to partially suppress light reflected from the glass surface, thereby relatively enhancing the light scattered by internal bubbles. Furthermore, image difference processing techniques are employed. This involves capturing two sets of images with different polarization states and performing a subtraction operation to eliminate the shared surface reflection background and highlight the internal bubble defects.

[0004] However, in actual applications, the surface quality of some thick glass is uneven, and simple image difference methods still have limitations. The reflection intensity and texture complexity of the glass surface may vary in different areas, resulting in a non-uniform surface reflection background. If a globally unified differential weight is used for processing, it will be difficult to effectively adapt to such local changes, which may lead to incomplete surface reflection suppression or excessive suppression in certain areas, which will affect the detection effect of defects. Especially in harsh scenarios where internal bubble defects need to be identified and characterized with high precision and real-time feedback is required, this problem of insufficient surface reflection suppression will seriously restrict the accuracy and reliability of defect detection, which may lead to the accidental outflow of defective products and cause consumer dissatisfaction. Summary of the Invention

[0005] The purpose of the present invention is to provide a door and window glass defect feedback control method, device, electronic device and storage medium, which solves the problem that the accuracy and reliability of defect detection are seriously restricted due to insufficient glass surface reflection suppression, overcomes the interference caused by the uneven reflection of the glass surface, and improves the adaptability of image difference processing to surface reflection background suppression, so as to more effectively and accurately extract the image features of internal bubble defects.

[0006] In a first aspect, the present invention provides a door and window glass defect feedback control method, which is applied to a production control system with a polarized light imaging detection device, comprising the following steps:

[0007] S1. After adjusting the polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles inside the glass, a first image is captured;

[0008] S2. After adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, a second image is collected;

[0009] S3. Calculating an adaptive weight group by analyzing local characteristics of the second image; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local second image;

[0010] S4. According to the adaptive weight group, weighting the first image is performed, and the weighted first image is subtracted from the second image to obtain a difference image;

[0011] S5. By analyzing the differential image, the bubble defect information inside the glass is extracted; the bubble defect information includes the location of the bubble defect, the size of the bubble defect and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and is used to indicate the existence of quality defects.

[0012] The door and window glass defect feedback control method provided by the present invention introduces an adaptive weight mechanism in the image difference processing process, dynamically adjusts the difference weight according to the characteristics of the local area of the image, and more finely suppresses the uneven surface reflection background, thereby more effectively highlighting the bubble defects inside the glass.

[0013] Furthermore, the specific steps in step S3 include:

[0014] S31. Dividing the second image into a plurality of non-overlapping image blocks;

[0015] S32. For each image block, calculate the gray level co-occurrence matrix of each image block, and extract corresponding texture feature parameters based on the gray level co-occurrence matrix, wherein the texture feature parameters include energy, contrast, correlation and entropy;

[0016] S33. For each image block, calculate the gradient magnitude of each image block, and count the number of pixels whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter;

[0017] S34. Predicting the reflection intensity weight value of each image block using a pre-trained prediction model based on the texture feature parameters and gradient complexity parameters of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters, and reflection intensity weight values corresponding to the image blocks at different reflection intensities, and the prediction model uses a cross-validation method for parameter optimization;

[0018] S35. Combining the reflection intensity weight values of all the image blocks into the adaptive weight group.

[0019] The adaptive weight group can more accurately characterize the local reflection intensity or texture complexity of the second image, laying the foundation for the subsequent generation of differential images and extraction of bubble defect information, thereby improving the accuracy and reliability of defect detection.

[0020] Furthermore, the specific steps in step S34 include:

[0021] S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, the texture feature parameter and the gradient complexity parameter to obtain a fused feature vector;

[0022] S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block based on the fused feature vector; the prediction model is a deep neural network comprising multiple hidden layers, the input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value, and the deep neural network is trained using a Dropout regularization method.

[0023] The Dropout method randomly discards some neurons, reducing the network's reliance on specific neurons and enhancing its generalization and robustness. As a result, the network can more stably predict reflection intensity weights even when there are minor defects like oil stains and scratches on the glass surface.

[0024] Furthermore, the deep neural network is trained using a Dropout regularization method through the following steps:

[0025] S3421. For each hidden layer of the deep neural network, randomly discard the output of the neurons in the layer with a probability of 50%, and generate a mask vector, and multiply the mask vector by the output of the neurons in the layer to obtain the neuron output after Dropout;

[0026] S3422. For each hidden layer, the neuron output of the hidden layer after Dropout is input to the output layer of the deep neural network, and a trained deep neural network is obtained after training; the deep neural network is used to output the reflection intensity weight value through a Sigmoid activation function, and the output reflection intensity weight value is between 0 and 1.

[0027] By applying the Dropout regularization method, deep neural networks can effectively reduce their sensitivity to minor defects such as oil stains and scratches on the glass surface, improving the accuracy and robustness of reflection intensity weight prediction under complex glass surface conditions. As a result, the weight prediction results become more reliable, ensuring the accuracy of subsequent defect detection.

[0028] Furthermore, the specific steps in step S3422 include:

[0029] S34221. For each hidden layer, calculate the weighted value of the hidden layer based on the connection weight between the hidden layer and the output layer and the neuron output after Dropout of the hidden layer;

[0030] S34222. Calculate a weighted average value based on the weighted values of all the hidden layers and represent it as a fusion feature;

[0031] S34223. Input the fused feature representation into the Sigmoid activation function, adjust the parameters of the Sigmoid function through training, and obtain a deep neural network that has completed training.

[0032] Furthermore, the specific steps in step S4 include:

[0033] S41. For the first image and the second image, accurately align them using an image registration algorithm based on mutual information and calculate a spatial transformation matrix;

[0034] S42. Performing a geometric transformation on the first image according to the spatial transformation matrix to eliminate the misalignment between the first image and the second image;

[0035] S43. Performing weighted processing on the first image after geometric transformation according to the adaptive weight group to obtain a weighted first image;

[0036] S44. Subtracting the weighted first image from the second image to obtain an initial difference image;

[0037] S45. Perform multi-scale morphological filtering on the initial differential image to suppress artifacts caused by registration errors, thereby obtaining the differential image.

[0038] Furthermore, the specific steps in step S5 include:

[0039] S51. Adaptively filtering the differential image to reduce noise interference;

[0040] S52. Segmenting the filtered difference image using a region growing-based image segmentation algorithm to obtain multiple candidate bubble regions;

[0041] S53. For each candidate bubble region, shape characteristic parameters are calculated; the shape characteristic parameters include area, perimeter, roundness and convexity;

[0042] S54. Based on the shape characteristic parameters, a pre-trained classifier is used to determine whether each candidate bubble region is a real bubble defect. If the determination is yes, bubble defect information of the real bubble defect is recorded; otherwise, the bubble defect information is not recorded. The bubble defect size in the bubble defect information is extracted by analyzing the shape characteristic parameters of the corresponding candidate bubble region.

[0043] S55. Feedback the bubble defect information to the production control system in real time.

[0044] In a second aspect, the present invention provides a door and window glass defect feedback control device, which is applied to a production control system with a polarized light imaging detection device, comprising:

[0045] A first acquisition module is configured to acquire a first image after adjusting the polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles inside the glass;

[0046] a second acquisition module, configured to acquire a second image after adjusting the polarized light imaging detection device to enhance the reflected light from the glass surface;

[0047] An analysis and calculation module, configured to calculate an adaptive weight group by analyzing local characteristics of the second image; the adaptive weight group is used to characterize the local reflection intensity or texture complexity of the second image;

[0048] a difference module, configured to perform weighted processing on the first image according to the adaptive weight group, and subtract the weighted first image from the second image to obtain a difference image;

[0049] An analysis and feedback module is used to extract bubble defect information inside the glass by analyzing the differential image; the bubble defect information includes the location of the bubble defect, the size of the bubble defect and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and is used to indicate the existence of quality defects.

[0050] The door and window glass defect feedback control device provided by the present invention can better adapt to the local unevenness of glass surface reflection, improve the surface reflection suppression effect, and thus improve the accuracy and reliability of internal bubble defect detection.

[0051] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are executed.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, runs the steps of the method provided in the first aspect above.

[0053] From the above, it can be seen that the door and window glass defect feedback control method provided by the present invention can dynamically adjust the differential weight according to the local characteristics of the image through an adaptive weight mechanism, and more finely suppress the surface reflection background, especially for thick glass with uneven surface reflection. The effect is significant, making the image features of internal bubble defects clearer and more prominent, thereby improving the signal-to-noise ratio and accuracy of defect detection, thereby better and strictly controlling product quality, effectively avoiding the accidental outflow of defective products, and helping to improve consumers' consumption experience.

[0054] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the embodiments of the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flow chart of a door and window glass defect feedback control method provided by an embodiment of the present invention.

[0056] Figure 2 A schematic structural diagram of a door and window glass defect feedback control device provided in an embodiment of the present invention.

[0057] Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0058] Description of labels:

[0059] 100, first acquisition module; 200, second acquisition module; 300, analysis and calculation module; 400, difference module; 500, analysis and feedback module; 13, electronic device; 1301, processor; 1302, memory; 1303, communication bus. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0061] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance.

[0062] Reference Attachment Figure 1 The present invention provides a door and window glass defect feedback control method, which is applied to a production control system with a polarized light imaging detection device, and includes the following steps:

[0063] S1. After adjusting the polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles inside the glass, a first image is captured;

[0064] S2. After adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, a second image is captured;

[0065] S3. Calculating an adaptive weight group by analyzing local characteristics of the second image; an adaptive weight group is used to characterize the reflection intensity or texture complexity of the local second image;

[0066] S4. According to the adaptive weight group, the first image is weighted and the weighted first image is subtracted from the second image to obtain a difference image;

[0067] S5. Extract bubble defect information inside the glass by analyzing the differential image; the bubble defect information includes the location, size, and number of bubble defects; the bubble defect information is fed back to the production control system in real time and used to indicate the presence of quality defects.

[0068] In step S1, the adjustment of the polarized light imaging detection device can be achieved by controlling the polarization state of the incident light and the angle of the polarizer at the receiving end. The purpose is to reduce the reflected light on the glass surface and enhance the scattered light of bubbles inside the glass, thereby collecting a first image that mainly contains internal bubble defect information.

[0069] In step S2, the polarized light imaging detection device is adjusted to a state of enhancing the reflected light from the glass surface, in order to collect a second image mainly containing the reflected light information from the glass surface.

[0070] In step S3, the calculation of the adaptive weight group is based on dividing the second image into image blocks and analyzing the local characteristics of each image block. The local characteristics may include texture feature parameters and gradient complexity parameters of the image block. The value of the adaptive weight group is associated with the reflection intensity or texture complexity of the image block.

[0071] In step S4, weighted processing of the first image is performed based on the adaptive weight group calculated in step S3, with the purpose of applying different weights to different areas of the first image so as to more accurately eliminate the influence of surface reflection in subsequent differential operations and obtain a differential image that can highlight internal bubble defects.

[0072] Specifically, assuming that the size of the first image is 1280x1280 pixels, the adaptive weight group is composed of a 40x40 weight matrix, and each weight value corresponds to an image block of 32x32 pixels.

[0073] First, the first image is divided into 40x40 blocks of 32x32 pixels each. The division process can be performed from left to right and from top to bottom. The first block is located in the upper left corner of the first image, at coordinates (0,0), and has a size of 32x32 pixels. The second block is located to the right of the first block, at coordinates (32,0), and also has a size of 32x32 pixels. This process continues until all 40 blocks in the first row are divided. Then, starting from the second row, the block division process continues until the entire first image is divided.

[0074] Next, each weight value in the adaptive weight group is applied to the corresponding 32x32 pixel image block. For example, the weight value at (0,0) in the adaptive weight group is applied to the 32x32 pixel image block at coordinates (0,0) in the first image. The weight value at (1,0) in the adaptive weight group is applied to the 32x32 pixel image block at coordinates (32,0) in the first image. This process continues until all weight values in the adaptive weight group have been applied to the corresponding image blocks.

[0075] Weighting can be achieved by multiplying the grayscale value of each pixel in the image block by the corresponding weight value. For example, if the grayscale value of the pixel at coordinates (0,0) in the first image is 100 and the corresponding weight value is 0.5, then the grayscale value of this pixel becomes 100*0.5=50 after weighting. This weighting process is performed on all pixels in each image block to complete the weighting of the first image.

[0076] In this way, each local area of the first image is adjusted according to the adaptive weight set, which can better eliminate the influence of surface reflections and highlight internal bubble defects.

[0077] In step S5, the analysis and processing of the differential image includes filtering, segmentation, and shape feature parameter calculation of the differential image. The purpose is to extract the bubble defect information inside the glass from the differential image and feed the bubble defect information back to the production control system to achieve real-time monitoring and feedback control of defects.

[0078] Specifically, this method uses a polarized light imaging detection device. First, two images are captured by adjusting the device: the first image is captured in a state where surface reflected light is suppressed and internal bubble scattered light is enhanced, and the second image is captured in a state where surface reflected light is enhanced. Then, the local characteristics of the second image are analyzed and an adaptive weight group is calculated. This function characterizes the reflection intensity or texture complexity of the local area of the second image. The adaptive weight group is then used to perform weighted processing on the first image. The weighted first image is then differentially calculated with the second image to eliminate the common surface reflection background, thereby highlighting the internal bubble defects and obtaining a differential image. Finally, the differential image is analyzed to extract information such as the location, size, and number of the bubble defects, and this information is fed back to the production control system in real time for quality control. Due to the introduction of the adaptive weight group, this method can better adapt to the local unevenness of the glass surface reflection, improve the surface reflection suppression effect, and thus improve the accuracy and reliability of internal bubble defect detection.

[0079] In some specific embodiments, for online quality inspection of door and window glass, the production control system is equipped with a polarized light imaging inspection device. During the inspection process, the polarizer angle of the polarized light imaging inspection device is first adjusted to suppress light reflected from the glass surface and enhance light scattered by bubbles within the glass, thereby capturing a first image. Subsequently, the polarizer angle of the polarized light imaging inspection device is adjusted to enhance light reflected from the glass surface, thereby capturing a second image. After image acquisition, the system analyzes the second image, dividing it into multiple non-overlapping image blocks of 32x32 pixels. For each image block, the gray-level co-occurrence matrix is calculated and four texture feature parameters (energy, contrast, correlation, and entropy) are extracted. The gradient amplitude is also calculated and the gradient complexity parameter is calculated. Using a pre-trained prediction model, the reflection intensity weight value for each image block is predicted based on the texture feature parameters and the gradient complexity parameter. The weight values of all image blocks are combined into an adaptive weighting set. The first image is then weighted according to the adaptive weighting set, and the weighted first image is subtracted from the second image to obtain an initial difference image. Multi-scale morphological filtering is performed on the initial difference image to eliminate artifacts caused by registration errors, resulting in a final difference image. The difference image is analyzed, and adaptive filtering is used to reduce noise. A region growing algorithm is then used to segment candidate bubble regions, and shape feature parameters such as area, perimeter, roundness, and convexity are calculated. Finally, a pre-trained classifier is used to determine whether a bubble defect is a true bubble based on these shape feature parameters. If so, the location, size, and number of the defect are recorded, and this information is fed back to the production control system in real time to indicate a quality defect. This approach enables automated detection and feedback control of bubble defects within window and door glass.

[0080] In some embodiments, the specific steps in step S3 include:

[0081] S31. Dividing the second image into a plurality of non-overlapping image blocks;

[0082] S32. For each image block, calculate the gray level co-occurrence matrix of each image block, and extract the corresponding texture feature parameters based on the gray level co-occurrence matrix, the texture feature parameters including energy, contrast, correlation and entropy;

[0083] S33. For each image block, calculate the gradient magnitude of each image block and count the number of pixels whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter;

[0084] S34. Based on the texture feature parameters and gradient complexity parameters of each image block, a pre-trained prediction model is used to predict the reflection intensity weight value of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters and reflection intensity weight values corresponding to the image blocks under different reflection intensities, and the prediction model uses a cross-validation method for parameter optimization;

[0085] S35. The reflection intensity weight values of all image blocks are combined into an adaptive weight group (it should be noted that the weight value of each block is mapped back to its original spatial position to construct a global weight matrix, and finally a block constant value matrix is generated as the adaptive weight group. The adaptive weight group is spatially distributed in a checkerboard pattern, and the weight of each block reflects its local characteristics. It should be noted that the adaptive weight group can actually be regarded as a function on the image area. The image is divided into multiple image blocks, and each image block corresponds to a function value. The function value is the reflection intensity weight value of the image block. The adaptive weight group can be based on the local characteristics of the image block. Characteristics, adaptively adjust the weight value, the weight value is not fixed, but changes with the change of local characteristics of the image, so it meets the definition of a function. At the same time, the adaptive weight group can be expressed as a mapping relationship from the image block position to the reflection intensity weight value. The position coordinates of each image block correspond to a reflection intensity weight value. This mapping relationship also meets the definition of a function. Therefore, although the adaptive weight group is composed of multiple numerical values, it has the basic characteristics of a function and can adaptively adjust the weight value according to the local characteristics of the image. It can also be expressed as a mapping relationship from the image block position to the reflection intensity weight value, so it can also be regarded as a function).

[0086] In step S31 , the second image is divided into multiple regions, providing a basis for subsequent analysis of local characteristics of the image.

[0087] In step S32, for each image block, a gray level co-occurrence matrix is calculated to extract texture feature parameters such as energy, contrast, correlation and entropy. The texture feature parameters can quantify the local texture information of the image.

[0088] In step S33, the gradient amplitude and the gradient complexity parameter are calculated. The gradient complexity parameter is obtained by counting the number of pixels whose gradient amplitude exceeds a threshold value, and is used to characterize the degree of change in the local reflection intensity of the image.

[0089] In step S34, a prediction model is used, and the training samples of the prediction model include texture feature parameters, gradient complexity parameters and reflection intensity weight values. The prediction model realizes the prediction of reflection intensity weight values by learning the mapping relationship between texture features, gradient complexity and reflection intensity weight values. The cross-validation method is used to optimize the prediction model parameters and improve the model generalization ability.

[0090] In step S35 , the reflection intensity weight values of all image blocks are combined to form an adaptive weight group, which is used in subsequent image processing steps.

[0091] Specifically, this embodiment aims to solve the problem that the adaptive weight group cannot accurately represent the actual surface reflection situation due to uneven illumination and uneven glass thickness. First, the second image is divided into multiple non-overlapping image blocks, which realizes the analysis of the local characteristics of the image. Then, for each image block, the texture feature parameters and gradient complexity parameters are extracted. The texture feature parameters and gradient complexity parameters can reflect the reflection intensity and texture complexity of the local area of the image. Subsequently, the pre-trained prediction model is used to predict the reflection intensity weight value of each image block based on the texture feature parameters and gradient complexity parameters. The prediction model can learn the mapping relationship between texture features, gradient complexity and reflection intensity weight values to realize the prediction of adaptive weight values. Finally, the reflection intensity weight values of all image blocks are combined into an adaptive weight group. The adaptive weight group can more accurately represent the local reflection intensity or texture complexity of the second image, laying the foundation for the subsequent generation of differential images and the extraction of bubble defect information, thereby improving the accuracy and reliability of defect detection.

[0092] In some specific embodiments, the size of the image block is set to 32x32 pixels. In the gray level co-occurrence matrix calculation, the distance between pixel pairs is set to 1, and the directions include 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The gradient amplitude is calculated by the Sobel operator, and the preset threshold is set to 1.5 times the average value of the gradient amplitude. The prediction model is, for example, a support vector regression model using a radial basis kernel function, and the optimal model parameters are determined by a 5-fold cross validation method. The training samples are obtained by collecting image blocks of glass surfaces with different reflection intensities and manually annotating the reflection intensity weight values.

[0093] In some embodiments, the specific steps in step S34 include:

[0094] S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, texture feature parameters and gradient complexity parameters to obtain a fused feature vector;

[0095] S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block based on the fused feature vector; the prediction model is a deep neural network containing multiple hidden layers (it should be noted that traditional support vector regression models usually use kernel functions such as radial basis kernel functions to map input data to a high-dimensional feature space, and then perform linear regression in this space. However, in actual applications, traditional kernel functions may not be able to effectively extract the features of the input data. Therefore, this embodiment chooses to use a deep neural network to predict the reflection intensity weight value). The input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value. The deep neural network is trained using the Dropout regularization method to reduce the interference of minor defects such as oil stains and scratches on the glass surface on the weight value prediction.

[0096] For each image block, a local binary pattern (LBP) feature is calculated. Texture feature parameters and gradient complexity parameters are fused with the LBP feature to generate a fused feature vector. This fused feature vector is then fed into a pre-trained prediction model to predict the reflection intensity weight. The prediction model is a deep neural network consisting of multiple hidden layers. The deep neural network takes the fused feature vector as input and outputs the reflection intensity weight. Dropout regularization is used during training to reduce the impact of minor defects such as oil stains and scratches on the weight prediction.

[0097] Specifically, the local binary pattern (LBP) feature is introduced as an effective way to describe the local texture information of an image to more comprehensively characterize the local characteristics of an image block, while using texture feature parameters and gradient complexity parameters. By fusing these three features, a multi-dimensional and more accurate description of the local characteristics of an image block can be achieved. The deep neural network uses its powerful nonlinear fitting ability to learn the complex mapping relationship between the fusion feature vector and the reflection intensity weight value, thereby improving the accuracy of the weight value prediction. In order to reduce the interference of minor defects such as oil stains and scratches on the glass surface on the weight value prediction, the Dropout regularization method is introduced when training the deep neural network. During the training process, the Dropout method randomly discards a part of the neurons, reduces the network's dependence on specific neurons, and enhances the network's generalization ability and robustness. As a result, even if there are minor defects such as oil stains and scratches on the glass surface, the network can more stably predict the reflection intensity weight value.

[0098] In some specific embodiments, for each image block, the local binary pattern (LBP) feature is first calculated. The specific process may be as follows: Taking each pixel in the image block as the center, for example, a 3x3 neighborhood is used to compare the grayscale values of the neighboring pixels with the grayscale value of the center pixel. If the grayscale value of the neighboring pixel is greater than that of the center pixel, it is marked as 1; otherwise, it is marked as 0. This results in an 8-bit binary number, which is the LBP value of the center pixel. The LBP values of all pixels in the image block are counted to form an LBP histogram, which can be used as the LBP feature of the image block. The LBP features, texture feature parameters, and gradient complexity parameters are concatenated into a fused feature vector. A deep neural network with three hidden layers is constructed, with each layer containing 128 neurons. Dropout regularization is applied to each hidden layer, with the dropout ratio set to 50%. The deep neural network is trained using a large amount of sample data, including the fused feature vectors corresponding to the image blocks at different reflection intensities and reflection intensity weight values. The trained deep neural network can be used to predict the reflection intensity weight values of new image blocks.

[0099] In some embodiments, the deep neural network is trained using the Dropout regularization method by the following steps:

[0100] S3421. For each hidden layer of the deep neural network, randomly discard the output of the neurons in that layer with a probability of 50%, and generate a mask vector. The mask vector is multiplied by the output of the neurons in that layer to obtain the neuron output after dropout.

[0101] S3422. For each hidden layer, the neuron output after Dropout of the hidden layer is input to the output layer of the deep neural network, and a trained deep neural network is obtained after training; the deep neural network is used to output a reflection intensity weight value through a Sigmoid activation function, and the output reflection intensity weight value is between 0 and 1.

[0102] When using a deep neural network to predict the reflection intensity weight value, the deep neural network is prone to overfitting during training, resulting in sensitivity to minor defects such as oil stains and scratches on the glass surface, which affects the accuracy of weight value prediction. This embodiment proposes an improvement plan for this.

[0103] In step S3421, the Dropout regularization method is applied to each hidden layer of the deep neural network. This method randomly discards the output of neurons in that layer with a probability of 50%. This random discarding of neuron outputs is achieved by generating a mask vector and multiplying it by the neuron output to obtain the neuron output after Dropout. Dropout technology reduces the network's dependence on specific neurons by randomly omitting some neurons during training, thereby enhancing the network's generalization ability and reducing the risk of overfitting.

[0104] In step S3422, the outputs of all hidden layer neurons that have undergone dropout processing are input to the output layer of the deep neural network for training, resulting in the final deep neural network. The deep neural network is configured to output reflection intensity weight values using a sigmoid activation function, and the weight values are limited to between 0 and 1. The sigmoid function ensures that the output weight values are within a reasonable range, facilitating subsequent weighting processing.

[0105] Specifically, this embodiment introduces the Dropout regularization method in the training process of the deep neural network. In each hidden layer of the deep neural network, the output of the neuron is randomly discarded with a probability of 50%. This random discarding reduces the network's sensitivity to noise and irrelevant details in the training data by forcing the network to learn more robust and generalized feature representations. The output of the hidden layer neurons that have undergone Dropout processing is then input into the output layer, and the final deep neural network model is obtained through training. The deep neural network model uses the Sigmoid activation function to output the reflection intensity weight value between 0 and 1. Through the application of the Dropout regularization method, the deep neural network can effectively reduce the sensitivity to minor defects such as oil stains and scratches on the glass surface, and improve the accuracy and robustness of the reflection intensity weight value prediction under complex glass surface conditions. As a result, the weight value prediction result becomes more reliable, which provides a guarantee for the accuracy of subsequent defect detection.

[0106] In some specific embodiments, the deep neural network can be a multilayer perceptron comprising three hidden layers. During the training phase, the Dropout regularization method is applied to each hidden layer, and 50% of the neuron outputs are randomly discarded. For example, for the first hidden layer, a mask vector with the same dimension as the neuron output of this layer is generated, and each element in the vector is 0 with a probability of 50%, and the rest are 1. The mask vector is multiplied element-by-element by the neuron output to achieve random discarding of the neuron output. The same operation is applied to the second and third hidden layers. The outputs of all hidden layer neurons after Dropout processing are sent to the output layer. The output layer uses the Sigmoid activation function to map the fused features to a reflection intensity weight value between 0 and 1. Through the backpropagation algorithm and the gradient descent optimization method, the parameters of the deep neural network are continuously adjusted and optimized to complete the training of the deep neural network. The trained deep neural network can more accurately predict the reflection intensity weight value of each image block and reduce the interference of small defects on the glass surface on the weight value prediction.

[0107] It should be noted that the input of each hidden layer of the present invention is a fused feature vector, and all hidden layers are connected to the output layer. This means that the output of each hidden layer is used to calculate the final fused feature representation and input into the Sigmoid activation function to obtain the reflection intensity weight value. Therefore, each hidden layer independently extracts features from the fused feature vector and directly contributes the extracted feature information to the final output result. This structure allows the network to learn features from multiple different levels of abstraction and fuse these features to make more accurate predictions of the reflection intensity weight value.

[0108] In some embodiments, the specific steps in step S3422 include:

[0109] S34221. For each hidden layer, calculate the weighted value of the hidden layer based on the connection weight between the hidden layer and the output layer and the neuron output after Dropout of the hidden layer;

[0110] S34222. Calculate a weighted average value based on the weighted values of all hidden layers and represent it as a fusion feature;

[0111] S34223. Input the fused feature representation into the Sigmoid activation function, adjust the parameters of the Sigmoid function through training, and obtain a trained deep neural network.

[0112] In step S34221, a weighted value is calculated for each hidden layer, which takes into account the degree of influence of each hidden layer on the final output. Specifically, the connection weight reflects the sensitivity of the output layer to the output of each hidden layer neuron. By applying the connection weight to the neuron output after dropout, the contribution of each hidden layer to the overall network can be quantified.

[0113] In step S34222, the weighted values of all hidden layers are integrated and a weighted average is calculated as the fused feature representation. Thus, information from different hidden layers is effectively fused to form a comprehensive feature vector that can more comprehensively represent the multi-level features learned by the deep neural network.

[0114] In step S34223, the fused feature representation is input to the Sigmoid activation function, which maps the feature value to the range of 0 to 1, thus achieving the initial range limitation of the reflection intensity weight value. Further, by adjusting the parameters of the Sigmoid function, for example, the Sigmoid function is expressed as ,in This application adjusts the learnable parameters to shift the output curve of the Sigmoid function, adjust the activation threshold, and enhance the expressiveness of the model. Specifically, ,in, is the preset weight, Represented as transpose, For input data, For the bias term, in practical applications, changing the bias term can adjust the decision boundary of the Sigmoid function. For example, when the bias term is 0, When (i.e., no bias), the activation threshold is the origin of the input control, and the decision boundary is ; The bias term is not 0, when When (introducing bias), the activation threshold , the decision boundary becomes Setting the bias term allows the decision boundary to shift freely, adapting to more complex data distributions. Adjusting the parameters of the Sigmoid function further controls the output range of the reflection intensity weight value, resulting in a more accurate output. This ensures that the weight value is neither too small, causing the weighting process to fail, nor too large, excessively suppressing the information of the first image.

[0115] Specifically, the purpose of this embodiment is to solve the problem of uncontrollable output range of reflection intensity weight values. During the training process of deep neural networks, directly inputting the output of neurons after Dropout of all hidden layers into the output layer may cause the output reflection intensity weight value range to be too wide or offset, affecting the accuracy of subsequent defect detection. In order to solve this problem, this embodiment finely controls the output range of weight values through three steps. First, the weighted value of each hidden layer is calculated to evaluate the importance of each layer. Then, the information of all hidden layers is fused by weighted averaging to obtain a comprehensive feature representation. Finally, the Sigmoid activation function and bias term adjustment are used to more accurately control the output of the reflection intensity weight value. In this way, it is ensured that the adaptive weight group can play an effective role in the subsequent steps, thereby improving the accuracy and reliability of defect detection.

[0116] In some specific embodiments, the deep neural network includes two hidden layers (denoted as H1 and H2), each hidden layer includes 3 neurons, represented as:

[0117] H1 neuron output: [0.9, 0.6, 0.7];

[0118] H2 neuron output: [0.3, 0.8, 0.6];

[0119] After applying Dropout, it is expressed as:

[0120] Assuming that the first and third neurons in H1 are not discarded, the output becomes: [0.9, 0, 0.7];

[0121] Assuming that the second and third neurons in H2 are not discarded, the output becomes: [0, 0.8, 0.6];

[0122] The connection weights between the output layer and each hidden layer are expressed as:

[0123] Weights from H1 to output layer = [0.8, 0.2, 0.5];

[0124] H2 to output layer weight = [0.4, 0.6, 0.3];

[0125] The weighted value calculated for each hidden layer is expressed as:

[0126] 0.8*0.9+0.2*0+0.5*0.7=1.07;

[0127] 0.4*0+0.6*0.8+0.3*0.6=0.66;

[0128] The calculated weighted average (i.e., fused feature representation) is equal to (1.07+0.66) / 2=0.865.

[0129] In step S34223, the bias term of the Sigmoid activation function is set to -0.5, so that when the fused feature is represented as 0, the output of the Sigmoid function is close to 0.38, and when the fused feature is positive infinity, the output is close to 1. Through such bias term adjustment, and the characteristics of the Sigmoid function itself, the output range of the reflection intensity weight value is ultimately further controlled, which is conducive to outputting more accurate reflection intensity weight values. For example, the actual output range may be [0.15, 0.85], ensuring that the weight value fluctuates within a reasonable range and preventing extreme values from affecting the defect detection effect.

[0130] In some embodiments, the specific steps in step S4 include:

[0131] S41. For the first image and the second image, an image registration algorithm based on mutual information is used to accurately align the image and calculate the spatial transformation matrix;

[0132] S42. Performing a geometric transformation on the first image according to the spatial transformation matrix to eliminate the misalignment between the first image and the second image;

[0133] S43. According to the adaptive weight group, weighting is performed on the first image after the geometric transformation to obtain a weighted first image;

[0134] S44. Subtract the weighted first image from the second image to obtain an initial difference image;

[0135] S45. Perform multi-scale morphological filtering on the initial differential image to suppress artifacts caused by registration errors, thereby obtaining a differential image.

[0136] In step S41, a mutual information registration algorithm is used to achieve precise alignment of the first and second images. This is an image registration technique that effectively handles grayscale differences and geometric deformations between images. In specific implementations, normalized mutual information can be used as a similarity metric, and spatial transformation parameters are iteratively calculated using an optimization algorithm, such as gradient descent, to ultimately obtain a spatial transformation matrix. This spatial transformation matrix can represent geometric transformations such as translation, rotation, or scaling.

[0137] In step S42, a geometric transformation is performed on the first image based on the spatial transformation matrix calculated in step S41. The geometric transformation process aims to eliminate misalignment between the first and second images caused by shooting angles or positional deviations. As an embodiment, an affine transformation can be used to process the first image so that the transformed first and second images are precisely aligned in space.

[0138] In step S45, multiscale morphological filtering is used to suppress artifacts caused by possible registration errors in the initial difference image. Multiscale morphological filtering uses structuring elements of different sizes and shapes to perform morphological operations such as erosion and dilation, effectively removing noise and small artifacts from the image while preserving the image's primary structural information. For example, circular or square structuring elements of different sizes can be used, and opening and closing operations can be performed on each element separately. The results are then fused to achieve multiscale filtering.

[0139] Specifically, for glass defect detection applications, after acquiring the first and second images, the image registration operation in step S41 is first performed. For example, open-source image processing libraries such as OpenCV can be used to implement mutual information-based image registration. The registration process specifically involves inputting the first and second images, setting mutual information as the similarity metric, selecting an affine transformation model, setting parameters such as the number of iterations and the convergence threshold, and running the registration algorithm to obtain a spatial transformation matrix. Then, in step S42, the obtained spatial transformation matrix is input into the image geometry transformation function to perform a coordinate transformation on each pixel in the first image, thereby achieving geometric correction of the first image. Subsequently, in step S43, the aligned first image is weighted according to the adaptive weighting scheme described above. In step S44, image subtraction is performed to subtract the weighted first image from the second image to obtain an initial difference image. Finally, in step S45, the initial difference image is subjected to multi-scale morphological filtering. For example, a multi-scale morphological filter can be designed that contains circular structuring elements at three scales, with scales set to 3x3, 5x5, and 7x7, respectively. These structural elements are used to perform opening and closing operations on the initial differential image, and all resulting images are weighted averaged to obtain the final differential image. Through these steps, a differential image with effectively suppressed artifacts can be obtained, providing high-quality image data for the subsequent accurate extraction of bubble defects.

[0140] In some specific embodiments, for the feedback control method of door and window glass defects, in order to solve the problem of artifacts in the differential results caused by slight image misalignment, step S4 is designed to include a process of image registration and multi-scale morphological filtering. In the image registration link, the mutual information algorithm is selected as the registration algorithm because the mutual information is insensitive to changes in image grayscale and can better adapt to the polarization illumination differences that may exist between the two sets of images. The spatial transformation matrix adopts an affine transformation model, which can better correct image misalignment such as translation, rotation and scaling. In the multi-scale morphological filtering link, the shape of the structural element of the filter is selected as a circle, and the size is set to three scales of 3x3, 5x5 and 7x7, which can effectively remove small noise and artifacts introduced by residual errors of registration, while retaining the edge information of the bubble defect as much as possible. Through the synergistic effect of image registration and multi-scale morphological filtering, the quality of the differential image is improved, the interference of artifacts is reduced, and the subsequent extraction of bubble defects is more accurate and reliable.

[0141] In some embodiments, the specific steps in step S5 include:

[0142] S51. Performing adaptive filtering on the differential image to reduce noise interference;

[0143] S52. Segmenting the filtered difference image using a region growing-based image segmentation algorithm to obtain multiple candidate bubble regions;

[0144] S53. For each candidate bubble region, shape feature parameters are calculated; shape feature parameters include area, perimeter, roundness and convexity;

[0145] S54. Based on the shape feature parameters, a pre-trained classifier is used to determine whether each candidate bubble region is a true bubble defect. If the true bubble defect is determined, the bubble defect information is recorded; otherwise, the bubble defect information is not recorded. The bubble defect size in the bubble defect information is extracted by analyzing the shape feature parameters of the corresponding candidate bubble region (area, perimeter, roundness, and convexity, etc., can effectively describe the geometric shape characteristics of the candidate region. For example, area reflects the size of the candidate region, perimeter reflects the outline length of the candidate region, roundness reflects the degree to which the candidate region's shape is close to a circle, and convexity reflects the degree of convexity and concavity of the candidate region's shape).

[0146] S55. Feedback bubble defect information to the production control system in real time.

[0147] Adaptive filtering is implemented to address potential noise interference in the difference image. Adaptive filtering adjusts filtering parameters based on the noise level and texture characteristics of a local image region, effectively removing noise while better preserving image detail. For example, methods such as Wiener filtering or adaptive median filtering can be used. The filtered difference image is then used for subsequent bubble defect detection, reducing noise interference on defect identification. Furthermore, to screen potential bubble defect regions from the filtered difference image, an image segmentation algorithm based on region growing is employed. This algorithm begins with one or more seed pixels in the image and gradually merges surrounding pixels with similar characteristics into the same region. In this approach, seed pixels are automatically selected based on high grayscale value regions in the difference image, and the similarity criterion is set as the difference between the pixel grayscale value and the seed pixel grayscale value being less than a preset threshold. This generates multiple connected candidate bubble regions, enabling preliminary localization of bubble defects. To distinguish true bubble defects from non-defective regions, multiple shape feature parameters are calculated for each candidate bubble region. Shape feature parameters include area, which describes the size of the candidate region; perimeter, which describes the length of the candidate region's outline; roundness, which describes the degree to which the candidate region's shape approximates a circle; and convexity, which describes the degree of convexity or concavity in the candidate region's shape. These shape feature parameters effectively characterize the geometric characteristics of the candidate region. Furthermore, to automatically identify true bubble defects, a pre-trained classifier is employed. The classifier uses the shape feature parameters of the candidate bubble region as input and automatically classifies the candidate region by learning the features of a large number of bubble defect and non-defect samples. The classifier can use models such as support vector machines, random forests, or deep neural networks. The classifier outputs a judgment result, indicating whether the candidate region is a true bubble defect. If the judgment result is yes, the bubble defect information, including the location, size, and number of the bubble defects, is recorded. Otherwise, the bubble defect information is not recorded. Ultimately, the bubble defect information is fed back to the production control system in real time, allowing it to promptly monitor the quality status of glass products and make appropriate production adjustments.

[0148] Specifically, for the denoised differential image, the seed point for region growth is first determined. The seed point can be selected from the pixel points in the differential image whose grayscale value is higher than the preset threshold. The region growing process adopts an 8-neighborhood connection method, and the similarity criterion is set to the grayscale value difference between the current pixel and the seed pixel is less than 30. In this way, a series of candidate bubble regions can be obtained. Then, for each candidate bubble region, the shape feature parameters are calculated. For example, the area is obtained by counting the number of pixels in the region; the perimeter is obtained by calculating the chain code length of the region boundary pixels; the roundness calculation formula is 4π*area / perimeter squared (that is, roundness = ;in, is the area, is the perimeter); the convexity calculation formula is convex hull perimeter / region perimeter. Further, the calculated shape feature parameters are input into the pre-trained support vector machine classifier. The support vector machine classifier adopts the RBF kernel function and optimizes the model parameters through cross-validation. The classifier outputs the classification result of each candidate area to determine whether it is a real bubble defect. When the classifier determines that it is a real bubble defect, the position coordinates of the defect, the defect size (the size here usually refers to the physical size occupied by the bubble defect in the image, which can be quantified in a variety of ways, such as area, circumscribed rectangle size, diameter, etc.) and the number of defects are recorded. Finally, the recorded bubble defect information is sent to the monitoring terminal of the production control system in real time through the TCP / IP protocol, and is visualized and alarmed on the monitoring interface.

[0149] In some specific embodiments, the adaptive filtering process uses Gaussian filtering, and the standard deviation parameter of the Gaussian filter is adaptively adjusted according to the variance of the local area of the differential image. The seed point selection strategy of the region growing algorithm is: first, the differential image is binarized, and then the connected areas are extracted, and the center of mass of each connected area is used as the seed point. In addition to area, perimeter, roundness and convexity, shape feature parameters can also include rectangularity, elongation, etc. The classifier uses a deep convolutional neural network. The input of the deep convolutional neural network is the image block of the candidate bubble area, and the output is the probability of whether the area is a real bubble defect. By adjusting the structure and parameters of the deep convolutional neural network, the accuracy of bubble defect recognition can be further improved.

[0150] Please refer to Figure 2 , Figure 2 In some embodiments of the present invention, a door and window glass defect feedback control device is applied to a production control system with a polarized light imaging detection device. The door and window glass defect feedback control device is integrated into a back-end control device in the form of a computer program, and includes:

[0151] A first acquisition module 100 is configured to acquire a first image after adjusting the polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles inside the glass;

[0152] A second acquisition module 200 is configured to acquire a second image after adjusting the polarized light imaging detection device to enhance the reflected light from the glass surface;

[0153] The analysis and calculation module 300 is used to calculate an adaptive weight group by analyzing local characteristics of the second image; the adaptive weight group is used to characterize the local reflection intensity or texture complexity of the second image;

[0154] A difference module 400 is configured to perform weighted processing on the first image according to the adaptive weighting group, and to subtract the weighted first image from the second image to obtain a difference image;

[0155] The analysis and feedback module 500 is used to extract bubble defect information inside the glass by analyzing the differential image; the bubble defect information includes the location of the bubble defect, the size of the bubble defect and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and is used to indicate the existence of quality defects.

[0156] In some embodiments, the analysis and calculation module 300 performs the following when calculating the adaptive weight set by analyzing local characteristics of the second image:

[0157] S31. Dividing the second image into a plurality of non-overlapping image blocks;

[0158] S32. For each image block, calculate the gray level co-occurrence matrix of each image block, and extract the corresponding texture feature parameters based on the gray level co-occurrence matrix, the texture feature parameters including energy, contrast, correlation and entropy;

[0159] S33. For each image block, calculate the gradient magnitude of each image block and count the number of pixels whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter;

[0160] S34. Based on the texture feature parameters and gradient complexity parameters of each image block, a pre-trained prediction model is used to predict the reflection intensity weight value of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters and reflection intensity weight values corresponding to the image blocks under different reflection intensities, and the prediction model uses a cross-validation method for parameter optimization;

[0161] S35. Combine the reflection intensity weight values of all image blocks into an adaptive weight group.

[0162] In some embodiments, the analysis and calculation module 300 performs the following when predicting the reflection intensity weight value of each image block based on the texture feature parameters and gradient complexity parameters of each image block using a pre-trained prediction model:

[0163] S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, texture feature parameters and gradient complexity parameters to obtain a fused feature vector;

[0164] S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block based on the fused feature vector; the prediction model is a deep neural network containing multiple hidden layers, the input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value, and the deep neural network is trained using the Dropout regularization method.

[0165] In some embodiments, the difference module 400 performs the following when performing weighted processing on the first image according to the adaptive weighting group and subtracting the weighted first image from the second image to obtain a difference image:

[0166] S41. For the first image and the second image, an image registration algorithm based on mutual information is used to accurately align the image and calculate the spatial transformation matrix;

[0167] S42. Performing a geometric transformation on the first image according to the spatial transformation matrix to eliminate the misalignment between the first image and the second image;

[0168] S43. According to the adaptive weight group, weighting is performed on the first image after the geometric transformation to obtain a weighted first image;

[0169] S44. Subtract the weighted first image from the second image to obtain an initial difference image;

[0170] S45. Perform multi-scale morphological filtering on the initial differential image to suppress artifacts caused by registration errors, thereby obtaining a differential image.

[0171] In some embodiments, when the analysis and feedback module 500 is used to extract bubble defect information inside the glass by analyzing the differential image, it performs the following operations:

[0172] S51. Performing adaptive filtering on the differential image to reduce noise interference;

[0173] S52. Segmenting the filtered difference image using a region growing-based image segmentation algorithm to obtain multiple candidate bubble regions;

[0174] S53. For each candidate bubble region, shape feature parameters are calculated; shape feature parameters include area, perimeter, roundness and convexity;

[0175] S54. Based on the shape characteristic parameters, a pre-trained classifier is used to determine whether each candidate bubble region is a real bubble defect. If the judgment is yes, the bubble defect information of the real bubble defect is recorded; otherwise, the bubble defect information is not recorded. The size of the bubble defect in the bubble defect information is extracted by analyzing the shape characteristic parameters of the corresponding candidate bubble region;

[0176] S55. Feedback bubble defect information to the production control system in real time.

[0177] Please refer to Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The present invention provides an electronic device 13, including: a processor 1301 and a memory 1302. The processor 1301 and the memory 1302 are interconnected and communicate with each other via a communication bus 1303 and / or other forms of connection mechanisms (not shown). The memory 1302 stores computer-readable instructions executable by the processor 1301. When the electronic device is running, the processor 1301 executes the computer-readable instructions to execute the door and window glass defect feedback control method in any optional implementation of the above embodiment to achieve the following functions: when adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and increase After strengthening the light scattered by bubbles inside the glass, a first image is collected; after adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, a second image is collected; by analyzing the local characteristics of the second image, an adaptive weight group is calculated; the adaptive weight group is used to characterize the local reflection intensity or texture complexity of the second image; according to the adaptive weight group, the first image is weighted, and the weighted first image is subtracted from the second image to obtain a differential image; by analyzing the differential image, the bubble defect information inside the glass is extracted; the bubble defect information includes the location of the bubble defect, the size of the bubble defect and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and used to indicate the existence of quality defects.

[0178] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the door and window glass defect feedback control method in any optional implementation of the above-mentioned embodiment is executed to achieve the following functions: after adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of bubbles inside the glass, a first image is collected; after adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, a second image is collected; by analyzing the local characteristics of the second image, an adaptive weight group is calculated; the adaptive weight group is used to characterize the local reflection intensity or texture complexity of the second image; according to the adaptive weight group, the first image is weighted, and the weighted first image is subtracted from the second image to obtain a differential image; by analyzing the differential image, the bubble defect information inside the glass is extracted; the bubble defect information includes the position of the bubble defect, the size of the bubble defect and the number of the bubble defects; the bubble defect information is fed back to the production control system in real time and used to indicate the existence of quality defects.

[0179] Among them, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0180] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, the indirect coupling or communication connection of the device or unit may be electrical, mechanical or other forms.

[0181] In addition, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0182] Furthermore, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0183] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any actual relationship or order between these entities or operations.

[0184] The foregoing description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A door and window glass defect feedback control method, applied to a production control system with a polarized light imaging detection device, characterized in that: The following steps are involved: S1. After adjusting the polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles inside the glass, a first image is captured; S2. After adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, a second image is collected; S3. Calculating an adaptive weight group by analyzing local characteristics of the second image; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local second image; S4. According to the adaptive weight group, weighting the first image is performed, and the weighted first image is subtracted from the second image to obtain a difference image; S5. By analyzing the differential image, the bubble defect information inside the glass is extracted; the bubble defect information includes the location of the bubble defect, the size of the bubble defect and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and is used to indicate the existence of quality defects.

2. The door and window glass defect feedback control method according to claim 1, characterized in that: The specific steps in step S3 include: S31. Dividing the second image into a plurality of non-overlapping image blocks; S32. For each image block, calculate the gray level co-occurrence matrix of each image block, and extract corresponding texture feature parameters based on the gray level co-occurrence matrix, wherein the texture feature parameters include energy, contrast, correlation and entropy; S33. For each image block, calculate the gradient magnitude of each image block, and count the number of pixels whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter; S34. Predicting the reflection intensity weight value of each image block using a pre-trained prediction model based on the texture feature parameters and gradient complexity parameters of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters, and reflection intensity weight values corresponding to the image blocks at different reflection intensities, and the prediction model uses a cross-validation method for parameter optimization; S35. Combining the reflection intensity weight values of all the image blocks into the adaptive weight group.

3. The door and window glass defect feedback control method according to claim 2, characterized in that: The specific steps in step S34 include: S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, the texture feature parameter and the gradient complexity parameter to obtain a fused feature vector; S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block based on the fused feature vector; the prediction model is a deep neural network comprising multiple hidden layers, the input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value, and the deep neural network is trained using a Dropout regularization method.

4. The door and window glass defect feedback control method according to claim 3, characterized in that: The deep neural network is trained using the Dropout regularization method through the following steps: S3421. For each hidden layer of the deep neural network, randomly discard the output of the neurons in the layer with a probability of 50%, and generate a mask vector, and multiply the mask vector by the output of the neurons in the layer to obtain the neuron output after Dropout; S3422. For each hidden layer, the neuron output of the hidden layer after Dropout is input to the output layer of the deep neural network, and a trained deep neural network is obtained after training; the deep neural network is used to output the reflection intensity weight value through a Sigmoid activation function, and the output reflection intensity weight value is between 0 and 1.

5. The door and window glass defect feedback control method according to claim 4, characterized in that: The specific steps in step S3422 include: S34221. For each hidden layer, calculate the weighted value of the hidden layer based on the connection weight between the hidden layer and the output layer and the neuron output after Dropout of the hidden layer; S34222. Calculate a weighted average value based on the weighted values of all the hidden layers and represent it as a fusion feature; S34223. Input the fused feature representation into the Sigmoid activation function, adjust the parameters of the Sigmoid function through training, and obtain a deep neural network that has completed training.

6. The door and window glass defect feedback control method according to claim 1, characterized in that: The specific steps in step S4 include: S41. For the first image and the second image, accurately align them using an image registration algorithm based on mutual information and calculate a spatial transformation matrix; S42. Performing a geometric transformation on the first image according to the spatial transformation matrix to eliminate the misalignment between the first image and the second image; S43. Performing weighted processing on the first image after geometric transformation according to the adaptive weight group to obtain a weighted first image; S44. Subtracting the weighted first image from the second image to obtain an initial difference image; S45. Perform multi-scale morphological filtering on the initial differential image to suppress artifacts caused by registration errors, thereby obtaining the differential image.

7. The door and window glass defect feedback control method according to claim 1, characterized in that: The specific steps in step S5 include: S51. Adaptively filtering the differential image to reduce noise interference; S52. Segmenting the filtered difference image using a region growing-based image segmentation algorithm to obtain multiple candidate bubble regions; S53. For each candidate bubble region, shape characteristic parameters are calculated; the shape characteristic parameters include area, perimeter, roundness and convexity; S54. Based on the shape characteristic parameters, a pre-trained classifier is used to determine whether each candidate bubble region is a real bubble defect. If the determination is yes, bubble defect information of the real bubble defect is recorded; otherwise, the bubble defect information is not recorded. The bubble defect size in the bubble defect information is extracted by analyzing the shape characteristic parameters of the corresponding candidate bubble region. S55. Feedback the bubble defect information to the production control system in real time.

8. A door and window glass defect feedback control device, applied to a production control system with a polarized light imaging detection device, characterized in that: include: A first acquisition module is configured to acquire a first image after adjusting the polarized light imaging detection device to suppress light reflected from the glass surface and enhance light scattered by bubbles inside the glass; a second acquisition module, configured to acquire a second image after adjusting the polarized light imaging detection device to enhance the reflected light from the glass surface; An analysis and calculation module, configured to calculate an adaptive weight group by analyzing local characteristics of the second image; the adaptive weight group is used to characterize the local reflection intensity or texture complexity of the second image; a difference module, configured to perform weighted processing on the first image according to the adaptive weight group, and subtract the weighted first image from the second image to obtain a difference image; An analysis and feedback module is used to extract bubble defect information inside the glass by analyzing the differential image; the bubble defect information includes the location of the bubble defect, the size of the bubble defect and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and is used to indicate the existence of quality defects.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps of the door and window glass defect feedback control method according to any one of claims 1 to 7 are executed.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the door and window glass defect feedback control method according to any one of claims 1 to 7 are executed.

Images