Door and window glass defect feedback control method and device, electronic equipment and storage medium

Abstract

The invention provides a door and window glass defect feedback control method and device, electronic equipment and a storage medium, and relates to the technical field of door and window glass defect detection. The method comprises the steps of collecting a first image after a polarized light imaging detection device is adjusted to suppress reflected light on the surface of glass and enhance scattered light of bubbles in the glass; after the polarized light imaging detection device is adjusted to enhance reflected light on the glass surface, a second image is collected; calculating an adaptive weight group by analyzing local characteristics of the second image; weighting the first image according to the self-adaptive weight group, and subtracting the weighted first image from the second image to obtain a difference image; and extracting bubble defect information in the glass by analyzing the differential image. According to the method, interference caused by non-uniformity of glass surface reflection is overcome, the adaptability of image differential processing to surface reflection background suppression is improved, and therefore image features of internal bubble defects are extracted more effectively and more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of door and window glass defect detection, and in particular to a door and window glass defect feedback control method, device, electronic equipment and storage medium. Background Art

[0002] Customized high-end doors and windows generally have higher quality requirements, especially for doors and windows with thick glass, the surface quality of the glass and the control of internal defects are crucial. To ensure product quality, production lines generally use optical detection technology based on image sensors to achieve high-precision detection and real-time feedback control of glass defects. Existing technologies attempt to use the principle of optical reflection to detect internal bubble defects. However, thick transparent glass has two reflective surfaces, which complicates the reflected light. The image captured by the image sensor inevitably mixes the reflection signal from the glass surface and the reflection signal from the internal bubble, causing signal aliasing, which seriously affects the accurate identification of internal bubble defects.

[0003] In order to reduce the interference of surface reflection, the existing technology has tried to use the characteristics of polarized light. By controlling the polarization state of the incident light and the angle of the polarizer at the receiving end, the reflected light on the glass surface can be partially suppressed, thereby relatively enhancing the internal bubble scattered light. On this basis, the image difference processing technology is further used, that is, collecting two sets of images with different polarization states and performing subtraction operations, in order to eliminate the common surface reflection background and highlight the internal bubble defects.

[0004] However, in practical applications, the surface quality of some thick glass is uneven, and simple image difference methods still have limitations. The reflection intensity and texture complexity of the glass surface may vary in different areas, resulting in a non-uniform surface reflection background. If a globally unified differential weight is used for processing, it will be difficult to effectively adapt to such local changes, which may lead to incomplete suppression of surface reflections, or excessive suppression in certain areas, which will affect the detection of defects. Especially in demanding scenarios where internal bubble defects need to be identified and characterized with high precision and real-time feedback is required, this problem of insufficient surface reflection suppression will seriously restrict the accuracy and reliability of defect detection, which may lead to the accidental outflow of defective products, and further cause consumer dissatisfaction. Summary of the invention

[0005] The purpose of the present invention is to provide a door and window glass defect feedback control method, device, electronic device and storage medium, which solves the problem that the accuracy and reliability of defect detection are seriously restricted due to insufficient suppression of glass surface reflection, overcomes the interference caused by the uneven reflection of the glass surface, and improves the adaptability of image differential processing to surface reflection background suppression, so as to more effectively and accurately extract the image features of internal bubble defects.

[0006] In a first aspect, the present invention provides a method for defect feedback control of door and window glass, which is applied to a production control system equipped with a polarized light imaging detection device, and includes the following steps: S1. After adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the bubbles inside the glass, collect a first image; S2. After adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, collect a second image; S3. By analyzing the local characteristics of the second image, calculate an adaptive weight group; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local area of the second image; S4. According to the adaptive weight group, perform weighted processing on the first image, and subtract the weighted first image from the second image to obtain a difference image; S5. By analyzing the difference image, extract the bubble defect information inside the glass; the bubble defect information includes the position, size, and quantity of the bubble defects; the bubble defect information is real-time fed back to the production control system and used to prompt the existence of quality defects.

[0007] The method for defect feedback control of door and window glass provided by the present invention introduces an adaptive weight mechanism during the image difference processing, dynamically adjusts the difference weight according to the characteristics of the local area of the image, more finely suppresses the uneven surface reflection background, and thus more effectively highlights the bubble defects inside the glass.

[0008] Further, the specific steps in step S3 include: S31. Divide the second image into multiple non-overlapping image blocks; S32. For each image block, calculate the gray-level co-occurrence matrix of each image block, and extract the corresponding texture feature parameters based on the gray-level co-occurrence matrix. The texture feature parameters include energy, contrast, correlation, and entropy; S33. For each image block, calculate the gradient amplitude of each image block, and count the number of pixel points whose gradient amplitude is greater than a preset threshold as the corresponding gradient complexity parameter; S34. According to the texture feature parameters and gradient complexity parameters of each image block, use a pre-trained prediction model to predict the reflection intensity weight value of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters, and reflection intensity weight values corresponding to the image blocks under different reflection intensities, and the prediction model uses the cross-validation method for parameter optimization; S35. Combine the reflection intensity weight values of all the image blocks into the adaptive weight group.

[0009] The adaptive weight combination can more accurately represent the reflection intensity or texture complexity of the local part of the second image, laying a foundation for the subsequent generation of the differential image and the extraction of bubble defect information, thereby improving the accuracy and reliability of defect detection.

[0010] Furthermore, the specific steps in step S34 include: S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, the texture feature parameter, and the gradient complexity parameter to obtain a fused feature vector; S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block according to the fused feature vector; the prediction model is a deep neural network including multiple hidden layers, the input of the deep neural network is the fused feature vector, the output is the reflection intensity weight value, and the deep neural network is trained using the Dropout regularization method.

[0011] The Dropout method randomly discards a part of neurons, reduces the network's dependence on specific neurons, and enhances the generalization ability and robustness of the network. Thus, even if there are minor defects such as oil stains and scratches on the glass surface, the network can more stably predict the reflection intensity weight value.

[0012] Furthermore, the Dropout regularization method is used to train the deep neural network through the following steps: S3421. For each hidden layer of the deep neural network, randomly discard the output of the neurons in this layer with a probability of 50%, generate a mask vector, and multiply the mask vector by the output of the neurons in this layer to obtain the output of the neurons after Dropout; S3422. For each hidden layer, input the output of the neurons after Dropout in this hidden layer into the output layer of the deep neural network, and obtain the trained deep neural network through training; the deep neural network is used to output the reflection intensity weight value through the Sigmoid activation function, and the output reflection intensity weight value is between 0 and 1.

[0013] Through the application of the Dropout regularization method, the deep neural network can effectively reduce the sensitivity to minor defects such as oil stains and scratches on the glass surface, and improve the accuracy and robustness of predicting the reflection intensity weight value under complex glass surface conditions. Thus, the weight value prediction result becomes more reliable, providing a guarantee for the accuracy of subsequent defect detection.

[0014] Furthermore, the specific steps in step S3422 include: S34221. For each hidden layer, calculate the weighted value of the hidden layer based on the connection weights between the hidden layer and the output layer, and the neuron output after Dropout of the hidden layer; S34222. Calculate the weighted average based on the weighted values of all the hidden layers and use it as the fused feature representation; S34223. Input the fused feature representation into the Sigmoid activation function, and adjust the parameters of the Sigmoid function through training to obtain a trained deep neural network.

[0015] Further, the specific steps in step S4 include: S41. For the first image and the second image, use an image registration algorithm based on mutual information for precise alignment and calculate the spatial transformation matrix; S42. According to the spatial transformation matrix, perform a geometric transformation on the first image to eliminate the misalignment between the first image and the second image; S43. According to the adaptive weight set, perform a weighting process on the geometrically transformed first image to obtain a weighted first image; S44. Subtract the weighted first image from the second image to obtain an initial difference image; S45. Perform multi-scale morphological filtering on the initial difference image to suppress the artifacts caused by registration errors and obtain the difference image.

[0016] Further, the specific steps in step S5 include: S51. Perform an adaptive filtering process on the difference image to reduce noise interference; S52. Use an image segmentation algorithm based on region growing to segment the filtered difference image to obtain multiple candidate bubble regions; S53. For each candidate bubble region, calculate the shape feature parameters; the shape feature parameters include area, perimeter, circularity, and convexity; S54. According to the shape feature parameters, use a pre-trained classifier to determine whether each candidate bubble region is a real bubble defect, and when the determination is yes, record the bubble defect information of the real bubble defect, otherwise do not record the bubble defect information; wherein, the size of the bubble defect in the bubble defect information is extracted by analyzing the shape feature parameters of the corresponding candidate bubble region; S55. Real-time feedback the bubble defect information to the production control system.

[0017] In a second aspect, the present invention provides a door and window glass defect feedback control device, which is applied to a production control system with a polarized light imaging detection device, and includes: The first acquisition module is configured to acquire a first image after adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the internal bubbles in the glass. The second acquisition module is configured to acquire a second image after adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface. The analysis and calculation module is configured to calculate an adaptive weight group by analyzing the local characteristics of the second image; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local part of the second image. The difference module is configured to perform weighted processing on the first image according to the adaptive weight group, and subtract the weighted first image from the second image to obtain a difference image. The analysis and feedback module is configured to extract the bubble defect information inside the glass by analyzing the difference image; the bubble defect information includes the position, size and quantity of the bubble defects; the bubble defect information is real-time fed back to the production control system and used to prompt the existence of quality defects.

[0018] The device for feedback control of door and window glass defects provided by the present invention can better adapt to the local non-uniformity of the glass surface reflection, improve the surface reflection suppression effect, and further enhance the accuracy and reliability of the detection of internal bubble defects.

[0019] In a third aspect, the present invention provides an electronic device, including a processor and a memory, where the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect as described above are run.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method provided in the first aspect as described above are run.

[0021] As can be seen from the above, the method for feedback control of door and window glass defects provided by the present invention can dynamically adjust the difference weight according to the local characteristics of the image through an adaptive weight mechanism, more finely suppress the surface reflection background, especially for thick glass with uneven surface reflection, making the image features of internal bubble defects clearer and more prominent, improving the signal-to-noise ratio and accuracy of defect detection, thus better strictly controlling the product quality, effectively avoiding the accidental outflow of defective products, and being beneficial to enhancing the consumption experience of consumers.

[0022] Other features and advantages of the present invention will be described in the subsequent description, and part of them will become obvious from the description, or be understood by implementing the embodiments of the present invention. The objectives and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the written description and the drawings. Brief Description of the Drawings

[0023] Figure 1 It is a flowchart of a method for feedback control of window and door glass defects provided by an embodiment of the present invention.

[0024] Figure 2 It is a schematic structural diagram of a device for feedback control of window and door glass defects provided by an embodiment of the present invention.

[0025] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0026] Label Description: 100, the first acquisition module; 200, the second acquisition module; 300, the analysis and calculation module; 400, the difference module; 500, the analysis and feedback module; 13, the electronic device; 1301, the processor; 1302, the memory; 1303, the communication bus. Detailed Embodiments

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the present invention to be protected, but only represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0028] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used for distinguishing descriptions, and cannot be understood as indicating or implying relative importance.

[0029] Refer to the attached Figure 1 , the present invention provides a method for feedback control of window and door glass defects, which is applied to a production control system with a polarized light imaging detection device, and includes the following steps: S1. After adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the internal bubbles in the glass, acquire the first image; S2. After adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, acquire the second image; S3. By analyzing the local characteristics of the second image, calculate an adaptive weight set; the adaptive weight set is used to characterize the reflection intensity or texture complexity of the local part of the second image; S4. According to the adaptive weight set, perform weighted processing on the first image, and subtract the weighted first image from the second image to obtain a difference image; S5. By analyzing the difference image, extract the bubble defect information inside the glass; the bubble defect information includes the position of the bubble defect, the size of the bubble defect, and the number of bubble defects; the bubble defect information is fed back to the production control system in real time and used to indicate the existence of quality defects.

[0030] In step S1, the adjustment of the polarized light imaging detection device can be achieved by controlling the polarization state of the incident light and the angle of the receiving-end polarizer, with the aim of reducing the reflected light on the glass surface while enhancing the scattered light of the internal bubbles in the glass, so as to collect the first image mainly containing the internal bubble defect information.

[0031] In step S2, the polarized light imaging detection device is adjusted to enhance the reflected light on the glass surface, with the aim of collecting the second image mainly containing the reflected information on the glass surface.

[0032] In step S3, the calculation of the adaptive weight set is based on dividing the second image into image blocks and analyzing the local characteristics of each image block. The local characteristics can include the texture feature parameters and gradient complexity parameters of the image block, and the value of the adaptive weight set is associated with the reflection intensity or texture complexity of the image block.

[0033] In step S4, the weighted processing of the first image is carried out according to the adaptive weight set calculated in step S3, with the aim of applying different weights to different regions of the first image, so as to more accurately eliminate the influence of surface reflection in the subsequent difference operation and obtain a difference image that can prominently display the internal bubble defects.

[0034] Specifically, assume that the size of the first image is 1280x1280 pixels, and the adaptive weight set consists of a 40x40 weight matrix, and each weight value corresponds to an image block of 32x32 pixels.

[0035] First, the first image is divided into 40x40 image blocks of 32x32 pixels each. The division process can be carried out in the order from left to right and from top to bottom. The first image block is located in the upper left corner of the first image, with coordinates (0,0) and a size of 32x32 pixels. The second image block is located on the right side of the first image block, with coordinates (32,0) and the same size of 32x32 pixels. And so on, until 40 image blocks in the first row are divided. Then, starting from the second row, continue to divide the image blocks until the entire first image is divided.

[0036] Secondly, each weight value in the adaptive weight group is respectively applied to the corresponding 32x32 pixel image block. For example, the weight value at (0,0) in the adaptive weight group is applied to the 32x32 pixel image block at coordinates (0,0) in the first image. The weight value at (1,0) in the adaptive weight group is applied to the 32x32 pixel image block at coordinates (32,0) in the first image. And so on, until all the weight values in the adaptive weight group are applied to the corresponding image blocks.

[0037] The weighting process can be achieved by multiplying the gray value of each pixel in the image block by the corresponding weight value. For example, if the gray value of the pixel at coordinates (0,0) in the first image is 100 and the corresponding weight value is 0.5, then the gray value of this pixel after weighting becomes 100 * 0.5 = 50. Such weighting processing is performed on all the pixels in each image block to complete the weighting of the first image.

[0038] In this way, each local area of the first image is adjusted according to the adaptive weight group, so as to better eliminate the influence of surface reflection and highlight the internal bubble defects.

[0039] In step S5, the analysis and processing of the difference image includes operations such as filtering, segmentation, and calculation of shape feature parameters on the difference image. The purpose is to extract the bubble defect information inside the glass from the difference image and feedback the bubble defect information to the production control system to achieve real-time monitoring and feedback control of the defects.

[0040] Specifically, this method uses a polarized light imaging detection device. First, two images are collected by adjusting the device: the first image is collected in a state where the surface reflected light is suppressed and the internal bubble scattered light is enhanced, and the second image is collected in a state where the surface reflected light is enhanced. Then, the local characteristics of the second image are analyzed to calculate the adaptive weight group, and this function characterizes the reflection intensity or texture complexity of the local area of the second image. The adaptive weight group is then used to perform a weighting process on the first image. The weighted first image is then subjected to a difference operation with the second image, aiming to eliminate the common surface reflection background, so as to highlight the internal bubble defects and obtain a difference image. Finally, the difference image is analyzed to extract information such as the position, size, and quantity of the bubble defects, and this information is fed back to the production control system in real time for quality control. Due to the introduction of the adaptive weight group, this method can better adapt to the local non-uniformity of the glass surface reflection, improve the surface reflection suppression effect, and further enhance the accuracy and reliability of the internal bubble defect detection.

[0041] In some specific embodiments, for the on-line quality inspection of door and window glass, the production control system is configured with a polarized light imaging detection device. During the detection process, first, the polarization angle of the polarized light imaging detection device is adjusted so that the device can suppress the reflected light on the glass surface and enhance the scattered light of the bubbles inside the glass, and the first image is acquired. Subsequently, the polarization angle of the polarized light imaging detection device is adjusted so that the device can enhance the reflected light on the glass surface, and the second image is acquired. After the image acquisition is completed, the system analyzes the second image, divides the second image into multiple non-overlapping image blocks with a size of 32x32 pixels. For each image block, the gray-level co-occurrence matrix is calculated, and four texture feature parameters of energy, contrast, correlation, and entropy are extracted. At the same time, the gradient magnitude is calculated, and the gradient complexity parameter is statistically analyzed. Using a pre-trained prediction model, based on the texture feature parameters and the gradient complexity parameter, the reflection intensity weight value of each image block is predicted, and the weight values of all image blocks are combined into an adaptive weight group. Then, according to the adaptive weight group, the first image is weighted, and the second image is subtracted from the weighted first image to obtain an initial difference image. The initial difference image is subjected to multi-scale morphological filtering to eliminate the artifacts caused by registration errors, and the final difference image is obtained. The difference image is analyzed, adaptive filtering is used to reduce noise, and then the region growing algorithm is used to segment the candidate bubble regions, and the area, perimeter, roundness, and convexity shape feature parameters of the candidate regions are calculated. Finally, through a pre-trained classifier, according to the shape feature parameters, it is judged whether it is a real bubble defect. If so, the position, size, and quantity of the defect are recorded, and the bubble defect information is fed back to the production control system in real time to prompt the quality defect. Thus, the automatic detection and feedback control of the bubble defects inside the door and window glass are realized.

[0042] In certain embodiments, the specific steps in step S3 include: S31. Divide the second image into multiple non-overlapping image blocks; S32. For each image block, calculate the gray-level co-occurrence matrix of each image block, and extract the corresponding texture feature parameters based on the gray-level co-occurrence matrix. The texture feature parameters include energy, contrast, correlation, and entropy; S33. For each image block, calculate the gradient magnitude of each image block, and count the number of pixel points whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter; S34. According to the texture feature parameters and the gradient complexity parameter of each image block, use a pre-trained prediction model to predict the reflection intensity weight value of each image block; the training samples of the prediction model include the texture feature parameters, the gradient complexity parameter, and the reflection intensity weight value corresponding to the image block under different reflection intensities, and the prediction model uses the cross-validation method for parameter optimization; S35. The reflection intensity weight values ​​of all image blocks are combined into an adaptive weight group (it should be noted that the weight value of each block is mapped back to its original spatial position, a global weight matrix is ​​constructed, and finally a block constant value matrix is ​​generated as an adaptive weight group. The adaptive weight group presents a checkerboard distribution in space, and the weight of each block reflects its local characteristics; it should be noted that the adaptive weight group can actually be regarded as a function on the image area. The image is divided into multiple image blocks, each image block corresponds to a function value, and the function value is the reflection intensity weight value of the image block. The adaptive weight group can be based on the local characteristics of the image block. Characteristics, adaptively adjust the weight value, the weight value is not fixed, but changes with the change of local characteristics of the image, so it meets the definition of a function, and the adaptive weight group can be expressed as a mapping relationship from the image block position to the reflection intensity weight value. The position coordinates of each image block correspond to a reflection intensity weight value. This mapping relationship also meets the definition of a function. Therefore, although the adaptive weight group is composed of multiple numerical values, it has the basic characteristics of a function and can adaptively adjust the weight value according to the local characteristics of the image, and can be expressed as a mapping relationship from the image block position to the reflection intensity weight value, so it can also be regarded as a function).

[0043] In step S31, the second image is divided into multiple regions, providing a basis for subsequent analysis of local characteristics of the image.

[0044] In step S32, for each image block, a gray level co-occurrence matrix is ​​calculated to extract texture feature parameters such as energy, contrast, correlation and entropy. The texture feature parameters can quantify the local texture information of the image.

[0045] In step S33, the gradient amplitude and the gradient complexity parameter are calculated. The gradient complexity parameter is obtained by counting the number of pixel points whose gradient amplitude exceeds a threshold value, and is used to characterize the drastic degree of change in the local reflection intensity of the image.

[0046] In step S34, a prediction model is used, and the training samples of the prediction model include texture feature parameters, gradient complexity parameters and reflection intensity weight values. The prediction model predicts the reflection intensity weight value by learning the mapping relationship between texture features, gradient complexity and reflection intensity weight values. The cross-validation method is used to optimize the prediction model parameters and improve the model generalization ability.

[0047] In step S35, the reflection intensity weight values ​​of all image blocks are combined to form an adaptive weight group, which is used in subsequent image processing steps.

[0048] Specifically, this embodiment aims to solve the problem that the adaptive weight group cannot accurately represent the actual surface reflection situation due to uneven illumination and uneven glass thickness. First, the second image is divided into multiple non-overlapping image blocks, realizing the analysis of the local characteristics of the image. Then, for each image block, texture feature parameters and gradient complexity parameters are extracted. The texture feature parameters and gradient complexity parameters can reflect the reflection intensity and texture complexity of the local area of the image. Subsequently, a pre-trained prediction model is used to predict the reflection intensity weight value of each image block according to the texture feature parameters and gradient complexity parameters. The prediction model can learn the mapping relationship between texture features, gradient complexity, and reflection intensity weight values, realizing the prediction of adaptive weight values. Finally, the reflection intensity weight values of all image blocks are combined into an adaptive weight group. The adaptive weight group can more accurately represent the reflection intensity or texture complexity of the local area of the second image, laying a foundation for the generation of subsequent differential images and the extraction of bubble defect information, thereby improving the accuracy and reliability of defect detection.

[0049] In some specific embodiments, the size of the image block is set to 32x32 pixels. In the calculation of the gray-level co-occurrence matrix, the distance between pixel pairs is set to 1, and the directions include 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The gradient magnitude is calculated by the Sobel operator, and the preset threshold is set to 1.5 times the average value of the gradient magnitude. The prediction model is, for example, a support vector regression model using a radial basis kernel function, and the optimal model parameters are determined by the 5-fold cross-validation method. The training samples are obtained by collecting image blocks on the glass surface with different reflection intensities and manually annotating the reflection intensity weight values.

[0050] In certain embodiments, the specific steps in step S34 include: S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, texture feature parameters, and gradient complexity parameters to obtain a fused feature vector; S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block according to the fused feature vector; the prediction model is a deep neural network with multiple hidden layers (it should be noted that traditional support vector regression models usually use kernel functions such as radial basis kernel functions to map input data to a high-dimensional feature space and then perform linear regression in this space. However, in practical applications, traditional kernel functions may not be able to effectively extract the features of input data. Therefore, in this embodiment, a deep neural network is selected to predict the reflection intensity weight value). The input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value. Moreover, the deep neural network is trained using the Dropout regularization method to reduce the interference of minor defects such as oil stains and scratches on the glass surface to the prediction of weight values.

[0051] For each image patch, the local binary pattern (LBP) features are calculated. The texture feature parameters and gradient complexity parameters are fused with the LBP features to obtain a fused feature vector. The fused feature vector is input into a pre-trained prediction model to predict the reflection intensity weight value. The prediction model is a deep neural network with multiple hidden layers. The deep neural network takes the fused feature vector as the input and the reflection intensity weight value as the output. The Dropout regularization method is adopted during the training of the deep neural network to reduce the interference of minor defects such as oil stains and scratches on the glass surface to the prediction of the weight value.

[0052] Specifically, as an effective way to describe the local texture information of an image, the local binary pattern (LBP) features are introduced to more comprehensively characterize the local characteristics of the image patch, and at the same time, the texture feature parameters and gradient complexity parameters are used. By fusing these three features, a multi-dimensional and more accurate description of the local characteristics of the image patch can be achieved. The deep neural network utilizes its powerful non-linear fitting ability to learn the complex mapping relationship between the fused feature vector and the reflection intensity weight value, thereby improving the accuracy of the weight value prediction. To reduce the interference of minor defects such as oil stains and scratches on the glass surface to the prediction of the weight value, the Dropout regularization method is introduced during the training of the deep neural network. During the training process, the Dropout method randomly discards a part of the neurons, reducing the network's dependence on specific neurons and enhancing the generalization ability and robustness of the network. Thus, even if there are minor defects such as oil stains and scratches on the glass surface, the network can more stably predict the reflection intensity weight value.

[0053] In some specific embodiments, for each image patch, first, its local binary pattern (LBP) features are calculated. The specific process can be as follows: Taking each pixel in the image patch as the center and taking a 3x3 neighborhood as an example, the gray value of the neighborhood pixels is compared with the gray value of the center pixel. If the gray value of the neighborhood pixel is greater than the gray value of the center pixel, it is marked as 1; otherwise, it is marked as 0. Thus, an 8-bit binary number is obtained, which is the LBP value of the center pixel. The LBP values of all pixels in the image patch are statistically analyzed to form an LBP histogram, and this histogram can be used as the LBP features of the image patch. The LBP features, texture feature parameters, and gradient complexity parameters are concatenated into a fused feature vector. A deep neural network with three hidden layers is constructed, and each layer contains 128 neurons. The Dropout regularization method is applied to each hidden layer, and the Dropout ratio is set to 50%. The deep neural network is trained using a large amount of sample data, and the sample data includes the fused feature vectors corresponding to the image patches under different reflection intensities and the reflection intensity weight values. The trained deep neural network can be used to predict the reflection intensity weight value of a new image patch.

[0054] In some embodiments, the Dropout regularization method is used to train a deep neural network through the following steps: S3421. For each hidden layer of the deep neural network, randomly discard the outputs of the neurons in this layer with a probability of 50%, generate a mask vector, and multiply the mask vector by the outputs of the neurons in this layer to obtain the neuron outputs after Dropout; S3422. For each hidden layer, input the neuron outputs after Dropout of this hidden layer into the output layer of the deep neural network, and train to obtain a trained deep neural network; the deep neural network is used to output the reflection intensity weight value through the Sigmoid activation function, and the output reflection intensity weight value is between 0 and 1.

[0055] When using the deep neural network to predict the reflection intensity weight value, overfitting is likely to occur during the training of the deep neural network, resulting in the technical problem of being sensitive to small defects such as oil stains and scratches on the glass surface, which affects the prediction accuracy of the weight value. For this, an improved solution is proposed in this embodiment.

[0056] In step S3421, the Dropout regularization method is applied to each hidden layer of the deep neural network. The operation method is to randomly discard the outputs of the neurons in this layer with a probability of 50%, and multiply the generated mask vector by the neuron outputs to achieve the random discard of the neuron outputs and obtain the neuron outputs after Dropout. The Dropout technique reduces the network's dependence on specific neurons by randomly ignoring a part of the neurons during the training process, thereby enhancing the network's generalization ability and reducing the risk of overfitting.

[0057] In step S3422, the neuron outputs of all hidden layers after Dropout processing are input into the output layer of the deep neural network for training to obtain the final deep neural network. The deep neural network is configured to output the reflection intensity weight value through the Sigmoid activation function and limit the weight value between 0 and 1. The Sigmoid function ensures that the output weight value is within a reasonable range, facilitating subsequent weighted processing.

[0058] Specifically, in this embodiment, the Dropout regularization method is introduced during the training process of the deep neural network. In each hidden layer of the deep neural network, the output of neurons is randomly discarded with a probability of 50%. This random discarding reduces the sensitivity of the network to noise and irrelevant details in the training data by forcing the network to learn more robust and generalized feature representations. The output of the hidden layer neurons after Dropout processing is then input into the output layer, and the final deep neural network model is obtained through training. This deep neural network model uses the Sigmoid activation function to output the reflection intensity weight value between 0 and 1. Through the application of the Dropout regularization method, the deep neural network can effectively reduce its sensitivity to minor defects such as oil stains and scratches on the glass surface, and improve the accuracy and robustness of predicting the reflection intensity weight value under complex glass surface conditions. Thus, the weight value prediction result becomes more reliable, providing a guarantee for the accuracy of subsequent defect detection.

[0059] In some specific embodiments, the deep neural network can be a multi-layer perceptron with three hidden layers. During the training phase, the Dropout regularization method is applied to each hidden layer, randomly discarding 50% of the neuron outputs. For example, for the first hidden layer, a mask vector with the same dimension as the neuron output of this layer is generated, and each element in the vector has a 50% probability of being 0 and the rest being 1. The mask vector is multiplied element-wise with the neuron output to achieve random discarding of the neuron output. The same operation is applied to the second and third hidden layers. The outputs of all hidden layer neurons after Dropout processing are fed into the output layer. The output layer uses the Sigmoid activation function to map the fused features to the reflection intensity weight value between 0 and 1. Through the backpropagation algorithm and the gradient descent optimization method, the parameters of the deep neural network are continuously adjusted and optimized to complete the training of the deep neural network. The trained deep neural network can more accurately predict the reflection intensity weight value of each image patch, reducing the interference of minor defects on the glass surface to the weight value prediction.

[0060] It should be noted that the input of each hidden layer of the present invention is a fused feature vector, and all hidden layers are respectively connected to the output layer, which means that the output of each hidden layer will be used to calculate the final fused feature representation and input into the Sigmoid activation function to obtain the reflection intensity weight value. Therefore, each hidden layer independently extracts features from the fused feature vector and directly contributes the extracted feature information to the final output result. This structure allows the network to learn features from multiple different abstraction levels and fuse these features for more accurate prediction of the reflection intensity weight value.

[0061] In certain embodiments, the specific steps in step S3422 include: S34221. For each hidden layer, calculate the weighted value of the hidden layer based on the connection weights between the hidden layer and the output layer and the neuron outputs after Dropout of the hidden layer; S34222. Calculate the weighted average based on the weighted values of all hidden layers and use it as the fused feature representation; S34223. Input the fused feature representation into the Sigmoid activation function, and adjust the parameters of the Sigmoid function through training to obtain the trained deep neural network.

[0062] In step S34221, for each hidden layer, calculate the weighted value, which takes into account the influence degree of each hidden layer on the final output. Specifically, the connection weights reflect the sensitivity of the output layer to the neuron outputs of each hidden layer. By applying the connection weights to the neuron outputs after Dropout, the contribution of each hidden layer in the overall network can be quantified.

[0063] In step S34222, integrate the weighted values of all hidden layers, calculate the weighted average as the fused feature representation. Thus, the information from different hidden layers is effectively fused to form a comprehensive feature vector, which can more comprehensively represent the multi-level features learned by the deep neural network.

[0064] In step S34223, input the fused feature representation into the Sigmoid activation function. The activation function maps the feature values to the interval from 0 to 1, realizing the preliminary range limitation of the reflection intensity weight value. Further, by adjusting the parameters of the Sigmoid function, for example, the Sigmoid function is expressed as , where is the learnable parameter. In this application, by adjusting this learnable parameter, the output curve of the Sigmoid function is translated, the activation threshold is adjusted, and the expression ability of the model is enhanced. Specifically, , where, is the preset weight, represents the transpose, is the input data, is the bias term. In actual application, changing the bias term can adjust the decision boundary of the Sigmoid function. For example, when the bias term is 0, when (i.e., without bias), the activation threshold is the origin of the input control, and the decision boundary is ; when the bias term is not 0, when (introducing bias), the activation threshold , and the decision boundary becomes Setting a bias term allows the decision boundary to translate freely to adapt to more complex data distributions. By adjusting the parameters of the Sigmoid function, the output range of the reflection intensity weight value is further controlled to achieve a more accurate output. This ensures that the weight value is neither too small to cause the weighted processing to fail nor too large to overly suppress the information of the first image.

[0065] Specifically, the purpose of this embodiment is to solve the problem that the output range of the reflection intensity weight value is uncontrollable. During the training process of the deep neural network, directly inputting the neuron outputs after dropout of all hidden layers into the output layer may result in an overly wide or offset output range of the reflection intensity weight value, affecting the accuracy of subsequent defect detection. To solve this problem, this embodiment finely controls the output range of the weight value through three steps. First, calculate the weighted value of each hidden layer to evaluate the importance of each layer. Then, fuse the information of all hidden layers through weighted averaging to obtain a comprehensive feature representation. Finally, use the Sigmoid activation function and bias term adjustment to more precisely control the output of the reflection intensity weight value. Thus, it is ensured that the adaptive weight group can effectively play its role in subsequent steps, improving the accuracy and reliability of defect detection.

[0066] In some specific embodiments, the deep neural network includes two hidden layers (denoted as H1 and H2), and each hidden layer includes 3 neurons, expressed as: Neuron outputs of H1: [0.9, 0.6, 0.7]; Neuron outputs of H2: [0.3, 0.8, 0.6]; After applying Dropout, it is expressed as: Assume that the 1st and 3rd neurons in H1 are not discarded, then the output becomes: [0.9, 0, 0.7]; Assume that the 2nd and 3rd neurons in H2 are not discarded, then the output becomes: [0, 0.8, 0.6]; The connection weights between the output layer and each hidden layer are expressed as: Weights from H1 to the output layer = [0.8, 0.2, 0.5]; Weights from H2 to the output layer = [0.4, 0.6, 0.3]; The weighted values calculated for each hidden layer are expressed as: 0.8 * 0.9 + 0.2 * 0 + 0.5 * 0.7 = 1.07; 0.4 * 0 + 0.6 * 0.8 + 0.3 * 0.6 = 0.66; The calculated weighted average (i.e., the fused feature representation) is equal to (1.07 + 0.66) / 2 = 0.865.

[0067] In step S34223, the bias term of the Sigmoid activation function is set to -0.5, such that when the fused feature representation is 0, the output of the Sigmoid function is close to 0.38, and when the fused feature is positive infinity, the output is close to 1. Through such bias term adjustment and the characteristics of the Sigmoid function itself, the output range of the reflection intensity weight value is ultimately further controllable, which is conducive to outputting a more accurate reflection intensity weight value. For example, the actual output range may be [0.15, 0.85], ensuring that the weight value fluctuates within a reasonable range and avoiding the influence of extreme values on the defect detection effect.

[0068] In some embodiments, the specific steps in step S4 include: S41. For the first image and the second image, an image registration algorithm based on mutual information is used for precise alignment, and a spatial transformation matrix is calculated; S42. According to the spatial transformation matrix, a geometric transformation is performed on the first image to eliminate the misalignment between the first image and the second image; S43. According to the adaptive weight group, the geometrically transformed first image is weighted to obtain the weighted first image; S44. The weighted first image is subtracted from the second image to obtain an initial difference image; S45. The initial difference image is subjected to multi-scale morphological filtering to suppress artifacts caused by registration errors, obtaining a difference image.

[0069] In step S41, the mutual information registration algorithm is used to achieve the precise alignment of the first image and the second image. The mutual information registration algorithm is an image registration technique that can effectively handle the gray difference and geometric deformation between images. In specific implementation, normalized mutual information can be used as the similarity metric standard, and through an optimization algorithm such as the gradient descent method, the spatial transformation parameters are iteratively calculated, and finally the spatial transformation matrix is obtained. The spatial transformation matrix can represent geometric transformations such as translation, rotation, or scaling.

[0070] In step S42, according to the spatial transformation matrix calculated in step S41, a geometric transformation is performed on the first image. The geometric transformation process aims to eliminate the misalignment between the first image and the second image caused by the shooting angle or position deviation. As an implementation method, an affine transformation can be used to process the first image so that the transformed first image is precisely aligned with the second image in terms of spatial position.

[0071] In step S45, multi-scale morphological filtering is used to suppress artifacts caused by possible registration errors in the initial difference image. By performing morphological operations such as erosion and dilation using structuring elements of different sizes and shapes, multi-scale morphological filtering can effectively remove noise and small artifacts in the image while retaining the main structural information in the image. For example, circular or square structuring elements can be used, and multiple different sizes can be selected. Opening and closing operations are performed separately, and the results are fused to achieve multi-scale filtering.

[0072] Specifically, for the application scenario of glass defect detection, after the first image and the second image are acquired, the image registration operation in step S41 is first performed. For example, an open-source image processing library such as OpenCV can be used to implement image registration based on mutual information. The registration process is as follows: Input the first image and the second image, set mutual information as the similarity metric, select the affine transformation model, set parameters such as the number of iterations and the convergence threshold, run the registration algorithm, and obtain the spatial transformation matrix. Then, in step S42, the obtained spatial transformation matrix is input into the image geometric transformation function to perform coordinate transformation on each pixel in the first image, thereby achieving geometric correction of the first image. Subsequently, in step S43, according to the foregoing scheme, the aligned first image is weighted according to the adaptive weight group. Step S44 performs image subtraction, subtracting the weighted first image from the second image to obtain the initial difference image. Finally, in step S45, multi-scale morphological filtering is performed on the initial difference image. For example, a multi-scale morphological filter including circular structuring elements of 3 scales can be designed, and the scale sizes are set to 3x3, 5x5, and 7x7 respectively. Opening and closing operations are performed on the initial difference image using these structuring elements respectively, and all the resulting images are weighted and averaged to obtain the final difference image. Through the above steps, a difference image with effectively suppressed artifacts can be obtained, providing high-quality image data for the accurate extraction of subsequent bubble defects.

[0073] In some specific embodiments, for the feedback control method of door and window glass defects, in order to solve the problem of artifacts in the differential result caused by slight image misalignment, step S4 is designed to include image registration and multi-scale morphological filtering. In the image registration link, the mutual information algorithm is selected as the registration algorithm because the mutual information is insensitive to the grayscale change of the image and can better adapt to the polarized illumination difference that may exist between the two groups of images. The spatial transformation matrix adopts the affine transformation model, which can better correct the misalignment of the image such as translation, rotation and scaling. In the multi-scale morphological filtering link, the shape of the structural element of the filter is selected as a circle, and the size is set to three scales of 3x3, 5x5 and 7x7, which can effectively remove the artifacts introduced by small noise and registration residual errors, while retaining the edge information of the bubble defect as much as possible. Through the synergistic effect of image registration and multi-scale morphological filtering, the quality of the differential image is improved, the interference of artifacts is reduced, and the subsequent extraction of bubble defects is more accurate and reliable.

[0074] In some embodiments, the specific steps in step S5 include: S51. Adaptively filter the differential image to reduce noise interference; S52. Segmenting the filtered difference image using a region growing-based image segmentation algorithm to obtain a plurality of candidate bubble regions; S53. For each candidate bubble region, calculating shape feature parameters; shape feature parameters include area, perimeter, roundness and convexity; S54. According to the shape feature parameters, a pre-trained classifier is used to determine whether each candidate bubble region is a real bubble defect, and if it is determined to be so, the bubble defect information of the real bubble defect is recorded, otherwise the bubble defect information is not recorded; wherein the size of the bubble defect in the bubble defect information is extracted by analyzing the shape feature parameters of the corresponding candidate bubble region (area, perimeter, roundness and convexity, etc., can effectively describe the geometric shape characteristics of the candidate region. For example, the area reflects the size of the candidate region, the perimeter reflects the contour length of the candidate region, the roundness reflects the degree to which the shape of the candidate region is close to a circle, and the convexity reflects the degree of convexity and concavity of the shape of the candidate region); S55. Feedback the bubble defect information to the production control system in real time.

[0075] Adaptive filtering was implemented to address potential noise interference in the differential image. Adaptive filtering can adjust the filtering parameters according to the noise level and texture features of local regions in the image, effectively removing noise while better preserving the detailed information of the image. For example, methods such as Wiener filtering or adaptive median filtering can be employed. The filtered differential image is used for subsequent bubble defect detection, reducing the interference of noise on defect recognition. Further, to screen out potential bubble defect regions from the filtered differential image, an image segmentation algorithm based on region growing is adopted. The region growing algorithm starts from one or more seed pixels in the image and gradually merges the surrounding pixels with similar features into the same region. In this solution, the seed pixels can be automatically selected based on the high gray value regions in the differential image, and the similarity criterion can be set such that the difference between the pixel gray value and the seed pixel gray value is less than a preset threshold. Thus, multiple connected candidate bubble regions can be obtained, initially realizing the localization of bubble defects. To distinguish real bubble defects from non-defect regions, multiple shape feature parameters are calculated for each candidate bubble region. The shape feature parameters include the area used to describe the size of the candidate region; the perimeter used to describe the contour length of the candidate region; the circularity used to describe the degree to which the shape of the candidate region approaches a circle; and the convexity used to describe the degree of convexity and concavity of the shape of the candidate region. These shape feature parameters can effectively characterize the geometric shape characteristics of the candidate region. Furthermore, to automatically identify real bubble defects, a pre-trained classifier is adopted. The classifier takes the shape feature parameters of the candidate bubble region as input and realizes the automatic classification of the candidate region by learning the features of a large number of bubble defect samples and non-defect samples. The classifier can select models such as support vector machines, random forests, or deep neural networks. The classifier outputs a judgment result indicating whether the candidate region is a real bubble defect. When the judgment result is yes, the bubble defect information of the real bubble defect is recorded, and the bubble defect information includes the location, size, and quantity of the bubble defect, etc. Otherwise, the bubble defect information is not recorded. Finally, the bubble defect information is fed back to the production control system in real time so that the production control system can timely grasp the quality status of the glass products and make corresponding production adjustments.

[0076] Specifically, for the denoised differential image, the seed points for region growing are first determined. The seed points can be selected as the pixel points in the differential image with gray values higher than the preset threshold. The 8-neighborhood connection method is adopted in the region growing process, and the similarity criterion is set such that the difference between the current pixel and the seed pixel gray value is less than 30. Thus, a series of candidate bubble regions can be obtained. Then, for each candidate bubble region, the shape feature parameters are calculated. For example, the area is obtained by counting the number of pixels in the region; the perimeter is obtained by calculating the chain code length of the boundary pixels of the region; the circularity calculation formula is 4π * area / perimeter squared (i.e., circularity = ; where is the area, (taking the perimeter as the perimeter); the convexity calculation formula is the convex hull perimeter / region perimeter. Further, the calculated shape feature parameters are input into a pre-trained support vector machine classifier. The support vector machine classifier uses an RBF kernel function, and the model parameters are optimized through cross-validation. The classifier outputs the classification result of each candidate region to determine whether it is a real bubble defect. When the classifier determines it to be a real bubble defect, record the position coordinates, defect size (the size mentioned here usually refers to the physical size occupied by the bubble defect in the image, which can be quantified in various ways, such as area, circumscribed rectangle size, diameter, etc.) and the number of defects of the defect. Finally, the recorded bubble defect information is sent to the monitoring terminal of the production control system in real time through the TCP / IP protocol, and visual display and alarm prompts are carried out on the monitoring interface.

[0077] In some specific embodiments, the adaptive filtering process uses Gaussian filtering, and the standard deviation parameter of the Gaussian filter is adaptively adjusted according to the variance of the local region of the difference image. The seed point selection strategy of the region growing algorithm is as follows: first, the difference image is binarized, then the connected regions are extracted, and the centroid of each connected region is used as the seed point. In addition to area, perimeter, roundness, and convexity, the shape feature parameters can also include rectangularity, elongation, etc. The classifier uses a deep convolutional neural network. The input of the deep convolutional neural network is the image patch of the candidate bubble region, and the output is the probability that the region is a real bubble defect. By adjusting the structure and parameters of the deep convolutional neural network, the accuracy of bubble defect recognition can be further improved.

[0078] Please refer to Figure 2 , Figure 2 which is a device for feedback control of window and door glass defects in some embodiments of the present invention, and is applied to a production control system with a polarized light imaging detection device. The device for feedback control of window and door glass defects is integrated in the backend control device in the form of a computer program, and includes: The first acquisition module 100 is used to acquire the first image after adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the internal bubbles in the glass; The second acquisition module 200 is used to acquire the second image after adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface; The analysis and calculation module 300 is used to calculate the adaptive weight group by analyzing the local characteristics of the second image; the adaptive weight group is used to characterize the local reflection intensity or texture complexity of the second image; The difference module 400 is used to perform weighted processing on the first image according to the adaptive weight group, and subtract the weighted first image from the second image to obtain a difference image; The analysis and feedback module 500 is used to extract the bubble defect information inside the glass by analyzing the differential image; the bubble defect information includes the position, size, and quantity of the bubble defects; the bubble defect information is fed back to the production control system in real time and used to indicate the existence of quality defects.

[0079] In some embodiments, when the analysis and calculation module 300 is used to calculate the adaptive weight group by analyzing the local characteristics of the second image, it performs: S31. Divide the second image into a plurality of non-overlapping image blocks; S32. For each image block, calculate the gray-level co-occurrence matrix of each image block, and extract the corresponding texture feature parameters based on the gray-level co-occurrence matrix. The texture feature parameters include energy, contrast, correlation, and entropy; S33. For each image block, calculate the gradient magnitude of each image block, and count the number of pixel points whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter; S34. According to the texture feature parameters and gradient complexity parameters of each image block, use a pre-trained prediction model to predict the reflection intensity weight value of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters, and reflection intensity weight values corresponding to the image blocks under different reflection intensities, and the prediction model uses the cross-validation method for parameter optimization; S35. Combine the reflection intensity weight values of all image blocks into an adaptive weight group.

[0080] In some embodiments, when the analysis and calculation module 300 is used to predict the reflection intensity weight value of each image block according to the texture feature parameters and gradient complexity parameters of each image block by using a pre-trained prediction model, it performs: S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, texture feature parameters, and gradient complexity parameters to obtain a fused feature vector; S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block according to the fused feature vector; the prediction model is a deep neural network with multiple hidden layers. The input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value, and the deep neural network is trained using the Dropout regularization method.

[0081] In some embodiments, when the difference module 400 is used to perform weighted processing on the first image according to the adaptive weight group and subtract the weighted first image from the second image to obtain a differential image, it performs: S41. For the first image and the second image, use an image registration algorithm based on mutual information for precise alignment and calculate the spatial transformation matrix; S42. Geometrically transform the first image according to the spatial transformation matrix to eliminate the misalignment between the first image and the second image; S43. Weight the geometrically transformed first image according to the adaptive weight combination to obtain the weighted first image; S44. Subtract the weighted first image from the second image to obtain the initial difference image; S45. Perform multi-scale morphological filtering on the initial difference image to suppress the artifacts caused by registration errors and obtain the difference image.

[0082] In some embodiments, when the analysis feedback module 500 is used to extract the bubble defect information inside the glass by analyzing the difference image, it performs: S51. Perform adaptive filtering on the difference image to reduce noise interference; S52. Segment the filtered difference image using an image segmentation algorithm based on region growing to obtain multiple candidate bubble regions; S53. For each candidate bubble region, calculate the shape feature parameters; the shape feature parameters include area, perimeter, roundness, and convexity; S54. According to the shape feature parameters, use a pre-trained classifier to determine whether each candidate bubble region is a real bubble defect, and when the determination is yes, record the bubble defect information of the real bubble defect, otherwise do not record the bubble defect information; wherein, the size of the bubble defect in the bubble defect information is extracted by analyzing the shape feature parameters of the corresponding candidate bubble region; S55. Real-time feedback the bubble defect information to the production control system.

[0083] Please refer to Figure 3 , Figure 3A schematic structural diagram of an electronic device provided by an embodiment of the present invention. The present invention provides an electronic device 13, including: a processor 1301 and a memory 1302. The processor 1301 and the memory 1302 are interconnected and communicate with each other through a communication bus 1303 and / or other forms of connection mechanisms (not marked). The memory 1302 stores computer-readable instructions executable by the processor 1301. When the electronic device runs, the processor 1301 executes the computer-readable instructions to execute the window and door glass defect feedback control method in any optional implementation manner of the above embodiment to achieve the following functions: after adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the bubbles inside the glass, collect a first image; after adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, collect a second image; calculate an adaptive weight group by analyzing the local characteristics of the second image; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local part of the second image; according to the adaptive weight group, perform weighted processing on the first image, and subtract the weighted first image from the second image to obtain a difference image; extract the bubble defect information inside the glass by analyzing the difference image; the bubble defect information includes the position of the bubble defect, the size of the bubble defect, and the number of bubble defects; the bubble defect information is real-time fed back to the production control system and used to prompt the existence of quality defects.

[0084] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it executes the window and door glass defect feedback control method in any optional implementation manner of the above embodiment to achieve the following functions: after adjusting the polarized light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the bubbles inside the glass, collect a first image; after adjusting the polarized light imaging detection device to enhance the reflected light on the glass surface, collect a second image; calculate an adaptive weight group by analyzing the local characteristics of the second image; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local part of the second image; according to the adaptive weight group, perform weighted processing on the first image, and subtract the weighted first image from the second image to obtain a difference image; extract the bubble defect information inside the glass by analyzing the difference image; the bubble defect information includes the position of the bubble defect, the size of the bubble defect, and the number of bubble defects; the bubble defect information is real-time fed back to the production control system and used to prompt the existence of quality defects.

[0085] Among them, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disc.

[0086] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0087] In addition, the units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] Furthermore, in each embodiment of the present invention, the functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0089] In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0090] The above are only embodiments of the present invention and are not intended to limit the protection scope of the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for defect feedback control of door and window glass, which is applied to a production control system with a polarized light imaging detection device, is characterized in that, It includes the following steps: S1. After adjusting the polarization light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the bubbles inside the glass, acquire the first image; S2. After adjusting the polarization light imaging detection device to enhance the reflected light on the glass surface, acquire the second image; S3. By analyzing the local characteristics of the second image, calculate the adaptive weight group; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local part of the second image; S4. According to the adaptive weight group, perform weighted processing on the first image, and subtract the weighted first image from the second image to obtain a differential image; S5. By analyzing the differential image, extract the bubble defect information inside the glass; the bubble defect information includes the position, size, and quantity of the bubble defects; the bubble defect information is fed back to the production control system in real time and is used to indicate the existence of quality defects.

2. The method for controlling the feedback of window and door glass defects according to claim 1, characterized in that, The specific steps in step S3 include: S31. Divide the second image into multiple non-overlapping image blocks; S32. For each image block, calculate the gray-level co-occurrence matrix of each image block, and extract the corresponding texture feature parameters based on the gray-level co-occurrence matrix. The texture feature parameters include energy, contrast, correlation, and entropy; S33. For each image block, calculate the gradient magnitude of each image block, and count the number of pixel points whose gradient magnitude is greater than a preset threshold as the corresponding gradient complexity parameter; S34. According to the texture feature parameters and gradient complexity parameters of each image block, use a pre-trained prediction model to predict the reflection intensity weight value of each image block; the training samples of the prediction model include the texture feature parameters, gradient complexity parameters, and reflection intensity weight values corresponding to the image blocks under different reflection intensities, and the prediction model uses the cross-validation method for parameter optimization; S35. Combine the reflection intensity weight values of all the image blocks into the adaptive weight group.

3. The method for controlling the feedback of window and door glass defects according to claim 2, wherein, The specific steps in step S34 include: S341. For each image block, calculate the LBP feature of the local binary pattern, and fuse the LBP feature, the texture feature parameters, and the gradient complexity parameter to obtain a fused feature vector; S342. Use a pre-trained prediction model to predict the reflection intensity weight value of each image block according to the fused feature vector; the prediction model is a deep neural network with multiple hidden layers. The input of the deep neural network is the fused feature vector, and the output is the reflection intensity weight value, and the deep neural network is trained using the Dropout regularization method.

4. The method for defect feedback control of door and window glass according to claim 3, wherein The deep neural network is trained using the Dropout regularization method through the following steps: S3421. For each hidden layer of the deep neural network, randomly discard the output of the neurons in this layer with a probability of 50%, generate a mask vector, and multiply the mask vector by the output of the neurons in this layer to obtain the output of the neurons after Dropout; S3422. For each hidden layer, input the neuron outputs after Dropout of that hidden layer into the output layer of the deep neural network, and obtain a trained deep neural network through training; the deep neural network is used to output the reflection intensity weight value through the Sigmoid activation function, and the output reflection intensity weight value is between 0 and 1.

5. The method for defect feedback control of door and window glass according to claim 4, characterized in that, The specific steps in step S3422 include: S34221. For each hidden layer, calculate the weighted value of that hidden layer according to the connection weight between the hidden layer and the output layer and the neuron outputs after Dropout of that hidden layer; S34222. Calculate the weighted average value based on the weighted values of all the hidden layers and use it as the fused feature representation; S34223. Input the fused feature representation into the Sigmoid activation function, and adjust the parameters of the Sigmoid function through training to obtain a trained deep neural network.

6. The method for defect feedback control of door and window glass according to claim 1, wherein, The specific steps in step S4 include: S41. For the first image and the second image, use an image registration algorithm based on mutual information for precise alignment and calculate the spatial transformation matrix; S42. According to the spatial transformation matrix, perform a geometric transformation on the first image to eliminate the misalignment between the first image and the second image; S43. According to the adaptive weight group, perform a weighting process on the first image after geometric transformation to obtain a weighted first image; S44. Subtract the weighted first image from the second image to obtain an initial difference image; S45. Perform multi-scale morphological filtering on the initial difference image to suppress the artifacts caused by registration errors and obtain the difference image.

7. The method for defect feedback control of door and window glass according to claim 1, wherein The specific steps in step S5 include: S51. Perform an adaptive filtering process on the difference image to reduce noise interference; S52. Use an image segmentation algorithm based on region growing to segment the filtered difference image to obtain multiple candidate bubble regions; S53. For each candidate bubble region, calculate the shape feature parameters; the shape feature parameters include area, perimeter, circularity, and convexity; S54. According to the shape feature parameters, use a pre-trained classifier to determine whether each candidate bubble region is a real bubble defect, and when the determination is yes, record the bubble defect information of the real bubble defect, otherwise do not record the bubble defect information; among them, the size of the bubble defect in the bubble defect information is extracted by analyzing the shape feature parameters of the corresponding candidate bubble region; S55. Real-time feedback the bubble defect information to the production control system.

8. A defect feedback control device for door and window glass, which is applied to a production control system with a polarized light imaging detection device, is characterized in that including: A first acquisition module, configured to acquire a first image after adjusting the polarization light imaging detection device to suppress the reflected light on the glass surface and enhance the scattered light of the internal bubbles in the glass; A second acquisition module, configured to acquire a second image after adjusting the polarization light imaging detection device to enhance the reflected light on the glass surface; An analysis and calculation module, configured to calculate an adaptive weight group by analyzing the local characteristics of the second image; the adaptive weight group is used to characterize the reflection intensity or texture complexity of the local part of the second image; A differential module, configured to perform weighted processing on the first image according to the adaptive weight combination, and subtract the weighted first image from the second image to obtain a differential image; An analysis and feedback module, configured to extract bubble defect information inside the glass by analyzing the differential image; the bubble defect information includes the position, size and quantity of the bubble defects; the bubble defect information is real-time fed back to the production control system and used to indicate the existence of quality defects.

9. An electronic device, characterized in that, It includes a processor and a memory, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method for feedback control of door and window glass defects according to any one of claims 1-7 are run.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps in the method for feedback control of door and window glass defects according to any one of claims 1-7 are run.

Images