Glass production process automatic monitoring system and method

Abstract

The invention relates to the field of glass production, and particularly discloses an automatic monitoring system and method for a glass production process, and the method comprises the steps: firstly obtaining a laminated glass packaging monitoring video collected by a camera, then carrying out the feature extraction and correlation analysis of the laminated glass packaging monitoring video through a deep learning technology, and finally obtaining a classification result through a classifier, therefore, abnormal conditions can be found in advance, subsequent rework caused by product defects can be avoided, the production cost can be reduced, and the overall production efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of glass production, and more specifically, to an automatic monitoring system and method for a glass production process. Background Art

[0002] Laminated glass is a kind of safety glass made of two or more pieces of glass bonded together by an interlayer (usually polyvinyl butyral, PVB for short). Its production process involves multiple links. After the production is completed, due to the thin thickness of laminated glass, in order to prevent scratches or adhesion caused by contact between two adjacent laminated glasses, spacer paper is usually placed on the outside of the glass during packaging to maintain a proper spacing.

[0003] However, the existing technology often faces some problems when using spacer paper for glass packaging. For example, the spacer paper may be folded, deviated (too low or too high), etc., which will lead to improper packaging and affect the overall quality of the product. In addition, if the glass is broken during the packaging process, it is often difficult to detect it in time, which not only affects the smooth progress of subsequent processes, but also may increase production costs.

[0004] Therefore, a system and method for automatically monitoring a glass production process is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a glass production process automatic monitoring system and method, which first obtains the laminated glass packaging monitoring video collected by the camera, then uses deep learning technology to perform feature extraction and association analysis, and finally obtains the classification result through the classifier to determine whether to issue an abnormal warning for laminated glass packaging, so as to detect abnormal conditions in advance, avoid subsequent rework due to product defects, reduce production costs, and improve overall production efficiency.

[0006] According to one aspect of the present application, a glass production process automatic monitoring system is provided, which includes: The laminated glass production data acquisition module is used to obtain the laminated glass packaging monitoring video collected by the camera; A laminated glass production data extraction module, used to extract laminated glass target significant feature vectors and spacer paper target interesting feature vectors from the laminated glass packaging monitoring video collected by the camera; The laminated glass packaging abnormality warning judgment module is used to judge whether to issue a laminated glass packaging abnormality warning based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector.

[0007] According to another aspect of the present application, a method for automatically monitoring a glass production process is provided, comprising: Obtaining monitoring video of laminated glass packaging collected by a camera; Extracting a laminated glass target salient feature vector and a spacer paper target interesting feature vector from the laminated glass packaging monitoring video collected by the camera; Based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector, it is determined whether to issue a laminated glass packaging abnormality warning.

[0008] Compared with the prior art, the present application provides an automatic monitoring system and method for a glass production process, which first obtains the monitoring video of the laminated glass packaging collected by a camera, then uses deep learning technology to perform feature extraction and association analysis, and finally obtains the classification result through a classifier to determine whether to issue an abnormal warning for the laminated glass packaging, thereby discovering abnormal situations in advance, avoiding subsequent rework due to product defects, reducing production costs, and improving overall production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 4 is a block diagram of a glass production process automatic monitoring system according to an embodiment of the present application.

[0011] Figure 2 4 is a block diagram of a laminated glass production data extraction module in a glass production process automatic monitoring system according to an embodiment of the present application.

[0012] Figure 3 4 is a block diagram of a laminated glass region feature extraction unit in a glass production process automatic monitoring system according to an embodiment of the present application.

[0013] Figure 4 The block diagram is a module for early warning and judging abnormality of laminated glass packaging in the automatic monitoring system of glass production process according to an embodiment of the present application.

[0014] Figure 5 Flow chart of a method for automatically monitoring a glass production process according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0016] Figure 1 FIG. 1 is a block diagram of the automatic monitoring system for the glass production process according to an embodiment of the present application. Figure 1 As shown, according to the glass production process automatic monitoring system 100 of the embodiment of the present application, it includes: a laminated glass production data acquisition module 110, which is used to obtain the laminated glass packaging monitoring video collected by a camera; a laminated glass production data extraction module 120, which is used to extract the laminated glass target significant feature vector and the spacer paper target interesting feature vector from the laminated glass packaging monitoring video collected by the camera; a laminated glass packaging abnormality warning judgment module 130, which is used to judge whether to issue a laminated glass packaging abnormality warning based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector.

[0017] In the above-mentioned automatic monitoring system 100 for the glass production process, the laminated glass production data acquisition module 110 is used to obtain the laminated glass packaging monitoring video collected by the camera. It should be understood that laminated glass is a safety glass made of two or more pieces of glass bonded together by an intermediate interlayer (usually polyvinyl butyral, i.e., PVB). Its production process is complex and involves multiple links. After the production is completed, due to the thin thickness of the laminated glass, in order to prevent scratches or adhesion caused by contact between two adjacent pieces of glass, spacer paper is usually placed on the outside of the glass during packaging to maintain an appropriate spacing. However, the prior art often faces some problems when using spacer paper for glass packaging. For example, the spacer paper may be folded, deviated from the position (too low or too high), etc., resulting in improper packaging, thereby affecting the overall quality of the product. In addition, if glass breakage occurs during the packaging process, it is often difficult to detect in time, which will not only affect the smooth progress of subsequent processes, but also increase production costs.

[0018] In the technical solution of the present application, a camera with high-definition resolution and night vision function is selected to capture the status of laminated glass production and packaging, so as to effectively ensure the monitoring and management of the laminated glass packaging process and improve production efficiency and safety. Therefore, in the technical solution of the present application, by obtaining the laminated glass packaging monitoring video collected by the camera and combining it with deep learning technology, it is determined whether it is necessary to issue an abnormal warning for laminated glass packaging, so as to detect abnormal conditions in advance and avoid subsequent rework due to product defects, thereby reducing production costs and improving overall production efficiency.

[0019] In the above-mentioned glass production process automatic monitoring system 100, the laminated glass production data extraction module 120 is used to extract the laminated glass target significant feature vector and the spacer paper target interesting feature vector from the laminated glass packaging monitoring video collected by the camera. It should be understood that by accurately extracting the significant feature vectors of laminated glass and spacer paper, the intelligence level of the monitoring system can be improved, and an effective technical means is provided for the quality control of industrial production.

[0020] Figure 2 FIG. 1 is a block diagram of a laminated glass production data extraction module in a glass production process automatic monitoring system according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the laminated glass production data extraction module 120 includes: a laminated glass area feature extraction unit 121, used to perform laminated glass area feature extraction on the laminated glass packaging monitoring video collected by the camera to obtain the laminated glass target significant feature vector; a spacer paper area feature extraction unit 122, used to perform spacer paper area feature extraction on the laminated glass packaging monitoring video collected by the camera to obtain the spacer paper target interesting feature vector.

[0021] It should be understood that laminated glass is easily affected by various factors during the production process, such as breakage, scratches or stains. Therefore, through feature extraction, these problems can be discovered in time to ensure product quality. Furthermore, the spacer paper plays a buffering and protective role in the packaging of laminated glass. Therefore, timely identification and detection of the state of the spacer paper is a key link in ensuring product quality.

[0022] Figure 3 FIG. 1 is a block diagram of a laminated glass region feature extraction unit in a glass production process automatic monitoring system according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the laminated glass area feature extraction unit 121 includes: a laminated glass area key frame extraction subunit 1211, used to extract multiple laminated glass area key frames from the laminated glass packaging monitoring video collected by the camera; a laminated glass area feature encoding subunit 1212, used to feature encode the multiple laminated glass area key frames to obtain a laminated glass target significant feature map; a laminated glass target significant maximum pooling subunit 1213, used to perform maximum pooling on the laminated glass target significant feature map to obtain the laminated glass target significant feature vector.

[0023] It should be understood that extracting key frames of multiple laminated glass areas from the laminated glass packaging monitoring video collected by the camera can comprehensively analyze the packaging status of the laminated glass, so as to timely discover potential quality problems. In particular, key frames refer to frames that are representative and informative in the video, which can reflect important events or state changes. Extracting these key frames can effectively reduce the amount of video data while retaining important information, which is convenient for subsequent analysis and processing. Specifically, first, in the video preprocessing stage, the video is basically processed, including denoising and adjusting the image quality, to ensure the accuracy of subsequent processing. This can be done by methods such as image smoothing and histogram equalization, thereby improving the visibility of the laminated glass area. Next, each frame in the video is analyzed to determine its importance. Common methods include calculating the difference between each frame and the previous frame or using optical flow to detect motion changes. Frames with high inter-frame differences usually indicate important state changes and should be selected first. In addition, specific algorithms (such as clustering or key frame selection algorithms) can be used to classify frames and filter out representative frames. Finally, the selected key frames are saved as image files. These key frames can not only reflect the state changes of laminated glass, but also can be used for subsequent image analysis, feature extraction and pattern recognition. By analyzing these key frames, automatic detection and quality assessment of laminated glass can be achieved.

[0024] Furthermore, feature encoding of multiple laminated glass region key frames can effectively extract and express the visual features of laminated glass, helping the system to identify and classify laminated glass in different states and ensure product quality during the packaging process. Feature encoding converts the visual information in the key frames into a processable format, making subsequent analysis and decision-making more efficient.

[0025] Furthermore, the maximum value pooling of the laminated glass target salient feature map can reduce the dimension of the feature map while retaining the most significant feature information. Among them, the main purpose of maximum value pooling is to extract the strongest features from the feature map, highlight the salient features of laminated glass, remove redundant information, and thus improve the generalization ability of the model. Specifically, first, the salient feature map is divided into several fixed-size regions (such as 2x2 or 3x3 windows). This block method helps to find the most significant features in the local range and ensure that the feature information of each region is fully utilized. Next, the features in each block area are compared to extract the maximum value in the region. This step can effectively capture the important information in the local features, and by selecting only the maximum value, those smaller or unimportant features can be suppressed, making the extracted features more representative. This method helps to reduce the size of the feature map, reduce the computational complexity, and thus increase the speed of subsequent processing. Finally, after maximum value pooling, all the extracted maxima are combined into a one-dimensional feature vector to form a laminated glass target salient feature vector. This vector contains the most representative feature information and can be used for subsequent classification and recognition tasks. By converting significant feature maps into feature vectors, the system can perform subsequent machine learning and analysis more conveniently, thereby improving the efficiency of automatic detection and recognition of laminated glass.

[0026] In a specific embodiment of the present application, the laminated glass area feature encoding subunit 1212 is used to: pass the multiple laminated glass area key frames through a laminated glass target of interest detection network to obtain multiple laminated glass target regions of interest; pass the multiple laminated glass target regions of interest through a laminated glass target salient target detector to obtain the laminated glass target salient feature map.

[0027] It should be understood that by processing multiple laminated glass area key frames through the laminated glass object detection network of interest, the key areas of laminated glass can be effectively identified and located, thereby improving the accuracy and efficiency of product quality detection. Among them, by using a dedicated target detection network, it can be ensured that the system accurately identifies the characteristics of laminated glass in a complex environment and detects potential problems in a timely manner. Specifically, first, a deep learning algorithm such as a convolutional neural network is used. By inputting the labeled data into the target detection network, the network will optimize its own parameters through multiple rounds of iterations to learn the characteristics and patterns of laminated glass. In this process, modern target detection frameworks such as YOLO (You Only Look Once) and Faster R-CNN can be used, which can achieve more accurate detection effects at a higher speed. During the training process, the network will minimize the error between the predicted results and the actual annotations by continuously adjusting the weights, and finally form an efficient detection model. Then, the key frames of multiple laminated glass areas are input into the trained target detection network. The network will analyze the characteristics of each pixel in the image through forward propagation and generate regions of interest (ROIs), which are the detected laminated glass targets. The network output result usually includes the location information of each target (such as the frame coordinates) and the corresponding confidence score to help the user judge the accuracy of the target. Specifically, the multiple laminated glass area key frames are passed through the laminated glass object of interest detection network to obtain multiple laminated glass object interest regions, including: using the laminated glass object of interest detection network to process the multiple laminated glass area key frames with the following object detection formula to obtain the multiple laminated glass object interest regions; Among them, the target detection formula is: ;in, For multiple laminated glass area keyframes, is the anchor frame, For multiple laminated glass target areas of interest, Indicates classification, Indicates regression.

[0028] Furthermore, multiple laminated glass target regions of interest are processed by laminated glass target salient object detectors, and salient features of laminated glass can be extracted. Among them, the salient feature map can emphasize the key features in the target area, making it easier for the system to focus on important information and improve the overall intelligence level of monitoring. Specifically, firstly, relying on the laminated glass target salient object detector (usually a model based on deep learning, such as a convolutional neural network), the input laminated glass target region of interest is analyzed to automatically extract salient features related to the target. These features may include color, texture, shape and other visual information, which can accurately reflect the state of laminated glass. By learning features at different levels, the network can capture subtle changes in laminated glass, thereby improving the sensitivity and accuracy of detection. Then, the detector integrates the extracted features into a salient feature map. This image reflects the most important feature information in the laminated glass area. By weighting and combining different features, a high-dimensional feature representation can be formed, which is convenient for subsequent analysis and processing. This salient feature map improves the recognition ability of laminated glass.

[0029] Specifically, the multiple laminated glass target regions of interest are passed through a laminated glass target salient target detector to obtain the laminated glass target salient feature map, including: using each layer of the laminated glass target salient target detector to respectively perform the following on the input data in the forward pass of the layer: using a first convolution kernel to perform convolution processing on the input data to obtain a first convolution feature map; using a second convolution kernel to perform convolution processing on the first convolution feature map to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than the size of the second convolution kernel; performing pooling processing on the second convolution feature map to obtain a pooling feature map; performing activation processing on the pooling feature map to obtain an activation feature map; wherein the output of the last layer of the laminated glass target salient target detector is the laminated glass target salient feature map, and the input of the first layer of the laminated glass target salient target detector is the multiple laminated glass target regions of interest.

[0030] In a specific embodiment of the present application, the laminated glass target significant maximum pooling subunit is used to: install TensorFlow, NumPy and OpenCV dependent libraries; read the input laminated glass target significant feature map; use the MaxPooling2D maximum pooling layer in TensorFlow provided by the deep learning framework, and configure the parameters of the pooling layer; use the Stack mechanism to manage the input and output of the model; push the laminated glass target significant feature map into the stack; take out the laminated glass target significant feature map from the stack and apply the maximum pooling operation; push the pooled laminated glass target significant feature vector back to the stack.

[0031] Part of the deployment code is shown below.

[0032] import torch import torch.nn as nn import numpy as np # Laminated glass target salient feature map class FeatureMapGenerator(nn.Module): def __init__(self): super(FeatureMapGenerator, self).__init__() # Define the structure of the model self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1,padding=1) self.relu = nn.ReLU() self.pool = nn.MaxPool2d(kernel_size=2, stride=2) def forward(self, x): x = self.conv1(x) x = self.relu(x) x = self.pool(x) return x # Load the pre-trained model feature_map_generator = FeatureMapGenerator() feature_map_generator.load_state_dict(torch.load('feature_map_generator.pth')) feature_map_generator.eval() # Define the maximum pooling layer max_pool = nn.MaxPool2d(kernel_size=2, stride=2) # Stack mechanism class Stack: def __init__(self): self.items = [] def push(self, item): self.items.append(item) def pop(self): return self.items.pop() def is_empty(self): return len(self.items) == 0 # Create a stack instance stack = Stack() # Laminated glass target salient feature map feature_map = torch.randn(1, 64, 64, 64) # Assume the shape of the feature map is (1, 64,64, 64) # Push the feature map onto the stack stack.push(feature_map) # Take the feature map from the stack and apply max pooling feature_map = stack.pop() pooled_feature_map = max_pool(feature_map) # Push the pooled feature vector back to the stack stack.push(pooled_feature_map) # Take the final feature vector from the stack final_feature_vector = stack.pop() # Print the final feature vector print("Final Feature Vector Shape:", final_feature_vector.shape) print("Final Feature Vector:", final_feature_vector) The above deployment code demonstrates the core functions of the laminated glass packaging anomaly detection system implemented using the PyTorch framework. The system extracts key information from the surveillance video through a series of deep learning models and image processing techniques, and finally determines whether there is an anomaly in the laminated glass packaging. Among them, FeatureMapGenerator is a simple convolutional neural network model used to generate a laminated glass target salient feature map. The model processes the input image through convolutional layers, activation functions, and pooling layers to extract representative features. MaxPool2d is a maximum pooling layer in PyTorch, which is used to perform pooling operations on feature maps to reduce the dimension of feature maps while retaining the most important feature information. Stack is a simple stack implementation for managing feature maps and pooled feature vectors. The stack mechanism ensures the orderly processing of feature maps and feature vectors, which is convenient for subsequent fusion and classification operations. In the stack operation, the feature map is first pushed into the stack, then the feature map is taken out of the stack and the maximum pooling operation is applied, and finally the pooled feature vector is pushed back into the stack. This process ensures the orderly processing of feature maps and feature vectors, which is convenient for subsequent fusion and classification operations. In this way, by using the pre-trained convolutional neural network model, the system is able to extract representative feature maps from surveillance videos and further simplify the feature representation through maximum pooling operations.

[0033] Among them, the introduction of the stack mechanism ensures the orderly processing of feature maps and feature vectors, which facilitates subsequent fusion and classification operations. In practical applications, the system can efficiently process a large amount of monitoring video data, extract key feature information, and ultimately determine whether there is an abnormality in the laminated glass packaging. By fusing the significant feature vector of the laminated glass target and the feature vector of interest of the spacer paper target, the system can generate a comprehensive classification feature vector and make accurate classification judgments through the classifier. The deployment of this system not only improves the accuracy and reliability of laminated glass packaging anomaly detection, but also greatly reduces the workload of manual detection. Through automated processing, the system can complete the analysis of a large amount of video data in a short time, promptly discover and warn of potential packaging anomalies, thereby improving production efficiency and product quality.

[0034] In a specific embodiment of the present application, the spacer paper area feature extraction unit 122 is used to: extract key frames from the laminated glass packaging monitoring video collected by the camera to obtain multiple spacer paper area key frames; pass the multiple spacer paper area key frames through a spacer paper target of interest detection network to obtain multiple spacer paper target regions of interest; arrange the multiple spacer paper target regions of interest into spacer paper target of interest input tensors; pass the spacer paper target of interest input tensor through a spacer paper target of interest feature encoder based on a three-dimensional convolutional neural network to obtain the spacer paper target of interest feature vector.

[0035] It should be understood that by selecting representative image frames from a continuous video stream, redundant data can be reduced and important information can be focused on. Among them. Extracting key frames not only helps to improve the processing speed of the algorithm, but also enhances the model's ability to recognize specific areas, ensuring that the packaging process of laminated glass is effectively monitored. Furthermore, processing multiple spacer paper area key frames through the spacer paper target of interest detection network can effectively identify and locate the key areas of the spacer paper, ensure that potential problems are discovered in time during production and application, and improve the overall quality management efficiency. Through the target detection network, spacer paper in different states can be accurately detected, thereby providing reliable data support for subsequent quality assessment and control. Furthermore, arranging multiple spacer paper target interest regions as spacer paper target interest input tensors can effectively integrate information, ensuring that the model can process high-dimensional data when performing feature learning and reasoning, thereby improving detection accuracy and efficiency. Among them, the process of integrating multiple spacer paper target interest regions into input tensors can ensure the uniformity and standardization of input data.

[0036] In particular, the interpolated object input tensor of interest is processed by the interpolated object feature encoder based on a three-dimensional convolutional neural network, which can take advantage of deep learning and extract and represent the input data through multi-layer convolution operations, so that the network can capture the spatial characteristics and temporal information of the interpolated paper and improve the accuracy of detection. Among them, the interpolated object input tensor of interest is a high-dimensional data structure, usually containing multiple channels, widths, and heights. Through the three-dimensional convolutional neural network, the model is able to process this structure and extract rich information with spatial and temporal characteristics. The three-dimensional convolutional neural network uses convolution kernels to slide in three dimensions (time, width, and height) to capture the complex characteristics of the target in space. This method is particularly suitable for processing video data or continuous image sequences because it can effectively learn dynamic changes in time series. Specifically, the feature encoder based on the three-dimensional convolutional neural network processes the input tensor with multiple convolutional layers. Each convolutional layer extracts different levels of features by learning weights, from low-level features (such as edges and textures) to high-level features (such as shapes and patterns). Through multi-layer convolution operations, the network will downsample the input data to reduce the data dimension while maintaining key information. Finally, after these convolutional layers, the feature map is flattened and converted into a feature vector, which can efficiently represent the feature information of the spacer paper object. The obtained feature vector of interest of the spacer paper object is not only compact but also rich in information, which is convenient for subsequent classification and regression tasks.

[0037] Specifically, the interval paper target interest input tensor is passed through an interval paper target interest feature encoder based on a three-dimensional convolutional neural network to obtain the interval paper target interest feature vector, including: using each layer of the interval paper target interest feature encoder based on a three-dimensional convolutional neural network to perform convolution processing, mean pooling processing based on a local feature matrix and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the interval paper target interest feature encoder based on the three-dimensional convolutional neural network outputs the interval paper target interest feature vector, wherein the input of the interval paper target interest feature encoder based on the three-dimensional convolutional neural network is the interval paper target interest input tensor.

[0038] In the above-mentioned glass production process automatic monitoring system 100, the laminated glass packaging abnormality warning judgment module 130 is used to determine whether to issue a laminated glass packaging abnormality warning based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector. In this way, intelligent monitoring and automatic quality control can be achieved. Among them, by analyzing these two feature vectors, the system can effectively identify abnormal situations that may occur in the laminated glass packaging process, thereby issuing an alarm in time to prevent potential quality problems and economic losses.

[0039] Figure 4 FIG. 1 is a block diagram of a laminated glass packaging abnormality warning judgment module in a glass production process automatic monitoring system according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the laminated glass packaging abnormality warning judgment module 130 includes: a laminated glass production feature fusion unit 131, which is used to perform eigenvalue granularity correlation modulation on the laminated glass target significant feature vector and the spacer paper target interesting feature vector based on the high-dimensional invisible space to obtain a laminated glass abnormality judgment classification feature vector; a packaging abnormality warning judgment classification unit 132, which is used to pass the laminated glass abnormality judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a laminated glass packaging abnormality warning.

[0040] It should be understood that the target salient feature vector of laminated glass contains the morphology, texture, color and other key quality indicators of the laminated glass itself, while the target interesting feature vector of spacer paper reflects the state, location and possible defects of the packaging material. Fusion of these two features can provide a more comprehensive perspective, allowing analysis in multiple dimensions. For example, the salient features of laminated glass may be affected by the state of the packaging material, while the characteristics of the packaging material may be related to the integrity of the laminated glass. Fusion of these two features helps capture potential correlations and more accurately judge the abnormal state of laminated glass.

[0041] In particular, in the technical solution of the present application, the laminated glass target salient feature vector is generated by analyzing the salient feature map extracted from the laminated glass packaging video, emphasizing the visual features of laminated glass, such as light reflection, surface texture and shape, focusing on its appearance and position, and being able to reflect the state and characteristics of laminated glass in a specific scene. It is rich in spatiotemporal information and geometric features, and is helpful in capturing the integrity and potential abnormalities of packaging. In contrast, the spacer paper target feature vector of interest is based on the video analysis of the spacer paper area, focusing on the structural characteristics, material and arrangement of the spacer paper relative to the laminated glass. The spacer paper target feature vector of interest is extracted through a three-dimensional convolutional neural network, focusing on the motion and deformation information in the time series, reflecting the supporting and protective role of the spacer paper in the laminated glass packaging process. Therefore, its feature information is relatively abstract and dynamic, and is more related to the integrity and stability of the packaging process. Due to the fundamental differences in content, structure and information expression between the laminated glass target salient feature vector and the spacer paper target feature vector of interest, they show significant incompatibility in feature space. When fusing the laminated glass target significant feature vector and the spacer paper target interesting feature vector, the incoordination of the feature manifold and feature space caused by heterogeneity will cause the fused laminated glass abnormality judgment classification feature vector to lack smoothness. This lack of smoothness will lead to unstable performance of the feature in the classifier, thereby affecting the certainty of the feature expression and reducing the classification accuracy and robustness. Therefore, in the technical solution of the present application, the laminated glass target significant feature vector and the spacer paper target interesting feature vector are modulated by eigenvalue granularity association based on high-dimensional invisible space to obtain the laminated glass abnormality judgment classification feature vector.

[0042] Among them, the laminated glass target significant feature vector and the spacer paper target feature vector of interest are subjected to eigenvalue granularity correlation modulation based on high-dimensional invisible space to obtain the laminated glass abnormality judgment classification feature vector, including: constructing a glass distance topological matrix between the laminated glass target significant feature vector and the spacer paper target feature vector of interest; constructing a glass eigenvalue granularity correlation matrix between the laminated glass target significant feature vector and the spacer paper target feature vector of interest; performing topological correlation modulation on the glass eigenvalue granularity correlation matrix based on the glass distance topological matrix to obtain a glass topological modulation correlation matrix; using the glass topological modulation correlation matrix as the perspective co-projection space, projecting the laminated glass target significant feature vector and the spacer paper target feature vector of interest to the perspective co-projection space to obtain the perspective modulated laminated glass target significant feature vector and the perspective modulated spacer paper target feature vector of interest; fusing the perspective modulated laminated glass target significant feature vector and the perspective modulated spacer paper target feature vector of interest to obtain the laminated glass abnormality judgment classification feature vector.

[0043] The fusion step is specifically expressed as: , is the glass distance topological matrix between the laminated glass target significant feature vector and the spacer paper target interesting feature vector, that is, ,and and are column vectors. ;in, represents the significant feature vector of the laminated glass target, represents the feature vector of interest of the spacer paper target, is the glass distance topological matrix between the laminated glass target significant feature vector and the spacer paper target interesting feature vector, represents the glass distance topology matrix, represents the transpose of a vector, represents matrix multiplication, represents the glass eigenvalue particle size correlation matrix, represents the glass topological modulation correlation matrix, represents the salient feature vector of the view-modulated laminated glass target, Represents the feature vector of interest of the target in the view modulation interval paper, and represents the weighted hyperparameter, Represents the laminated glass abnormality judgment classification feature vector.

[0044] In the technical solution of the present application, the laminated glass target significant feature vector and the spacer paper target interesting feature vector have relatively significant feature heterogeneity, which results in the feature manifold and feature space mismatch caused by the heterogeneity when performing feature fusion of the laminated glass target significant feature vector and the spacer paper target interesting feature vector, which will cause the laminated glass abnormality judgment classification feature vector to fail to have feature manifold smoothness, thereby affecting the certainty of its feature expression.

[0045] Based on this, in the technical solution of the present application, the laminated glass target significant feature vector and the spacer paper target feature vector of interest are subjected to eigenvalue granularity correlation modulation based on high-dimensional invisible space, which first constructs a glass distance topological matrix between the laminated glass target significant feature vector and the spacer paper target feature vector of interest. This step utilizes the idea of ​​graph theory, takes the eigenvalues ​​of each position in the laminated glass target significant feature vector and the spacer paper target feature vector of interest as nodes, and uses the distance metric function to determine the low-dimensional embedding expression of the edges between nodes, so as to construct a feature expression that can quantify the topological correlation relationship between the laminated glass target significant feature vector and the spacer paper target feature vector of interest. This step is crucial to understanding the spatial distribution of data points and their mutual connections, and also provides basic structural information for subsequent steps.

[0046] Next, a glass eigenvalue granularity association matrix is ​​constructed between the laminated glass target significant feature vector and the spacer paper target feature vector of interest. This step focuses on exploring the correlation between the internal attributes of the feature vector. In a specific example, the product between the laminated glass target significant feature vector and the transposed vector of the spacer paper target feature vector of interest is calculated to obtain the glass eigenvalue granularity association matrix. Furthermore, the glass eigenvalue granularity association matrix is ​​topologically modulated based on the glass distance topological matrix to obtain a glass topological modulation association matrix. That is, the eigenvalue granularity association information of the laminated glass target significant feature vector and the spacer paper target feature vector of interest is driven to wander on the distance topological map to construct a low-dimensional modulated perspective co-projection space domain for mapping the laminated glass target significant feature vector and the spacer paper target feature vector of interest to a continuous high-dimensional regression space attribute.

[0047] Next, the glass topology modulation correlation matrix is ​​used as the perspective co-projection space, and the laminated glass target significant feature vector and the spacer paper target interesting feature vector are projected into the perspective co-projection space to obtain the perspective modulation laminated glass target significant feature vector and the perspective modulation spacer paper target interesting feature vector. That is, the key to this technical solution is to find an implicit third space with modulation capability, in which as much information of the original high-dimensional data as possible is retained, so that the converted new feature vector not only reduces the dimension, but also maintains the relative position relationship between the original features.

[0048] Finally, the view-modulated laminated glass target significant feature vector and the view-modulated spacer paper target interesting feature vector are fused to obtain the laminated glass abnormality judgment classification feature vector. Fusion can be achieved in various forms, ranging from simple arithmetic average to complex ensemble learning algorithms, depending on the application scenario and the goals pursued. The final laminated glass abnormality judgment classification feature vector combines the main features of the two sets of input data and provides a more comprehensive data representation, which is not only conducive to improving the performance of machine learning models, but also promotes the ability of cross-modal or multi-source data analysis, thereby significantly enhancing the understanding and utilization efficiency of complex data structures.

[0049] Furthermore, the laminated glass abnormality judgment classification feature vector integrates multiple key features of laminated glass and spacer paper, and has good information representation capabilities. The feature vector is input into a classifier (such as a deep learning model), and the classifier will use a pre-trained model to classify the features. By learning normal and abnormal patterns in historical data, the classifier can effectively identify the similarities and differences between the current input features and known patterns. Specifically, the classifier processes the feature vector, calculates its probability distribution under different categories (normal and abnormal), and finally outputs the most likely classification result, that is, issuing an abnormal warning for laminated glass packaging, notifying relevant personnel to conduct further inspection and processing. This process ensures timely response and can reduce potential quality problems and economic losses. This method not only improves the accuracy and response speed of abnormality detection, but also provides strong data support and guarantee for product quality control.

[0050] In summary, the embodiment of the present application first obtains the monitoring video of the laminated glass packaging collected by the camera, and then uses deep learning technology to perform feature extraction and association analysis on it, and finally obtains the classification result through the classifier to determine whether to issue an abnormal warning for the laminated glass packaging, thereby discovering abnormal situations in advance, avoiding subsequent rework due to product defects, reducing production costs, and improving overall production efficiency.

[0051] As described above, the automatic monitoring system 100 for the glass production process according to the embodiment of the present application can be implemented in various terminal devices. In one example, the automatic monitoring system 100 for the glass production process can be integrated into the terminal device as a software module and / or a hardware module. For example, the automatic monitoring system 100 for the glass production process can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the automatic monitoring system 100 for the glass production process can also be one of the many hardware modules of the terminal device.

[0052] Alternatively, in another example, the glass production process automatic monitoring system 100 and the terminal device may also be separate devices, and the glass production process automatic monitoring system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0053] Figure 5 FIG. 1 is a flow chart of a method for automatically monitoring a glass production process according to an embodiment of the present application. Figure 5 As shown, according to the automatic monitoring method of the glass production process of the embodiment of the present application, it includes: S110, acquiring the laminated glass packaging monitoring video collected by the camera; S120, extracting the laminated glass target significant feature vector and the spacer paper target interesting feature vector from the laminated glass packaging monitoring video collected by the camera; S130, judging whether to issue an abnormal warning for laminated glass packaging based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector.

[0054] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned automatic monitoring method for glass production process have been described in detail above. Figures 1 to 4 The invention has been described in detail in the description of the automatic monitoring system for the glass production process, and therefore, its repeated description will be omitted.

Claims

1. A glass production process automatic monitoring system, characterized in that: include: The laminated glass production data acquisition module is used to obtain the laminated glass packaging monitoring video collected by the camera; A laminated glass production data extraction module, used to extract laminated glass target significant feature vectors and spacer paper target interesting feature vectors from the laminated glass packaging monitoring video collected by the camera; The laminated glass packaging abnormality warning judgment module is used to judge whether to issue a laminated glass packaging abnormality warning based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector.

2. The automatic monitoring system for glass production process according to claim 1, characterized in that: The laminated glass production data extraction module comprises: A laminated glass region feature extraction unit, used for performing laminated glass region feature extraction on the laminated glass packaging monitoring video collected by the camera to obtain a laminated glass target significant feature vector; The spacer paper region feature extraction unit is used to extract the spacer paper region features from the laminated glass packaging monitoring video collected by the camera to obtain the spacer paper target feature vector of interest.

3. The automatic monitoring system for glass production process according to claim 2, characterized in that: The laminated glass region feature extraction unit comprises: A laminated glass area key frame extraction subunit, used to extract a plurality of laminated glass area key frames from the laminated glass packaging monitoring video collected by the camera; A laminated glass region feature encoding subunit, used for performing feature encoding on the plurality of laminated glass region key frames to obtain a laminated glass target significant feature map; The laminated glass target significant maximum pooling subunit is used to perform maximum pooling on the laminated glass target significant feature map to obtain the laminated glass target significant feature vector.

4. The automatic monitoring system for glass production process according to claim 3, characterized in that: The laminated glass regional feature encoding subunit is used for: Passing the plurality of laminated glass region key frames through a laminated glass target of interest detection network to obtain a plurality of laminated glass target regions of interest; The multiple laminated glass target interest regions are passed through a laminated glass target interest salient target detector to obtain the laminated glass target salient feature map.

5. The automatic monitoring system for glass production process according to claim 4, characterized in that: The laminated glass targets a significant maximum pooling subunit, which is used to: Install TensorFlow, NumPy, and OpenCV dependency libraries; Read the input laminated glass target salient feature map; Use the MaxPooling2D maximum pooling layer in TensorFlow provided by the deep learning framework and configure the parameters of the pooling layer; Use the Stack mechanism to manage the input and output of the model; Push the laminated glass target salient feature map into the stack; Take out the laminated glass object salient feature map from the stack and apply the maximum pooling operation; Push the pooled laminated glass object salient feature vector back into the stack.

6. The automatic monitoring system for glass production process according to claim 5, characterized in that: The spacer paper region feature extraction unit is used to: Extracting key frames from the laminated glass packaging monitoring video collected by the camera to obtain a plurality of spacer paper area key frames; Passing the plurality of spacer paper region key frames through a spacer paper object of interest detection network to obtain a plurality of spacer paper object regions of interest; Arranging the plurality of spacer paper target regions of interest into a spacer paper target input tensor of interest; The spacer paper target interesting input tensor is passed through a spacer paper target interesting feature encoder based on a three-dimensional convolutional neural network to obtain the spacer paper target interesting feature vector.

7. The automatic monitoring system for glass production process according to claim 6, characterized in that: The laminated glass packaging abnormality early warning judgment module includes: The laminated glass production feature fusion unit is used to perform feature value granularity correlation modulation based on high-dimensional invisible space on the laminated glass target significant feature vector and the spacer paper target interesting feature vector to obtain the laminated glass abnormality judgment classification feature vector; The packaging abnormality warning judgment classification unit is used to pass the laminated glass abnormality judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a laminated glass packaging abnormality warning.

8. The automatic monitoring system for glass production process according to claim 7, characterized in that: The laminated glass production feature fusion unit includes: Constructing a glass distance topological matrix between the laminated glass target significant feature vector and the spacer paper target interesting feature vector; Constructing a glass eigenvalue granularity correlation matrix between the laminated glass target significant eigenvector and the spacer paper target interesting eigenvector; Performing topological correlation modulation on the glass eigenvalue particle size correlation matrix based on the glass distance topological matrix to obtain a glass topological modulation correlation matrix; Taking the glass topology modulation correlation matrix as the perspective co-projection space, projecting the laminated glass target significant feature vector and the spacer paper target interesting feature vector into the perspective co-projection space to obtain the perspective modulation laminated glass target significant feature vector and the perspective modulation spacer paper target interesting feature vector; The significant feature vector of the view angle modulated laminated glass target and the interesting feature vector of the view angle modulated spacer paper target are fused to obtain the laminated glass abnormality judgment classification feature vector.

9. A method for automatically monitoring a glass production process, characterized in that: include: Obtaining monitoring video of laminated glass packaging collected by a camera; Extracting a laminated glass target salient feature vector and a spacer paper target interesting feature vector from the laminated glass packaging monitoring video collected by the camera; Based on the laminated glass target significant feature vector and the spacer paper target interesting feature vector, it is determined whether to issue a laminated glass packaging abnormality warning.

10. The automatic monitoring method for glass production process according to claim 9, characterized in that: Extracting the laminated glass target significant feature vector and the spacer paper target interesting feature vector from the laminated glass packaging monitoring video collected by the camera, including: Extracting laminated glass region features from the laminated glass packaging monitoring video collected by the camera to obtain a laminated glass target significant feature vector; The laminated glass packaging monitoring video collected by the camera is subjected to spacer paper region feature extraction to obtain the spacer paper target feature vector of interest.

Images