An integrated platform for accurate product visual image recognition and processing

Abstract

The present application discloses an integrated platform for accurate identification and processing of product visual images, which involves the field of industrial monitoring, including: collecting product image data and equipment parameters through monitoring nodes; establishing multiple deep learning models on the cloud platform, and training corresponding neural network models for different types of industrial defects; setting up machine learning models on edge computing nodes, training support vector machine models; and setting up anomaly detection algorithms to detect new defect types; using the trained deep learning model to identify defects in product image data; performing defect detection on product image data; adopting a cascade fusion strategy, through multi-stage feature fusion, to improve the accuracy of defect detection; according to the detection results of the edge detection module, according to the preset rules, the defects are graded and warned, and corresponding control measures are set. In view of the low accuracy of defect recognition in industrial product visual monitoring in the prior art, the present application improves the monitoring accuracy.

Description

Technical Field

[0001] The present application relates to the field of industrial monitoring, and in particular to an integrated platform for accurate identification and processing of product visual images. Background Art

[0002] With the continuous development of industrial manufacturing, product quality control has become a key factor in the survival and development of enterprises. The appearance quality of industrial products directly affects their performance and user experience, so visual inspection of products is particularly important. Traditional manual inspection methods are inefficient and subjective, and can no longer meet the needs of modern industrial production. For this reason, more and more companies have begun to introduce machine vision technology and use computer vision algorithms to achieve automated visual inspection of industrial products. However, there are many types of industrial products and the types of defects are complex and diverse, which puts high demands on the generalization ability and adaptability of the detection algorithm.

[0003] At present, there have been many studies and applications on industrial product defect detection. Traditional methods are mainly based on image processing and machine learning technology, which manually extracts image features and then uses classifiers to identify defects. Common features include texture features, shape features, color features, etc., and classifiers use classic algorithms such as support vector machines and random forests. These methods have achieved certain results in specific scenarios, but their generalization ability is limited and it is difficult to cope with complex and changing industrial scenarios. In recent years, with the rise of deep learning technology, convolutional neural networks have achieved widespread success in the field of computer vision. Some scholars have tried to apply deep learning to industrial defect detection, using convolutional neural networks to automatically extract image features and then realize defect classification through fully connected layers. This type of method overcomes the limitations of manual features and has achieved good performance on some data sets. However, most solutions use fixed model architectures and parameters, and lack the ability to adapt to new products, new scenarios, and new defects.

[0004] In the related technology, for example, Chinese patent document CN117908490A provides an industrial defect monitoring system and method based on a cloud native platform, which relates to the field of industrial monitoring technology, and solves the problem that it is difficult to process and detect industrial defects at more monitoring points, and it is also difficult to comprehensively consider the characteristics of the data in the surrounding areas, and perform defect diagnosis on the industrial detection data in this area; including: a regional division module: used to divide the nodes for industrial defect monitoring in the area into several groups according to the positions of the nodes for industrial defect monitoring in the area, and set edge nodes in each industrial defect monitoring node group, and the edge nodes of each group are connected to the cloud native platform server; an industrial defect data collection module: used to obtain historical data when industrial defects occur and when industrial defects do not occur. However, the defects of industrial products may have diverse characteristics, and traditional methods may only extract some features or fail to fully consider the correlation between features. This results in the inability to fully and accurately capture key features when identifying defects, thereby affecting the monitoring accuracy. Summary of the invention

[0005] In response to the problem of low accuracy in defect recognition in visual monitoring of industrial products existing in the prior art, the present application provides an integrated platform for accurate recognition and processing of product visual images, which improves the monitoring accuracy by setting up an abnormality monitoring algorithm, combining attention mechanism and multi-scale feature extraction.

[0006] Technical solution, the purpose of this application is achieved through the following technical solution.

[0007] This manual provides an integrated platform for accurate identification and processing of product visual images, including: The data acquisition module is the data source of the entire system, and mainly collects product image data and equipment parameters through monitoring nodes deployed on the production line. The monitoring node is equipped with two types of sensors: visual sensors and environmental sensors. Among them, the visual sensor is responsible for capturing the image data of the product and obtaining product appearance quality information; the environmental sensor collects environmental parameters, such as temperature, humidity, vibration, etc., to reflect the working condition information of the production line. The visual sensor uses a high-resolution industrial camera and a dedicated light source to meet the requirements of industrial visual inspection for imaging quality. For different products, lenses with different focal lengths and fields of view are selected, and the best shooting angle is obtained by adjusting the camera position and angle. The visual sensor is connected to the edge computing node through industrial Ethernet to transmit the captured image data in real time. Environmental sensors mainly include temperature and humidity sensors, vibration sensors, etc., which are used to collect temperature and humidity data of the workshop environment and vibration data of the equipment respectively. These sensors are connected to the monitoring node through a bus or wireless network, and the collected environmental parameters are uploaded to the monitoring node for summary. The monitoring node preprocesses the collected data, packages the product image data and equipment parameters in a certain format, and uploads them to the edge node and cloud platform respectively through the 5G network.

[0008] The cloud training module is deployed on the server cluster of the cloud platform, and uses the rich computing and storage resources of the cloud to conduct large-scale deep learning model training. The cloud training module reads historical product defect data from the cloud storage, cleans, annotates and enhances the data, and builds a high-quality training data set. In view of various defects that may occur in industrial products, such as cracks, scratches, deformation, stains, etc., the cloud training module designs corresponding convolutional neural network models for each defect type. The model adopts mainstream backbone networks such as ResNet and DenseNet, and fine-tunes the pre-trained model through transfer learning to accelerate the training convergence speed. In order to improve the generalization performance of the model, the cloud training module organizes multiple edge nodes to participate in model training using federated learning. Each edge node uses the data collected locally to train the model locally, and uploads the model parameters to the cloud for aggregation. The cloud aggregates the model parameters of all nodes, updates the global model, and then distributes the updated model to each node to achieve collaborative optimization of the model. This decentralized learning method can achieve continuous improvement of model performance without directly sharing data.

[0009] Specifically, federated learning is a distributed machine learning paradigm that aims to enable multiple participants to collaboratively complete model training without sharing original data. In industrial vision applications, federated learning can help multiple edge nodes train models using local data, and achieve cross-node fusion of models through technologies such as encrypted communication and secure aggregation, thereby obtaining a globally optimal defect detection model. The cloud training module uses federated learning to organize multiple edge nodes to participate in model training, including: the cloud training module initializes a global model in the cloud, namely the convolutional neural network model for defect detection. The model uses mainstream backbone networks such as ResNet and DenseNet, and designs a specific network structure and loss function according to task requirements. After initialization, the cloud distributes the parameters of the global model to all edge nodes participating in the training. Each edge node uses locally collected data to train the global model locally. Specifically, the edge node randomly extracts a batch of samples from the local data set, inputs the samples into the global model for forward propagation, calculates the loss function, and updates the model parameters through backpropagation. This process is repeated for multiple epochs until the model converges on the local data. After local training is completed, each edge node uploads the updated model parameters to the cloud. In order to protect data privacy and model security, the edge node first encrypts the model parameters and converts the model parameters into ciphertext using cryptographic techniques such as homomorphic encryption and multi-party secure computing. In this way, even if the parameters are stolen during network transmission, the attacker cannot directly decrypt and obtain the original parameter values. After receiving the encrypted parameters uploaded by all edge nodes, the cloud performs a secure aggregation operation. The purpose of secure aggregation is to calculate the weighted average of all parameters without decrypting the parameters to obtain a globally updated model. Here, a variant of the Federated Average Algorithm (FedAvg) can be used, that is, the parameters are directly summed in the ciphertext space and then divided by the total number of participating nodes to obtain the encrypted average parameters. The cloud decrypts the aggregated global model parameters to obtain the updated parameters in plaintext. Here, technologies such as differential privacy can be introduced to inject random noise during the decryption process to further protect the privacy of the model. The cloud replaces the original global model with the updated parameters and distributes the new global model to all edge nodes to start a new round of local training. Repeat until the global model converges on the local data of all nodes or reaches the preset number of iterations. In each round of iteration, the edge nodes use the latest global model for local training and securely upload the training results to the cloud for aggregation. After multiple rounds of iterations, the global model can fully utilize the data of all nodes and continuously improve the performance of defect detection.

[0010] The edge training module is deployed on the edge computing node close to the production line, and uses the local computing power and storage resources of the node to achieve real-time online learning. Unlike the large model of the cloud training module, the edge training module mainly runs lightweight machine learning algorithms, such as support vector machines (SVM). The edge training module obtains the equipment parameter data uploaded by the monitoring node through the local storage of the node. Using this data, the edge training module constructs a multi-dimensional feature space and trains the SVM classifier to model the equipment status. When the equipment parameters change abnormally, the SVM model can provide timely warnings and provide a reference for subsequent defect detection. In addition, for unknown defect types not covered by the cloud training module, the edge training module also needs to have anomaly detection capabilities. The edge training module adopts an anomaly detection algorithm based on reconstruction error, and learns the feature representation of normal samples by training the autoencoder network. When the reconstruction error of the input sample exceeds the preset threshold, the sample is judged as abnormal and the alarm mechanism is triggered. The abnormal sample and its feature representation are recorded to update the anomaly detection model and expand the defect type library in the cloud. In order to further improve the accuracy and real-time performance of defect detection, the edge training module also adopts a multi-view learning strategy to fully explore local features. The input product image is divided into multiple overlapping local regions, each of which corresponds to an SVM sub-model. The sub-models extract local features from different perspectives and obtain the final defect prediction result through weighted fusion. The advantage of multi-view learning is that even if some areas are blocked or interfered, effective discriminant information can still be obtained from other areas.

[0011] The cloud recognition module is one of the core components of the entire platform, and is responsible for the precise identification of product defects. The cloud recognition module receives product image data uploaded by the edge node, uses the deep learning model obtained by the cloud training module to locate and classify defects in the image, generates structured recognition results, and returns the results to the edge node for subsequent processing. In order to cope with the complexity of industrial scenarios, the cloud recognition module adopts a multi-task learning framework and designs multiple independent but related sub-networks to jointly complete defect recognition. Among them, the shared feature extraction network is responsible for extracting general features of the image and learning the overall representation of the product. On this basis, for different types of defects, independent recognition networks are set up to perform precise positioning and classification of defects. Each recognition network calculates in parallel, promotes each other, and improves the overall performance through joint optimization. In the shared feature extraction part, the cloud recognition module selects computationally efficient backbone networks such as Mobile Net and ShuffleNet to control the computational overhead while meeting the accuracy requirements. In terms of recognition networks, for defects with relatively fixed scales and shapes, a fully convolutional network is used to accurately locate the defect area; for defects with variable scales and shapes, the receptive field is adaptively adjusted by introducing an attention mechanism and multi-scale feature fusion to capture the key features of the defects.

[0012] The edge detection module is a real-time detection unit deployed on the production line, directly serving quality control and defect warning. The edge detection module receives product image data uploaded by the monitoring node in real time, uses the support vector machine (SVM) model obtained by the edge training module to detect defects, and feeds back the detection results to the edge node for summary analysis. Considering the real-time requirements of the industrial site, the edge detection module adopts a lightweight SVM model to minimize the computational overhead while ensuring detection accuracy. At the same time, in order to further improve the accuracy of defect detection, the edge detection module adopts a cascade fusion strategy to achieve a fine characterization of defects through multi-stage feature fusion. Specifically, the edge detection module first divides the input image into multiple local areas, each corresponding to an SVM sub-model. Each sub-model extracts local features in parallel to obtain preliminary defect prediction results. Then, the edge detection module cascades the output feature maps of each sub-model according to a certain topological structure, and generates a higher-level feature representation through cross-scale and cross-regional feature fusion. The cascaded feature map is input into the SVM classifier again for defect discrimination to obtain a more refined and comprehensive detection result. The design idea of ​​this cascade fusion draws on the concept of pyramid structure and feature pyramid network (FPN) in computer vision. Through bottom-up multi-scale fusion, the edge detection module can take into account both local details and global context, and accurately locate defective areas in complex backgrounds. At the same time, cross-regional information interaction also enhances the robustness of the model to interference factors such as partial occlusion and illumination changes. In the implementation process, cascade fusion can flexibly combine feature maps through operations such as concat and add, and introduce attention mechanisms to adaptively adjust the weights of different regions and scales.

[0013] The control feedback module is the key link between visual inspection and the actuator. It is responsible for generating corresponding control instructions based on the defect detection results, realizing the automatic rejection of defective products and real-time optimization of the production process. The control feedback module receives the output of the edge detection module, that is, the defect type and position coordinates of each product. For the detected defects, the control feedback module first classifies the defects according to the preset rules. The classification rules comprehensively consider factors such as the type, size, quantity, and location of the defects, and classify the defects into different severity levels, such as fatal defects, serious defects, and minor defects. According to the severity of the defects, the control feedback module generates corresponding warning information and control instructions. For fatal defects and serious defects, that is, defects that may cause major quality accidents or customer complaints, the system immediately triggers an alarm and notifies relevant personnel to conduct defect review and cause analysis. At the same time, the location information of the product is sent to the downstream rejection mechanism to automatically remove the defective products from the production line. For minor defects, such as appearance defects that do not affect use, the control feedback module records the defect information and generates quality reports regularly, but does not perform real-time rejection processing. This hierarchical control method not only ensures the effective interception of serious defects, but also avoids excessive intervention in the production process, improving the flexibility and availability of the system. In addition to defect removal, the control feedback module also needs to coordinate the connection between visual inspection and other production links.

[0014] Furthermore, while the edge training module uses the support vector machine (SVM) to train known defect types, it is also necessary to set up an anomaly detection algorithm to discover and identify newly emerging unknown defect types. The core idea of ​​the anomaly detection algorithm is to establish a probability distribution model through historical normal sample data. When the deviation of the newly collected samples from the model exceeds a certain threshold, it is judged as abnormal. The emergence of abnormal samples means a new type of defect, which requires further clustering analysis and labeling. Including: Constructing a normal sample probability model: First, a batch of image samples of normal products are selected from historical data, and multi-dimensional feature vectors of these samples are extracted. Feature vectors can use traditional image features such as SIFT, HOG, etc., or they can use pre-trained convolutional neural networks to extract deep features. Then, these feature vectors obey a multi-dimensional Gaussian distribution, and the mean vector μ and covariance matrix Σ of the Gaussian distribution are obtained through methods such as maximum likelihood estimation, thereby obtaining the probability density function of normal samples. . Calculate sample abnormality: When the edge node collects new product image data, it inputs it into the constructed normal sample probability model to calculate the abnormality value of the image data. The abnormality value reflects the degree of deviation of the new sample from the normal sample distribution, which can be measured by the improved Mahalanobis distance. Abnormality judgment and data storage: According to the pre-set abnormality threshold λ, samples with an abnormality value greater than λ are judged as abnormal defects. The abnormality threshold λ can be set according to historical data and expert experience, or it can be adaptively adjusted through methods such as cross-validation. Defect data judged as abnormal will be transmitted to the abnormal defect data set for subsequent clustering analysis.

[0015] Abnormal defect clustering: When the amount of data in the abnormal defect data set reaches the preset clustering data volume threshold, the clustering analysis process is triggered. Cluster analysis aims to divide the abnormal defect data into several subsets with internal similarities and external differences, each of which represents a potential new defect type. A clustering algorithm based on shared nearest neighbors (SNN) is used here. Generate new defect types: Define each cluster obtained by clustering as a new defect type, and select some representative samples from it for manual labeling and verification. The labeled new defect types can be used to expand the library of known defect types, and iteratively update the SVM classifier to achieve continuous learning and identification of new defects.

[0016] Furthermore, the Improved Mahalanobis Distance introduces mechanisms such as data whitening, feature weighting, and penalty terms on the basis of the traditional Mahalanobis distance to better measure the degree of abnormality of high-dimensional data. Compared with common metrics such as Euclidean distance, the Improved Mahalanobis Distance takes into account the correlation and importance differences between feature dimensions and is more adaptable to data distribution. Specifically, data whitening: The original product image data vector x is subjected to data whitening processing to eliminate the correlation between feature dimensions so that the dimensions of the whitened feature vector are independent of each other and obey a standard normal distribution with a mean of 0 and a variance of 1. Data whitening can be achieved through linear transformation, namely: , where μ is the mean vector of the original data and W is the whitening matrix. W can be obtained by eigenvalue decomposition or singular value decomposition, satisfying: ,here is the covariance matrix of the original data. The covariance matrix of the eigenvector y after whitening is the unit matrix I, indicating that the dimensions have been decorrelated. Feature dimensionality reduction: Based on data whitening, the eigenvector y can be reduced in dimension to reduce computational complexity and noise interference. Common dimensionality reduction methods include principal component analysis (PCA) and linear discriminant analysis (LDA). After dimensionality reduction, a d-dimensional eigenvector is obtained, where d is less than the original feature dimension. The dimensionality reduction process can extract the main pattern of the data and filter out minor details. Calculate feature distance: According to the d-dimensional eigenvector y after data whitening and dimensionality reduction, calculate the weighted Euclidean distance between it and the mean vector ν of the multidimensional Gaussian distribution model. The larger the eigenvalue, the more obvious the data difference in this dimension, and the more important its role in anomaly detection, so the corresponding weight coefficient should also be larger. Introduce penalty term: In order to more comprehensively measure the degree of data anomaly, a penalty term P(x) is introduced on the basis of the feature distance to measure the degree to which the original data vector x deviates from the mean in each dimension. The penalty term can be in the form of L1 norm or L2 norm. The penalty term reflects the degree of abnormality of the data in the original space and can complement the feature distance. Calculate the improved Mahalanobis distance: The feature distance and the penalty term are weighted to obtain the expression of the improved Mahalanobis distance: , where α is the weight coefficient of the penalty term, which is used to balance the proportion between the feature distance and the penalty term. The value of α can be adjusted according to the data characteristics and the needs of anomaly detection. The larger α is, the greater the penalty for the deviation of the original data from the mean, and the higher the sensitivity of anomaly judgment. Anomaly judgment: According to the size of the improved Mahalanobis distance, judge whether the product image data is abnormal. Set an abnormality threshold λ. When MD(x) is greater than λ, x is judged as abnormal data, otherwise it is normal data. The threshold λ can be determined by cross-validation or empirical setting to achieve the best effect of anomaly detection.

[0017] in, , where W is a d×D whitening matrix, μ is the D-dimensional mean vector of the original data set, and y is the d-dimensional feature vector after whitening; ,in is the D-dimensional mean vector of the abnormal data set, ν is the representation of the mean vector of the abnormal data after whitening in the d-dimensional feature space; ,in Represents the weighting coefficient of the i-th feature dimension, which can be set according to the importance and abnormal sensitivity of the feature; ,in represents element-wise multiplication, represents element-level division, σ is the D-dimensional standard deviation vector of the original data set, represents element-level multiplication, represents element-level division, σ is the D-dimensional standard deviation vector of the original data set, and ε is a small positive constant used to avoid the problem of division by zero; It measures the degree to which x deviates from the mean in each dimension in the original data space. As a penalty term, it imposes additional penalties on outliers. α is the penalty term The weight coefficient is used to balance the weight between feature distance and penalty term, which can be adjusted according to the characteristics of the data set and the needs of anomaly detection. Specifically, the feature weight matrix Λ can be adaptively adjusted according to the importance and abnormal sensitivity of different features to improve the accuracy and robustness of anomaly detection; the penalty term Additional penalties can be imposed on abnormal points in the original data space that deviate greatly from the mean to enhance the effect of anomaly detection; the weight coefficient α can flexibly control the balance between feature distance and penalty terms to adapt to the needs of different data sets and anomaly detection tasks.

[0018] Furthermore, Shared Nearest Neighbors (SNN) clustering is a density-based hierarchical clustering method, which is particularly suitable for clustering scenarios of high-dimensional data and non-convex distribution. The core idea of ​​SNN clustering is to transfer the similarity between data points from the original feature space to the nearest neighbor relationship space, and measure their similarity by examining the number of shared nearest neighbors of data points in the local neighborhood, thereby realizing an adaptive clustering process. In the clustering analysis of abnormal defect data, the specific steps of using the SNN clustering algorithm are as follows: Calculate the Mahalanobis distance: First, for each pair of data points in the abnormal defect data set D and , calculate the Mahalanobis distance between them The Mahalanobis distance takes into account the covariance structure of data features and can effectively measure the similarity of high-dimensional data. The calculation formula of the Mahalanobis distance is: , where Σ is the covariance matrix of the data set, Represents the inverse matrix of the covariance matrix Σ of the data set. Determine the ε-neighborhood: According to the preset distance threshold ε, for each data point Determine its ε-neighborhood The ε-neighborhood contains All data points whose distance does not exceed ε. The distance threshold ε can be set according to the characteristics of data distribution and the granularity requirements of clustering. Calculate the local reachable density: For each data point , calculate its local reachable density ,express The number of data points in its ε-neighborhood, that is: , the local reachability density reflects the compactness of data points in the local area. The higher the density, the more concentrated the samples in the area where the data points are located. Calculate the mutual reachability density: For each pair of data points and , calculate the mutual reachability density between them The mutual reachability density takes into account the local density of two data points and the distance between them, and is defined as: The greater the mutual reachability density, the closer the two data points are in local density and distance, and the more likely they are to belong to the same cluster. Construct similarity matrix: Based on the mutual reachability density, construct the similarity matrix S between the data points. Matrix elements Represents data points and The mutual reachability density is: , the similarity matrix reflects the similarity of data points in shared nearest neighbor relationships, and is the basis for subsequent hierarchical clustering. Hierarchical clustering: Based on the similarity matrix S, a hierarchical clustering method is used to cluster data points. Commonly used hierarchical clustering methods include UPGMA, WPGMA, CompleteLinkage, etc.

[0019] Here we take UPGMA as an example. The clustering process is as follows: Initialize each data point as an independent cluster. Find the largest element in the similarity matrix S , the corresponding data points and The clusters where the data points are located are merged into a new cluster. Update the similarity matrix S, and set the similarity between the new cluster and other clusters to the average of the similarities of the original two clusters. Repeat until all data points are classified into the same cluster or the predetermined number of clusters is reached. Clustering result analysis: According to the results of hierarchical clustering, the mutually reachable density greater than the preset threshold is The data points are grouped into the same cluster. Each cluster represents a potential new defect type. The data points within the cluster have high similarity, and the data points between clusters have large differences. By analyzing and interpreting the clustering results, new defect patterns in abnormal defect data can be discovered and identified. The Mahalanobis distance is used to measure the similarity between data points, the ε-neighborhood and local reachability density are used to characterize the local characteristics of data points, and the mutual reachability density is introduced to measure the similarity of data points in shared nearest neighbor relationships. On this basis, the hierarchical clustering method is used to adaptively aggregate data points into different clusters, each cluster corresponding to a potential new defect type. The SNN clustering algorithm can effectively discover hidden patterns in abnormal defect data, providing strong support for the intelligent upgrade of industrial vision systems.

[0020] Furthermore, the cloud recognition module includes a shared feature extraction network, multiple independent recognition networks, and a zero-shot learning model. The shared feature extraction network aims to extract high-level semantic features from the input product image data and provide a universal feature representation for subsequent defect recognition and classification. Here, the Residual Neural Network (ResNet) is used as the infrastructure of the feature extraction network. ResNet effectively solves the problem of difficult deep network training by introducing identity mapping and skip connection, and realizes deeper feature learning. Specifically, the residual block in ResNet can be expressed as: , where x is the input of the residual block, It is a residual function, usually composed of two to three convolutional layers and activation functions. Through the identity mapping, the output y of the residual block is directly added to the input x, so that the network can learn the residual function between the input and output, rather than the complete mapping relationship. This structural design allows the gradient to be directly propagated to the shallow layer, alleviating the problem of gradient disappearance and improving the training efficiency and performance of the network. When building a shared feature extraction network, a pre-trained ResNet model can be used as the basis, such as ResNet-50, ResNet-101, etc. The pre-trained model has been trained on large-scale datasets such as Image Net, and has learned a rich set of general feature representations. In order to adapt to the defect recognition task, the pre-trained model can be fine-tuned (Fine-tuning) to adjust some layer parameters of the network to better adapt to the characteristics of the defect image.

[0021] Through the shared feature extraction network, the input product image data can be mapped into a high-dimensional semantic feature vector as a shared feature of the subsequent recognition network. The use of the shared feature extraction network can significantly reduce the workload of feature engineering and improve the versatility and robustness of feature representation. On the basis of shared features, multiple independent recognition networks are used to specifically identify defects of different types and scales. Each recognition network adopts the structure of a cascaded convolutional neural network. Through multiple serial convolutional sub-networks, defect features are gradually extracted and fused to generate the final defect recognition result. The design idea of ​​the cascaded convolutional neural network is to decompose the complex recognition task into multiple simple sub-tasks, each of which is completed by a convolutional sub-network. The sub-networks are connected in a cascaded manner, and the output of the previous sub-network is used as the input of the next sub-network to achieve hierarchical feature extraction and fusion. Specifically, each convolutional sub-network can include multiple convolutional layers, pooling layers, and activation functions. The convolution layer extracts local features through convolution operations, the pooling layer downsamples the feature map to reduce the size and amount of computation of the features, and the activation function introduces nonlinear transformation to enhance the network's expressiveness. Commonly used convolutional subnetwork structures include LeNet, AlexNet, VGGNet, etc.

[0022] In the defect recognition task, different convolutional subnetworks can be specially designed for defects of different scales and types. For small-scale point defects, smaller convolution kernels and shallower network structures can be used; for large-scale surface defects, larger convolution kernels and deeper network structures can be used. Through multiple specialized subnetworks, defects with different characteristics can be better captured and identified. At the end of the cascaded convolutional neural network, the outputs of multiple subnetworks are combined by feature fusion to generate the final defect recognition result. Common feature fusion methods include feature concatenation and feature weighting. Feature fusion can comprehensively utilize the feature information extracted by different subnetworks to improve the accuracy and robustness of recognition. Multiple independent recognition networks can work in parallel without interfering with each other, which improves the efficiency of defect recognition. At the same time, through the cascade structure and feature fusion, the recognition network can make full use of shared features to identify defects from different angles and different granularities, improving the accuracy and generalization ability of recognition.

[0023] Zero-Shot Learning refers to the task of identifying and classifying new categories that do not appear in the training data. In industrial defect recognition, new defect types often appear due to changes in products and processes, and it is usually difficult to collect and label samples of these new defects. The zero-shot learning model aims to use the existing knowledge of defect categories to identify and classify new defect types. Here, a meta-learning method based on a graph neural network (GNN) is used to achieve zero-shot learning. The goal of meta-learning is to learn a model with strong generalization ability so that it can quickly adapt to new tasks with a small number of samples or no samples. Graph neural network is a deep learning model that processes graph-structured data and can effectively model the associations and similarities between defect categories.

[0024] The construction process of the zero-shot learning model is as follows: Construct a defect category graph: Based on the existing defect category knowledge, a defect category graph is constructed, where nodes represent different defect types and edges represent semantic relationships between defect types, such as similarity, dependency, etc. The defect category graph can be constructed manually based on domain knowledge or automatically learned in a data-driven way. Learn defect category embedding: Use a graph neural network to embed the defect category graph and map each defect type to a low-dimensional vector representation. The graph neural network can learn the interactions and dependencies between nodes through message passing and aggregation operations to generate information-rich defect category embeddings. Meta-learning model training: Based on the defect category embedding, a meta-learning model is trained, such as a prototypical network, a matching network, etc. The meta-learning model builds a general classifier by learning the feature distribution and decision boundary of existing defect categories, which can quickly adapt to new defect categories. Taking the prototype network as an example, its core idea is to learn a prototype vector for each defect category as the representative of the category. The prototype vector is calculated by the average value of the sample features within the class. In the test phase, for a new defect sample, the prototype network classifies the sample into the closest category by calculating the distance between the sample features and the prototype vectors of each category. When training the prototype network, the idea of ​​meta-learning is adopted to simulate the scenario of few-shot learning. Specifically, a small support set (Support Set) and query set (Query Set) are randomly sampled from the existing defect categories. The support set is used to calculate the category prototype vector, and the query set is used to evaluate the generalization performance of the model. Through multiple sampling and training, the prototype network can learn a classifier with strong generalization ability. Zero-shot classification: For a new defect type, its feature representation is first extracted using a shared feature extraction network, and then classified through a meta-learning model. For the prototype network, the features of the new defect sample are compared with the prototype vectors of the existing categories, and the distance between them is calculated. The distance metric can be Euclidean distance, cosine similarity, etc. The new sample is classified into the existing category with the closest distance. At the same time, the prototype vector of the corresponding category is updated with the new sample to gradually adapt to the new defect type. This incremental learning method can continuously expand and improve the existing defect category knowledge base. Model evaluation and iteration: Evaluate the performance of the zero-shot learning model and calculate the recognition accuracy and generalization ability of new defect types. According to the evaluation results, iteratively optimize the model, such as adjusting the network structure, optimizing the loss function, adding data enhancement, etc. Continuously improve the model to enhance its robustness and adaptability in practical applications.

[0025] Furthermore, a meta-learning method based on graph neural network is used to identify and classify newly emerged defect types. First, based on the defect recognition results, the semantic information of different defect types is extracted to construct a defect type relationship graph. The specific steps are as follows: Defect type node: Each known defect type is taken as a node of the graph. The features of the node can include semantic information such as the name, description, and attributes of the defect type. Defect type edge: According to the semantic relationship between defect types, edge connections are established between nodes. Semantic relationships can include synonymous relationships, parent-child relationships, and related relationships. The weight of the edge can indicate the strength of the relationship. Graph construction: Using data structures such as adjacency matrix or adjacency list, the information of nodes and edges is organized into a complete defect type relationship graph. Based on the defect type relationship graph, a graph convolutional neural network (GCN) model is constructed to learn the semantic embedding representation of defect types. The GCN model contains the following key components: Input layer: Receives the defect type relationship graph as input and converts the graph structure data into a form that can be processed by the neural network. The adjacency matrix and node feature matrix are usually used to represent the structure and node information of the graph. Graph convolution layer: extracts and transforms the features of defect type nodes through multiple graph convolution kernels. Graph convolution operation can aggregate the neighborhood information of nodes and update the feature representation of nodes. Commonly used graph convolution kernels include spectral convolution kernel and spatial convolution kernel. Spectral convolution kernel defines convolution operation in Fourier domain based on the Laplacian matrix of graph, which can capture the global structural features of graph signal. Spatial convolution kernel directly defines convolution operation in the spatial domain of graph and updates node features by aggregating the neighborhood information of nodes. Output layer: maps the output of graph convolution layer to semantic embedding representation of defect type through fully connected layer. Semantic embedding is a low-dimensional vector that captures the semantic features of defect type and can be used for subsequent classification tasks. In order to train the GCN model, contrastive learning is adopted to optimize the specific loss function so that the semantic embeddings of defects of the same type are closer in semantic space, while the semantic embeddings of defects of different types are farther away in semantic space. Contrastive learning loss function: Design a contrastive learning loss function to measure the similarity between the semantic embeddings of defect types. Commonly used loss functions include triplet loss, contrastive loss, etc. Taking triplet loss as an example, for each defect type, a sample of the same type is randomly selected as a positive sample, and a sample of a different type is randomly selected as a negative sample to form a triplet.The triplet loss function encourages the semantic embedding distance between positive samples to be smaller than that between positive and negative samples, so that defects of the same type are clustered in the semantic space and defects of different types are separated in the semantic space.

[0026] Model optimization: Use optimization algorithms such as gradient descent to minimize the contrastive learning loss function and update the parameters of the GCN model. Through multiple rounds of iterative training, the model can learn a suitable semantic space so that the semantic embedding distribution of the defect type meets expectations. Semantic space construction: After training, the GCN model is used to generate semantic embedding vectors for all known defect types and construct a semantic space. In the semantic space, each point represents a defect type, and the coordinates of the point are the corresponding semantic embedding vectors. Similar defect types are close in the semantic space, and different defect types are far away in the semantic space. In the constructed semantic space, a prototype representation is constructed for each category using the labeled defect types to classify the newly emerged defect types. Prototype representation calculation: For each known defect type, the average value of the semantic embedding vectors of all samples under the category is calculated as the prototype representation of the category. The prototype representation represents the typical features of the category and can summarize the semantic information of the category. Zero-shot classification: For the newly emerged defect type, it is input into the trained GCN model to obtain the corresponding semantic embedding vector. Then, the cosine similarity between the semantic embedding vector and the prototype representation of each defect type is calculated. Classify the new sample into the defect type with the highest similarity. Cosine similarity measures the angle between two vectors, ranging from -1 to 1. The larger the cosine similarity, the closer the directions of the two vectors are, and the more similar the corresponding defect types are.

[0027] Furthermore, the edge detection module includes multiple parallel support vector machine sub-models and feature fusion units. The edge detection module uses multiple parallel support vector machine (SVM) sub-models to perform multi-view feature extraction on the input product image data to obtain multiple local area features. Each SVM sub-model corresponds to a local area and is responsible for defect detection in the area. Local area division: The input product image data is divided into multiple local areas, and each area corresponds to an SVM sub-model. The division of local areas can adopt methods such as fixed-size sliding windows and region extraction based on semantic segmentation. The local area should cover the entire image and contain as complete defect areas as possible. Feature extraction: Features are extracted from each local area as input to the SVM sub-model. Common feature extraction methods include color histograms, texture features, edge features, etc. Choosing a suitable feature representation is crucial to the performance of defect detection. Gaussian kernel mapping: The Gaussian kernel function is used to map local area features to a high-dimensional space. The Gaussian kernel function is defined as: , where x and y represent two data points or vectors in the original feature space, is the bandwidth parameter of the Gaussian kernel. The Gaussian kernel function can map nonlinearly separable features to high-dimensional space, making the data linearly separable in high-dimensional space and improving classification performance. Soft Margin SVM training: The soft margin SVM algorithm is used to construct the optimal classification hyperplane for defect detection in the region. Soft Margin SVM allows a certain degree of classification error. By introducing slack variables and penalty terms, it minimizes the classification error while maximizing the classification interval. Specifically, the Soft Margin Support Vector Machine is an extension of the Hard Margin SVM for processing linearly inseparable data sets. It allows a certain degree of classification error by introducing slack variables and penalty parameters, and strikes a balance between maximizing the interval and minimizing the classification error. Hard Margin SVM requires that all samples must strictly meet the constraints. Soft Margin SVM introduces slack variables on the basis of Hard Margin SVM, allowing a certain degree of classification error.

[0028] In order to improve the accuracy of defect detection, the edge detection module adopts a cascade fusion strategy. Through multi-stage feature fusion, it comprehensively utilizes the local area features extracted by multiple SVM sub-models and the global features of the original image. The first stage: parallel fusion, the local area features extracted by multiple SVM sub-models are fused in parallel to obtain the global features of the product image. Parallel fusion can use methods such as feature splicing and feature weighting to combine the features of different regions into an overall feature representation. Feature splicing is to connect the feature vectors of each local region end to end to form a long vector. Feature weighting is to assign different weights to each local area feature to generate a weighted fused feature vector. The weight can be determined based on factors such as the importance of the region and the detection confidence. The purpose of parallel fusion is to integrate the local information of different regions and obtain a global description of the entire image. The fused global features contain the overall features of the product image, which is helpful for subsequent defect detection. The second stage: serial fusion, the global features obtained in the first stage are serially fused with the features extracted from the original image through the multi-scale convolutional neural network to obtain the final fused features. The multi-scale convolutional neural network uses convolution kernels of different scales to extract features from the original image and capture the multi-scale information of the image. Through the stacking of multiple convolutional layers and pooling layers, the network can learn the hierarchical feature representation of the image. Serial fusion is to connect the global features and multi-scale convolutional features in a certain order to form the final fusion features. The fusion features combine the local area information extracted by the SVM sub-model and the global multi-scale information extracted by the convolutional neural network, and contain rich image content and defect features. Defect detection: The fusion features are input into the final classifier, such as the softmax layer or SVM, for defect detection. The classifier determines whether there are defects in the image based on the fusion features and gives the detection results. Through the feature fusion of the above two stages, the edge detection module can make full use of the local area features and global multi-scale features to improve the accuracy of defect detection.

[0029] Furthermore, the attention weighting unit and the multi-scale feature extraction unit in the edge detection module, the attention weighting unit uses the gated attention network to weight the local region features, calculates the attention weight of each region feature according to the global features and the local region features, obtains the weighted region features, and obtains the global features of the product image through parallel fusion. Input features: The attention weighting unit receives two parts of input features, one is the global feature and the other is the local region feature. The global feature is obtained by the previous feature fusion unit and contains the overall information of the product image. The local region feature is extracted by multiple SVM sub-models, and each region feature corresponds to a local region. Gated attention network: A multi-layer feedforward neural network is used to construct a gated attention network to calculate the attention weight of each region feature. The input of the network is the concatenation of the global feature and the local region feature, and the output is the attention weight of each region feature. Through the gated attention network, the importance of each region feature can be adaptively adjusted according to the correlation between the global feature and the local region feature. The larger the attention weight, the greater the contribution of the region feature to defect detection. Weighted regional features: Weight the local regional features according to the attention weights to obtain weighted regional features. The weighting operation can be achieved by multiplying the attention weights with the corresponding local regional features. Parallel fusion: All weighted regional features are fused in parallel through element-level addition operations to obtain the global features of the product image.

[0030] The multi-scale feature extraction unit uses a multi-scale convolutional neural network to extract features from the input original product image. By setting convolution kernels of different sizes and pyramid pooling operations, feature maps of different resolutions are obtained, and they are serially fused with global features layer by layer to obtain the final fused features. Multi-scale convolution: Convolution kernels of different sizes are used to perform convolution operations on the original product image to obtain multiple convolution feature maps. Convolution kernels of different sizes can capture features of different scales in the image. For example, small-sized convolution kernels can extract local texture and edge information, and large-sized convolution kernels can extract global structure and context information. Common convolution kernel sizes include 1x1, 3x3, 5x5, 7x7, etc. By combining convolution kernels of different sizes, a multi-scale convolutional neural network can be constructed to extract multi-level and multi-scale image features. Pyramid pooling: The convolution feature map is multi-level down-sampled and up-sampled to obtain feature maps of different resolutions. The pyramid pooling operation can be implemented by maximum pooling or average pooling to scale the feature map to different sizes. The convolution feature map can be downsampled by 1 / 2, 1 / 4, 1 / 8, etc. to obtain feature maps of different resolutions. Then, the downsampled feature map is upsampled accordingly to restore it to the original resolution. Upsampling can be achieved through methods such as bilinear interpolation or transposed convolution. Through pyramid pooling, feature maps of different scales and different receptive fields can be obtained to capture multi-scale information in the image.

[0031] Serial fusion: The feature maps of different resolutions are serially fused with the global features layer by layer to obtain the final fused features. Serial fusion can be achieved through operations such as feature concatenation or element-level addition. First, the global features are fused with the feature map with the highest resolution to obtain the first layer of fused features. Then, the first layer of fused features are fused with the feature map with the second highest resolution to obtain the second layer of fused features. And so on, until the feature map with the lowest resolution is fused to obtain the final fused features. The purpose of serial fusion is to integrate feature maps of different scales with global features layer by layer, so that the fused features contain multi-scale local information and global information. This fusion method can make full use of features at different levels and improve the representation ability of features. Defect detection: The final fused features are input into a classifier, such as a softmax layer or SVM, for defect detection. The classifier determines whether there are defects in the image based on the fused features and gives the detection results. Through the multi-scale feature extraction unit, the edge detection module can extract features of product images from different scales and different levels to capture the performance of defects at different scales. The layer-by-layer fusion of multi-scale features and global features can make full use of local and global information to improve the performance of defect detection.

[0032] Furthermore, the node grouping module is responsible for grouping the monitoring nodes according to their location, function and data characteristics, and setting multiple edge computing nodes in each group to form a distributed edge computing network. The purpose of node grouping is to optimize resource allocation, improve computing efficiency and real-time data processing. Monitoring node grouping: The node grouping module groups the monitoring nodes through a clustering algorithm. The clustering algorithm can group nodes with similar characteristics into a group according to the location, function and data characteristics of the nodes. Commonly used clustering algorithms include K-means, DBSCAN, hierarchical clustering, etc. Location characteristics: The geographical location or relative position of the monitoring node, such as longitude and latitude coordinates, workshop location, etc. Nodes with similar locations can be grouped together to reduce data transmission delays. Functional characteristics: The functional type of the monitoring node, such as image acquisition node, sensor node, control node, etc. Nodes with similar functions can be grouped together to facilitate function management and scheduling. Data characteristics: The data characteristics generated by the monitoring node, including data volume, data type and data generation frequency. Nodes with similar data characteristics can be grouped together to optimize data processing and storage strategies. Edge computing node setting: Multiple edge computing nodes are set in each group to form a distributed edge computing network. The edge computing node is responsible for data collection, preprocessing and real-time analysis of the monitoring nodes in the group. By deploying edge computing nodes in the group, the bandwidth and delay of data transmission can be reduced, and the real-time performance of data processing can be improved. Resource allocation adjustment: The node grouping module dynamically adjusts the resource allocation of edge computing nodes according to the grouping results. According to the data volume, computing tasks and performance requirements of each group, computing resources, storage resources and network resources are reasonably allocated.

[0033] The cloud storage module is responsible for transmitting the collected product image data to the distributed storage system of the cloud platform to achieve persistent storage and centralized management of the data. Distributed storage system: The cloud platform uses distributed storage systems, such as HDFS, Ceph, Gluster FS, etc., to store large-scale product image data. The distributed storage system achieves high scalability, high availability, and high performance by distributing and storing data on multiple storage nodes. Data transmission: The edge computing node transmits the collected product image data to the distributed storage system of the cloud platform through the network.

[0034] The edge storage module is responsible for storing the collected device parameters in the local storage of the corresponding edge computing node to achieve local storage and real-time access to data. Local storage: Each edge computing node is equipped with a local storage device, such as a solid-state drive, memory, etc., for storing the collected device parameters. Local storage provides low-latency, high-bandwidth data access capabilities, suitable for storing data with high real-time requirements. Time series database: The edge storage module uses a time series database for data management. Time series databases are database systems specifically optimized for the storage and query of time series data, such as InfluxDB, Open TSDB, Timescale DB, etc. Time series databases support efficient data writing, aggregate query and real-time analysis, and are suitable for storing time series data such as device parameters. Data management: The edge storage module provides data management functions, such as data storage, data query, data compression, etc. Through reasonable data organization and indexing strategies, device parameters can be quickly retrieved and accessed. At the same time, the edge storage module can also perform data compression and expired data deletion to save storage space. Data synchronization: In order to ensure the integrity and consistency of the data, the edge storage module also needs to synchronize data with the cloud storage module. The locally stored device parameters are regularly synchronized to the distributed storage system of the cloud platform to achieve centralized storage and backup of data.

[0035] Beneficial effects: Compared with the prior art, the advantages of this application are:

[0036] By using historical data on the cloud platform to build multiple deep learning models, corresponding neural network models are trained for different types of industrial defects, and federated learning is used to conduct collaborative model training between multiple edge computing nodes; at the same time, a machine learning model is set up on the edge computing node, and the support vector machine model is trained using the locally stored device parameters. The multi-view learning method is used to divide the product image data into multiple local areas to train the support vector machine sub-models separately, and then the final defect detection results are obtained through weighted fusion. This cloud-edge collaborative model training and reasoning method can make full use of cloud and edge computing resources to improve the accuracy of defect identification and classification.

[0037] By setting an anomaly detection algorithm in the edge training module, using historical normal sample data to establish a multidimensional Gaussian distribution model, and calculating the improved Mahalanobis distance of product image data, when the anomaly value exceeds the preset threshold, the corresponding product image data is judged as abnormal defect data; when the abnormal defect data set reaches the preset clustering data volume threshold, a clustering algorithm based on shared nearest neighbors is used for clustering analysis, and each category obtained by clustering is regarded as a newly emerged defect type. This anomaly detection and clustering analysis method can realize the rapid detection and identification of newly emerged defect types and improve the adaptability and robustness of the system.

[0038] In the cloud recognition module, a multi-task learning framework is adopted to design a shared feature extraction network and multiple independent recognition networks. The shared feature extraction network adopts a residual neural network structure to extract the semantic features of the input product image data through identity mapping and cross-layer connection; the independent recognition network adopts a cascaded convolutional neural network structure to hierarchically recognize shared features through multiple series of convolutional sub-networks, and each sub-network recognizes defect features of different scales and types. In addition, a zero-shot learning model is introduced, and a meta-learning method based on a graph neural network is adopted. According to the defect recognition results, a defect type relationship graph is constructed, a graph convolutional neural network is trained, and a semantic space is generated using a contrastive learning method to classify new defect types. This multi-task learning and meta-learning method can improve the accuracy of defect recognition and achieve rapid classification of new defect types.

[0039] In the edge detection module, multiple parallel support vector machine sub-models are designed to extract multi-view features from the input product image data to obtain multiple local area features. A cascade fusion strategy is adopted to obtain global features by fusing local area features in parallel, and then the global features are serially fused with the features extracted from the original image by a multi-scale convolutional neural network to obtain the final fused features. At the same time, an attention weighting unit and a multi-scale feature extraction unit are introduced to calculate the attention weights of regional features through a gated attention network, and multi-scale convolutional neural networks and pyramid pooling operations are used to extract multi-scale features, which are serially fused with global features layer by layer. This parallel sub-model design and cascade fusion strategy, combined with attention mechanism and multi-scale feature extraction, can improve the accuracy and real-time performance of defect detection.

[0040] By introducing the node grouping module and distributed storage mechanism, the monitoring nodes are grouped according to their location, function and data characteristics, and multiple edge computing nodes are set in each group to form a distributed edge computing network; the clustering algorithm is used to group the monitoring nodes, and the resource allocation of the edge computing nodes is adjusted according to the grouping results. At the same time, the collected product image data is transmitted to the distributed storage system of the cloud platform, and a multi-copy fault tolerance mechanism is adopted; the collected device parameters are stored in the local storage of the corresponding edge computing node, and the time series database is used for data management. This node grouping and distributed storage design can optimize the topological structure and resource allocation of the edge computing network and improve the scalability and fault tolerance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0042] Figure 1 is an exemplary module diagram of an integrated platform for accurate product visual image recognition and processing according to some embodiments of this specification;

[0043] Figure 2 is an exemplary flow chart for obtaining a new defect type according to some embodiments of this specification;

[0044] Figure 3 is an exemplary flow chart for classifying newly emerged defect types according to some embodiments of the present specification;

[0045] Figure 4 This is an exemplary flowchart for obtaining the final fusion features according to some embodiments of this specification. DETAILED DESCRIPTION

[0046] The method and system provided in the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0047] Figure 1 is an exemplary module diagram of an integrated platform for accurate product visual image recognition and processing according to some embodiments of this specification, such as Figure 1 As shown, the integrated platform provided by this embodiment includes a data acquisition module, a cloud training module, an edge training module, a cloud recognition module, an edge detection module and a control feedback module. Among them, the data acquisition module collects product image data and equipment parameters through the monitoring nodes set on the production line. The monitoring node is equipped with a visual sensor and an environmental sensor. The visual sensor collects product image data, and the environmental sensor collects environmental parameters. The collected product image data is transmitted to the edge computing node via industrial Ethernet for local processing, and is uploaded to the cloud platform via the 5G network for remote analysis. The collected equipment parameters are directly stored in the local storage of the edge computing node.

[0048] Node grouping module: A smart factory deploys a large number of equipment monitoring nodes, including image acquisition nodes, temperature sensor nodes, vibration sensor nodes, and control nodes. These monitoring nodes are distributed in different workshops and locations of the factory to collect the operation data and status information of the equipment. The node grouping module needs to group the monitoring nodes according to their location, function, and data characteristics, and set edge computing nodes in each group to form a distributed edge computing network. Collect the location information of the monitoring nodes, such as the workshop to which they belong, longitude and latitude coordinates, etc. Mark the functional type of the monitoring nodes, such as image acquisition, temperature monitoring, vibration monitoring, equipment control, etc. Statistic the data characteristics of the monitoring nodes, such as data volume, data type, data generation frequency, etc. Through K-means clustering, the monitoring nodes are divided into 3 groups: Group 1: Located in workshop A, mainly image acquisition, large data volume, and high data generation frequency. Group 2: Located in workshop B, mainly temperature and vibration monitoring, medium data volume, and medium data generation frequency. Group 3: Dispersed in various workshops, mainly equipment control, small data volume, and low data generation frequency. Group 1: Set up 5 edge computing nodes, each equipped with a GPU accelerator and large-capacity memory for real-time processing of image data. Group 2: Set up 3 edge computing nodes, each equipped with a medium-specification CPU and memory for real-time analysis of temperature and vibration data. Group 3: Set up 2 edge computing nodes, each equipped with a low-power CPU and small-capacity memory for issuing device control instructions. When the image data processing tasks of group 1 increase, dynamically increase the GPU resources and memory capacity of the edge computing nodes. When the temperature and vibration data analysis tasks of group 2 decrease, dynamically reduce the number of CPU cores and memory capacity of the edge computing nodes. When the device control tasks of group 3 are less, the resources of some edge computing nodes can be allocated to other groups.

[0049] On the cloud platform side, cloud storage module: The edge node uploads the collected product image data to the RESTful API interface of the cloud platform via the HTTPS protocol in the form of a POST request. The cloud storage module receives the uploaded image files and stores them in the HDFS (Hadoop Distributed File System) cluster. Each image file is replicated into 3 copies and stored on 3 different data nodes in the HDFS cluster to ensure data reliability and availability. The cloud storage module manages the image data by date and product type, " / data / images / 2023 / 05 / 09product_A". For historical image data older than 90 days, the cloud storage module automatically archives it to low-cost cold storage, such as Amazon Glacier. For historical image data older than 180 days, the cloud storage module automatically deletes it from the HDFS cluster to save storage space.

[0050] Cloud training module: The cloud training module reads historical product image data from the HDFS cluster, and reads 10,000 images each time for batch processing. For crack defect detection, the pre-trained ResNet-50 model is selected as the base model. This model achieves a Top-5 accuracy of 95.51% on the ImageNet dataset. The ResNet-50 model is fine-tuned and trained on a dataset containing 5,000 crack images, with a learning rate of 0.001, a batch size of 32, and training for 50 epochs. For scratch defect detection, the pre-trained Inception-V3 model is selected as the base model. This model achieves a Top-5 accuracy of 95.23% on the ImageNet dataset. The Inception-V3 model is fine-tuned and trained on a dataset containing 8,000 scratch images, with a learning rate of 0.0005, a batch size of 64, and training for 30 epochs.

[0051] The federated learning method is used to train the model in collaboration with 10 edge computing nodes, each of which has 500 local product image data. In each training iteration, the edge node uses local data to train the model and uploads the model parameters to the cloud platform. The uploaded parameter size is about 100MB. The cloud training module uses a secure aggregation algorithm to encrypt and aggregate the model parameters uploaded by the 10 edge nodes to obtain the global model parameters. During the encryption aggregation process, the Paillier homomorphic encryption algorithm is used to encrypt the model parameters to protect data privacy. The aggregated parameter size is about 120MB. The updated global model parameters are sent to the 10 edge nodes, and each node receives and updates the local model for subsequent defect detection tasks. The federated learning process is repeated 20 times, each iteration takes about 30 minutes, and the total training time is about 10 hours.

[0052] The edge computing node is responsible for collecting device parameter data and using this data to train defect detection models and detect abnormal defects. Edge storage module: Stores the collected device parameter data in a local time series database, such as InfluxDB or Open TSDB. Time series databases are specifically used to store time series data and support high concurrent writing and fast query. Each device parameter data includes fields such as timestamp, device ID, parameter name, and parameter value. The edge storage module also provides data management and query interfaces to facilitate other modules to read and analyze device parameter data.

[0053] Figure 2This is an exemplary flow chart for obtaining newly emerged defect types according to some embodiments of this specification. The edge training module has trained a support vector machine (SVM) model using device parameter data to detect known types of defects. An anomaly detection algorithm is set up to detect newly emerged unknown defect types. Using historical normal sample data, a multidimensional Gaussian distribution model is established to obtain the characteristic distribution of normal samples. Normal sample data obeys a multidimensional Gaussian distribution, and the model parameters are estimated by the maximum likelihood estimation method. The mean vector μ and covariance matrix Σ of the normal sample are obtained, and the probability distribution model N (μ, Σ) of the normal sample is constructed.

[0054] The product image data is a high-resolution color image. Each image has a size of 1024×1024 pixels and contains three RGB color channels. Preprocess the image data, extract features, calculate the abnormality value, and determine whether it is an abnormal defect. Expand the three RGB color channels of the image in row-first order to obtain a 3×1024×1024=3,145,728-dimensional vector x. Each element of the vector x represents the RGB color value of the pixel at the corresponding position of the image, ranging from [0, 255]. Perform data normalization on the vector x and scale the value range of each feature dimension to the interval [0, 1]. Perform data whitening and dimensionality reduction on the normalized data vector x_norm to obtain a d-dimensional feature vector y. First, calculate the covariance matrix Σ of x_norm, whose size is 3,145,728×3,145,728. Perform feature decomposition on Σ: , where U is the eigenvector matrix and Λ is the eigenvalue diagonal matrix. According to the size of the eigenvalue, select the eigenvectors corresponding to the first d largest eigenvalues ​​to form a dimensionality reduction matrix , whose size is 3,145,728×d. Calculate the transformation matrix for data whitening , where Λ_d is the diagonal matrix consisting of the first d largest eigenvalues. Transformed into a d-dimensional feature vector , the covariance matrix of y is the unit matrix. The reduced eigenvector y is input into the pre-built multidimensional Gaussian distribution model to calculate the Mahalanobis distance between it and the normal sample mean vector ν. Using the calculation formula of the Mahalanobis distance, finally, the abnormality value MD(x) of the product image data x is calculated by the improved Mahalanobis distance formula, and the abnormality threshold θ is set. When the abnormality value MD(x) of the product image data x is greater than θ, x is judged to be an abnormal defect. The abnormality threshold θ can be determined by statistical analysis of the abnormality values ​​of historical normal sample data, for example, selecting the 95% quantile of the abnormality value of the normal sample as the threshold. When MD(x) is greater than θ, it means that the image data x deviates far from the distribution of normal samples and is likely to contain defective areas, requiring further defect location and classification.

[0055] For each pair of data points in the abnormal defect dataset D and , calculate the Mahalanobis distance between them , the calculation results are stored in a symmetric Mahalanobis distance matrix M, where M(i, j) represents the data point The Mahalanobis distance between the abnormal defect dataset D contains 10 data points , each data point is a 5-dimensional vector. We have calculated the Mahalanobis distance matrix M between these 10 data points. Setting the distance threshold ε to 2.5 means that when the Mahalanobis distance between two data points is less than or equal to 2.5, they belong to each other's ε-neighborhood.

[0056] For each data point x_i, according to the Mahalanobis distance matrix M, find all data points whose distance is less than or equal to ε, forming The ε-neighborhood For data points , its ε-neighborhood is: , which represents the data point , and and The Mahalanobis distance is less than or equal to 2.5. The density threshold is set to 3, which means that the data points with more than or equal to 3 data points in the ε-neighborhood are marked as core objects. , count its ε-neighborhood If the number of data points is greater than or equal to the density threshold, Marked as a core object. For data points , its ε-neighborhood Contains 4 data points, which is greater than the density threshold of 3, so Marked as a core object. By calculation, we get 5 core objects For each core object , calculate its local reachable density ,Right now The number of data points in the ε-neighborhood of , its local reachable density is: The calculated local reachable density is: For each pair of core objects , calculate the mutual reachability density between them The mutual reachability density is defined as and The minimum of the local reachability density and the inverse of the Mahalanobis distance between them. and , the Mahalanobis distance between them is , then their mutual reachability density is: For all core object pairs, their mutual reachability density is calculated to form a 5×5 mutual reachability density matrix Λ. The calculated mutual reachability density matrix Λ is:

[0057] .

[0058] Figure 3 This is an exemplary flow chart for classifying newly emerged defect types according to some embodiments of this specification. Hierarchical clustering: Set the density threshold λ_min to 0.60, indicating that core objects with a mutual reachability density greater than 0.60 will be classified into the same class. Initialize the clustering result C, and treat each core object as an independent class: {c_1, c_3, c_4, c_6, c_9}. Find the maximum value λ_max=0.71 in the mutual reachability density matrix Λ, and the corresponding two classes are c_3 and c_4. Since λ_max>λ_min, merge classes c_3 and c_4, and update the clustering result C to: {c_1, c_34, c_6, c_9}, where c_34 represents the new class after the merger. Update the mutual reachability density matrix Λ, and after merging c_3 and c_4 into c_34, Λ becomes: Repeat the process of finding the maximum mutually reachable density and merging classes until no more merging is possible. The final clustering result C is: {c_134, c_69}, where c_134 represents the merged class, including core objects x_1, x_3, and x_4; c_69 represents the merged class, including core objects x_6 and x_9.

[0059] Classify non-core objects: There are 3 non-core objects {x_2, x_5, x_7} in the dataset D. For each non-core object, find the minimum distance between it and all core objects in the Mahalanobis distance matrix M, and classify it into the corresponding class. For the non-core object x_2, its Mahalanobis distance with the core object is: {d(x_2, x_1) = 1.5, d(x_2, x_3) = 2.0, d(x_2, x_4) = 1.8, d(x_2, x_6) = 2.5, d(x_2, x_9) = 2.2}. The minimum distance is d(x_2, x_1) = 1.5, so x_2 is classified into class c_134 to which x_1 belongs. Similarly, x_5 is classified into c_134 and x_7 is classified into c_69. Finally, the complete clustering results are obtained: c_134={x_1, x_3, x_4, x_2, x_5}, c_69={x_6, x_9, x_7}.

[0060] Update the SVM model: Take the clustered classes c_134 and c_69 as new defect types and extract their feature vectors respectively. The original SVM model has been trained and can recognize some known defect types. Use the new defect type data c_134 and c_69 to incrementally learn and update the SVM model. You can add the new defect type data to the original training set, retrain the SVM model, and optimize its parameters. You can also use online learning, that is, only use a part of the new defect type data each time to update the SVM model and gradually expand its recognition ability. Through incremental learning, the SVM model can continuously adapt to new defect types and improve its generalization ability and practical application effect.

[0061] During detection, the edge detection module receives product image data from the data acquisition module in real time, and uses the support vector machine model trained by the edge training module to perform defect detection. The edge detection module uses a multi-view learning method to divide the product image into multiple local areas, each of which corresponds to a support vector machine sub-model. Multiple sub-models extract local area features in parallel, and perform feature fusion through the attention weighting unit and the feature fusion unit to obtain the defect detection results of the product image. The detection results are first fed back to the control module, and the control module performs graded warning and disposal of defects according to preset rules. At the same time, the edge detection module also uploads the product image data to the cloud platform for secondary recognition. The cloud recognition module adopts a multi-task learning framework, uses a shared feature extraction network to extract image semantic features, and then uses an independent recognition network to finely identify different types of defects. For suspected new defects found by the edge detection module, the cloud recognition module uses a zero-sample learning model for classification. The zero-shot learning model first constructs a defect type relationship graph based on the output results of multiple independent recognition networks, and learns the semantic embedding representation of defect types through a graph convolutional neural network; then, a small number of labeled defect samples are used to construct defect prototype vectors in the semantic space; finally, the semantic embedding vectors of new samples are matched with each prototype vector for similarity to achieve the classification of new defects. The cloud recognition results are used as a supplement and verification of the edge detection results, and are fed back to the control module for final defect confirmation and processing decisions.

[0062] Figure 4This is an exemplary flow chart for obtaining the final fusion features according to some embodiments of this specification, cloud recognition module, shared feature extraction network: a residual neural network (ResNet) structure is used to build a shared feature extraction network. ResNet contains 50 convolutional layers, which are divided into 5 stages, and each stage contains multiple residual blocks. Each residual block consists of two convolutional layers, using 3x3 convolution kernels, and the number of convolution kernels is 64, 128, 256, and 512 respectively. In the first residual block of each stage, downsampling is performed through a 1x1 convolution layer to halve the size of the feature map. In the residual block, the input is directly passed to the output through an identity mapping and added to the output of the convolution layer to achieve cross-layer connection. For the input 224x224x3 product image data, 7x7x2048 shared features are extracted by ResNet.

[0063] Multiple independent recognition networks: 3 independent cascade convolutional neural networks are constructed to perform defect recognition on shared features respectively. Each cascade network contains 4 convolutional subnetworks, each of which consists of 3 convolutional layers and 1 pooling layer. The convolution kernel size of the first subnetwork is 5x5, and the number of convolution kernels is 128; the convolution kernel size of the remaining subnetworks is 3x3, and the number of convolution kernels is 256, 512, and 1024 respectively. The pooling layer uses maximum pooling with a pooling size of 2x2 and a step size of 2 to reduce the size of the feature map. After each subnetwork, the feature map is flattened through a fully connected layer, and the corresponding defect recognition result is output. The output of the three cascade networks is feature fused to generate the final defect recognition result, including the defect location, defect type, and defect severity.

[0064] Zero-shot learning model: Based on the defect recognition results, the semantic information of different defect types is extracted and a defect type relationship graph is constructed. There are 10 known defect types, and the semantic information of each defect type is represented by a 300-dimensional word vector. According to the semantic similarity between defect types, a 10x10 adjacency matrix is ​​constructed to represent the connection relationship between defect type nodes. A three-layer graph convolutional neural network (GCN) is constructed. The input layer receives 300-dimensional semantic features of defect types. The number of convolution kernels in the graph convolution layer is 64 and 128 respectively, and the output layer reduces the feature dimension to 128. The GCN is trained by contrastive learning. Each batch contains 5 positive sample pairs and 10 negative sample pairs. The positive sample pairs are composed of semantic embeddings of the same defect type, and the negative sample pairs are composed of semantic embeddings of different defect types. A 128-dimensional semantic space is learned by minimizing the semantic embedding distance of the positive sample pairs and maximizing the semantic embedding distance of the negative sample pairs. Using the labeled defect types, the prototype representation of each defect type is calculated in the semantic space as the center point of the category. For the newly emerged defect types, their semantic information is input into the trained GCN to obtain the corresponding 128-dimensional semantic embedding vector. The cosine similarity between the semantic embedding vector and the prototype representation of each defect type is calculated, and the defect type with the highest similarity is selected as the classification result. The cosine similarity between the newly emerged defect type and the existing third defect type is 0.85, which is higher than other categories, so it is classified as the third defect type.

[0065] The edge detection module includes: multiple parallel support vector machine sub-models: the input 1024x1024 product image data is divided into 16 256x256 local areas. For each local area, a support vector machine (SVM) sub-model is constructed, with a total of 16 parallel SVM sub-models. Each SVM sub-model uses a Gaussian kernel function to map the 256x256 local area features to a 1024-dimensional high-dimensional space. In the high-dimensional space, the soft margin SVM algorithm is used to construct the optimal classification hyperplane by setting the relaxation variables and penalty factors. The penalty factor C of the soft margin SVM is 1.0, and the upper bound of the relaxation variable ξ is 0.1, then the defect detection results of 16 local areas can be obtained.

[0066] Feature fusion unit: Phase 1: The 1024-dimensional local region features extracted by 16 SVM sub-models are fused in parallel, and an element-level average operation is used to obtain a 1024-dimensional global feature. Phase 2: The 1024-dimensional global feature is serially fused with the features extracted by the multi-scale convolutional neural network from the original image. The multi-scale convolutional neural network contains 4 convolutional layers and 2 pooling layers, with convolution kernel sizes of 7x7, 5x5, 3x3, and 3x3, and the number of convolution kernels is 64, 128, 256, and 512, respectively. The pooling layer uses maximum pooling with a pooling size of 2x2 and a step size of 2 to reduce the size of the feature map. The 512-dimensional features extracted by the multi-scale convolutional neural network are serially fused with the 1024-dimensional global features, and an element-level concatenation operation is used to obtain a 1536-dimensional final fused feature.

[0067] Attention weighting unit: A gated attention network is used to calculate the attention weights of 16 local region features. The gated attention network contains three feedforward neural network layers, with the number of neurons in each layer being 512, 256, and 16, respectively. The first layer receives 1024-dimensional global features and 16 1024-dimensional local region features, and reduces the feature dimension to 512 through a fully connected layer. The second and third layers use the ReLU activation function and the Sigmoid activation function, respectively, to reduce the feature dimension to 256 and 16. The output of the third layer is normalized by the Softmax function to obtain the attention weights of the 16 local region features, ranging from 0 to 1. The 16 1024-dimensional local region features are multiplied by the corresponding attention weights to obtain the weighted region features. The 16 weighted region features are fused in parallel through element-wise addition operations to obtain a 1024-dimensional global feature.

[0068] Multi-scale feature extraction unit: A multi-scale convolutional neural network is used to extract features from the original product image of 1024x1024. The multi-scale convolutional neural network contains 3 convolutional layers and 3 pooling layers, with convolution kernel sizes of 7x7, 5x5, and 3x3, and the number of convolution kernels is 64, 128, and 256, respectively. The pooling layer uses maximum pooling with a pooling size of 2x2 and a step size of 2 to reduce the size of the feature map. Pyramid pooling is performed on the convolution feature map. By setting different pooling sizes and step sizes, four feature maps of different resolutions are obtained, with sizes of 256x256, 128x128, 64x64, and 32x32, respectively. The four feature maps of different resolutions are serially fused layer by layer with the 1024-dimensional global feature, and an element-level splicing operation is used to obtain a final fused feature of 1792 dimensions.

Claims

1. An integrated platform for accurate product visual image recognition and processing, including: The data acquisition module collects product image data and equipment parameters through monitoring nodes; the monitoring nodes are equipped with visual sensors and environmental sensors, the visual sensors collect product image data, and the environmental sensors collect environmental parameters; The cloud training module uses historical data to build multiple deep learning models on the cloud platform server, and trains corresponding neural network models for different types of industrial defects. The cloud training module uses federated learning to conduct collaborative model training between multiple edge computing nodes. The edge training module sets up the machine learning model on the edge computing node and trains the support vector machine model using the device parameters stored locally on the edge computing node; And set up anomaly detection algorithms to detect new types of defects; The edge training module uses a multi-view learning method to divide the product image data into multiple local areas, train the support vector machine sub-models separately, and then obtain the final defect detection results through weighted fusion; The cloud recognition module uses the deep learning model trained by the cloud training module to identify defects in product image data and returns the identification results to the corresponding edge computing node. The cloud recognition module adopts a multi-task learning framework. On the basis of a shared feature extraction network, it sets up independent recognition networks for different types of defects to improve the accuracy of defect recognition. The edge detection module uses the support vector machine model trained by the edge training module to perform defect detection on product image data. The edge detection module adopts a cascade fusion strategy to improve the accuracy of defect detection through multi-stage feature fusion. The control feedback module, based on the detection results of the edge detection module, provides graded warnings for defects according to preset rules and sets corresponding control measures; Cloud identification module, including: The shared feature extraction network adopts a residual neural network structure, and constructs a feature extraction network through identity mapping and cross-layer connection. The constructed feature extraction network is used to extract the semantic features of the input product image data as shared features. Multiple independent recognition networks use a cascaded convolutional neural network structure to perform hierarchical recognition of shared features through multiple serially connected convolutional sub-networks. Each sub-network recognizes defect features of different scales and types, and generates defect recognition results of multiple sub-networks through feature fusion. Zero-shot learning model, based on the defect recognition results, uses a meta-learning method based on graph neural networks to identify and classify new defect types; Edge detection module, including: Multiple parallel support vector machine sub-models perform multi-view feature extraction on the input product image data to obtain multiple local area features. Each support vector machine sub-model corresponds to a local area, and the corresponding local area features are mapped to a high-dimensional space through a Gaussian kernel function. The soft margin support vector machine algorithm is used to construct the optimal classification hyperplane to detect defects in the area. The feature fusion unit adopts a cascade fusion strategy to improve the accuracy of defect detection through multi-stage feature fusion. In the first stage, the local area features extracted by multiple support vector machine sub-models are fused in parallel to obtain the global features of the product image. In the second stage, the global features are fused in series with the features extracted from the original image by a multi-scale convolutional neural network to obtain the final fused features.

2. The integrated platform for accurate product visual image recognition and processing according to claim 1, characterized in that: Edge training module, sets up anomaly detection algorithms to detect emerging defect types, including: Using historical normal sample data, a multidimensional Gaussian distribution model is established to obtain the characteristic distribution of normal samples and construct a probability distribution model of normal samples; The collected product image data is input into the constructed probability distribution model of normal samples, and the abnormality value of the product image data is obtained by calculating the improved Mahalanobis distance of the product image data; When the abnormality value of the product image exceeds a preset abnormality threshold, the corresponding product image data is judged as abnormal defect data, and the abnormal defect data is transmitted to the abnormal defect data set; When the data volume in the abnormal defect data set reaches the preset clustering data volume threshold, a clustering algorithm based on shared nearest neighbors is used to perform clustering analysis on the abnormal defect data; Each category obtained by clustering is regarded as a new defect type.

3. The integrated platform for accurate product visual image recognition and processing according to claim 2, characterized in that: The improved Mahalanobis distance of the product image data is calculated using the following formula: ； in, represents the improved Mahalanobis distance value of the product image data vector x; y is the d-dimensional feature vector obtained after x is processed by data whitening and dimensionality reduction. Data whitening is to linearly transform the original data vector x into a feature-independent vector y; is the mean vector of the multidimensional Gaussian distribution model in the d-dimensional feature space; It is a d×d diagonal matrix, representing the weighting coefficients of different feature dimensions; is the penalty term of x, which measures the degree to which x deviates from the mean in each dimension in the original data space; is the weight coefficient of the penalty term, which is used to balance the proportion between feature distance and penalty term.

4. The integrated platform for accurate product visual image recognition and processing according to claim 3, characterized in that: The clustering algorithm based on shared nearest neighbors is used to perform cluster analysis on abnormal defect data, including: Calculate the Mahalanobis distance between abnormal defect data points and use the preset distance threshold Determine each data point of -Neighborhood , -Neighborhood The data points in the -Neighborhood Defined as: ； Where D represents the abnormal defect dataset, Represents data points and The Mahalanobis distance between Count each core object The local reachable density , the local reachable density is defined as: ； in, Represents the core object of -The number of data points in the neighborhood; Calculate the mutual reachability density between core objects , two core objects and Mutual reachability density is defined as: ； According to the mutual reachability density , a hierarchical clustering method is used to cluster the core objects, and the objects with mutual reachability density greater than the preset density threshold are clustered. The core objects are classified into the same category, and each category obtained by clustering is regarded as a new defect type.

5. The integrated platform for accurate product visual image recognition and processing according to claim 4, characterized in that: A hierarchical clustering method is used to cluster the core objects, including: Set each core object as an independent class and create a category set C; Get mutual reachability density The two largest categories and ,if Greater than , then merge and , and update the category set C; Repeat the merging step until the mutual reachability density between all categories is less than or equal to , get the final core object clustering result; For each non-core object ,calculate The Mahalanobis distance to all core objects will be Divide into the class to which the closest core object belongs The division rules are: ； Among them, C represents the category set obtained by clustering core objects.

6. The integrated platform for accurate product visual image recognition and processing according to claim 1, characterized in that: Use a graph neural network-based meta-learning approach to identify and classify emerging defect types, including: According to the defect recognition results, the semantic information of different defect types is extracted, and a defect type relationship graph is constructed. The defect types are used as nodes of the graph, and the semantic relationships between defect types are used as edges of the graph. Construct a graph convolutional neural network, which includes an input layer, a graph convolution layer, and an output layer. The input layer receives a defect type relationship graph. The graph convolution layer extracts the features of defect type nodes through multiple graph convolution kernels. The output layer maps the output of the graph convolution layer to a semantic embedding representation of the defect type through a fully connected layer. The graph convolutional neural network is trained by contrastive learning to generate a semantic space by minimizing the semantic embedding distance between defect types of the same type and maximizing the semantic embedding distance between defect types of different types. Using the annotated defect types, a prototype representation of the defect type is constructed in the semantic space. The prototype representation of each defect type is calculated by the average value of the semantic embedding vectors of all defect types in the corresponding category. The newly emerged defect types are input into the trained graph convolutional neural network to obtain the corresponding semantic embedding vector; by calculating the cosine similarity between the semantic embedding vector and the prototype representation of each defect type, the corresponding defect type is classified into the defect type with the highest similarity.

7. The integrated platform for accurate product visual image recognition and processing according to claim 1, characterized in that: The edge detection module also includes: The attention weighting unit uses a gated attention network to calculate the attention weight of each regional feature through a multi-layer feedforward neural network based on the global features and local regional features; the weighted regional features are obtained according to the attention weights; all weighted regional features are fused in parallel through element-level addition operations to obtain the global features of the product image; The multi-scale feature extraction unit adopts a multi-scale convolutional neural network. By setting convolution kernels of different sizes, it performs convolution operations on the input original product image to obtain multiple convolution feature maps. It adopts pyramid pooling operations to perform multi-level downsampling and upsampling on the convolution feature map to obtain feature maps of different resolutions. The feature maps of different resolutions are serially fused with the global features layer by layer to obtain the final fused features.

8. The integrated platform for accurate product visual image recognition and processing according to any one of claims 1 to 7, characterized in that: Also includes: The node grouping module groups the monitoring nodes according to their location, function and data characteristics, and sets multiple edge computing nodes in each group to form a distributed edge computing network. The node grouping module groups the monitoring nodes through a clustering algorithm and adjusts the resource allocation of the edge computing nodes according to the grouping results. The data characteristics include data volume, data type and data generation frequency. The cloud storage module transmits the collected product image data to the distributed storage system of the cloud platform; the distributed storage system adopts a multi-copy fault-tolerant mechanism to improve data availability; The edge storage module stores the collected equipment parameters in the local storage of the corresponding edge computing node; the edge storage module uses a time series database to manage data; among them, the equipment parameters include equipment operation parameters, process parameters and environmental parameters.

Images