Workpiece surface quality prediction method and system
Through multimodal data fusion and deep learning technology, combined with phase shift interference, thermal imaging and ultrasonic detection, a high-precision workpiece surface quality prediction model is generated, solving the problems of single attribute dependence and high computational complexity in the existing technology, and realizing the Accurate and real-time prediction of workpiece surface quality.
Patent Information
- Application Number
- CN202411097645.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-22
AI Technical Summary
The existing workpiece surface quality prediction methods rely on single attribute data, ignore surface roughness and heat distribution characteristics, have high computational complexity, and fail to make full use of multiple data sources, resulting in inaccurate and incomplete predictions.
Multimodal data is obtained by phase shift interference, thermal imaging analysis and ultrasonic phased array detection, combined with multimodal variational autoencoder and graph neural network for joint feature learning and deep learning, and generate high-precision predictions through reinforcement learning optimization The model and the processing parameters are adjusted in real time through distributed sensing network.
It significantly improves the accuracy and real-time performance of workpiece surface quality prediction, comprehensively captures workpiece characteristics, solves the problems of single attribute grouping and high computational complexity, and realizes accurate prediction of workpiece surface quality.
Smart Images

Figure CN119006965A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent manufacturing and advanced detection technology, and in particular to a method and system for predicting the surface quality of a workpiece. Background Art
[0002] In the manufacturing industry, the surface quality of workpieces directly affects the performance, life and reliability of products. High-quality surfaces can significantly improve the wear and corrosion resistance of workpieces, while reducing the occurrence of fatigue cracks. Accurate surface quality prediction helps to adjust processing parameters in a timely manner, improve the stability and efficiency of the processing process, and ensure that the final product meets the design requirements.
[0003] The current workpiece quality often faces the following problems: uneven surface roughness, thermal damage, internal defects, etc. These problems are mainly caused by unstable factors in the processing process, equipment wear, improper process parameter settings, etc. For example, too high a cutting speed may lead to increased surface roughness, insufficient cooling may cause thermal damage, and failure to evenly release the internal stress of the material may lead to internal defects.
[0004] The existing technology (Chinese invention patent, publication number: CN118247269A, title: A method, device and computing equipment for predicting workpiece surface quality) has the following defects in dealing with the above problems:
[0005] Relying on the first attribute of the sample image (such as microhardness value) for grouping, other important features such as surface roughness and thermal distribution are ignored, resulting in inaccuracy of the grouping model;
[0006] Divide the sample image into multiple subdivisions and divide the states according to the second attribute data. This process is complicated and easily interfered by noise, resulting in inaccurate state division.
[0007] The Markov transition model parameters are calculated through the state transition probability matrix, which has high computational complexity and assumes that the state transition depends only on the current state, ignoring the potential impact of historical data on future states;
[0008] Mainly relying on image data and failing to make full use of other forms of data (such as thermal imaging and ultrasonic signals) leads to incomplete and inaccurate prediction of workpiece surface quality. Summary of the invention
[0009] In view of the many problems existing in the above-mentioned prior art, the present invention provides a method and system for predicting the surface quality of a workpiece. The present invention introduces multimodal data acquisition technologies such as phase shift interferometry, thermal imaging analysis, and ultrasonic phased array detection, and combines multimodal variational autoencoders (MM-VAE) and graph neural networks (GNN) to perform joint feature learning and deep learning of graph structure data on comprehensive feature data, and finally generates a high-precision prediction model through reinforcement learning optimization; the model can predict the surface quality of the workpiece in real time, and adjust the processing parameters in time to ensure the processing quality through a distributed sensor network feedback system; in effect, the present invention significantly improves the accuracy and real-time performance of the prediction of the workpiece surface quality, and solves many limitations of the prior art.
[0010] A method for predicting the surface quality of a workpiece comprises the following steps:
[0011] Using phase-shift interferometry, thermal imaging and ultrasonic phased array equipment to respectively acquire interference image data, thermal imaging data and ultrasonic signal data of the workpiece, and pre-processing to generate interference feature data, thermal feature data and ultrasonic feature data, and fusing the interference feature data, thermal feature data and ultrasonic feature data to generate comprehensive feature data;
[0012] Inputting the comprehensive feature data into a multimodal variational autoencoder for joint feature learning to generate a joint feature representation; constructing the joint feature representation as graph structure data, and using a graph neural network to perform deep learning and optimization on the graph structure data to generate a graph structure state model;
[0013] Performing multi-stage predictive modeling on the graph structure state model, including preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization, to generate a final optimization model;
[0014] The interference image data, thermal imaging data and ultrasonic signal data of the workpiece are collected in real time, preprocessed and fused to generate real-time comprehensive feature data, which are input into the final optimization model for prediction to generate surface quality prediction results, and the surface quality prediction results are fed back to the processing equipment and operators through a distributed sensor network to adjust the processing parameters.
[0015] Preferably, the pretreatment comprises:
[0016] De-noise the interference image data and apply the median filter algorithm to remove the noise in the interference image;
[0017] Perform image enhancement on the interference image data and use the histogram equalization algorithm to enhance the image contrast;
[0018] Perform thermal image correction on thermal imaging data and apply multi-point temperature calibration method to correct thermal images;
[0019] Remove thermal noise from thermal imaging data and use low-pass filtering algorithm to remove noise from thermal imaging data;
[0020] Perform signal filtering on the ultrasonic signal data, and apply a bandpass filter to remove low-frequency and high-frequency noise in the ultrasonic signal;
[0021] The ultrasonic signal data is subjected to feature extraction, and the characteristic spectrum of the ultrasonic signal is extracted using the time-frequency analysis method.
[0022] Preferably, the data fusion includes:
[0023] The interference characteristic data, thermal characteristic data and ultrasonic characteristic data are standardized, and each characteristic data is standardized to zero mean and unit variance;
[0024] Performing weighted averaging on the standardized interference feature data, thermal feature data, and ultrasonic feature data, assigning a weight to each feature data, and calculating the weighted average to generate comprehensive feature data;
[0025] The weight determination method includes selecting the optimal weight combination through a cross-validation method based on the prediction performance of each feature data.
[0026] Preferably, the multimodal variational autoencoder performs joint feature learning through the following steps:
[0027] The comprehensive feature data is input into the encoder, and a multi-layer neural network is used to encode the comprehensive feature data and map it to the latent space;
[0028] Decoding from latent space, using a multi-layer neural network to decode the latent representation into reconstructed feature data;
[0029] Calculate the reconstruction error, use the mean square error as the loss function, and calculate the error between the original comprehensive feature data and the reconstructed feature data;
[0030] Optimize model parameters and use the back-propagation algorithm to adjust the parameters of the encoder and decoder to reduce the reconstruction error and improve the accuracy of the joint feature representation.
[0031] Preferably, the graph neural network performs deep learning and optimization on the graph structure data through the following steps:
[0032] The joint feature representation is constructed as graph structure data, each feature representation is regarded as a node in the graph, and the relationship between the nodes is regarded as an edge;
[0033] Perform graph convolution processing, use a multi-layer graph convolutional network to extract high-order features of nodes, and update the representation of each node;
[0034] Apply the graph attention mechanism to calculate the attention weight of each node and enhance the focus on key feature nodes;
[0035] Generate a graph structure state model and integrate the high-order feature representations of all nodes to form a graph structure state model.
[0036] Preferably, the graph attention mechanism includes:
[0037] Calculate the correlation between nodes and use the dot product attention mechanism to calculate the correlation score between nodes;
[0038] Normalize the correlation scores and use the Softmax function to normalize the correlation scores;
[0039] Weighted summation of node features, taking the normalized correlation score as the weight, weighted summation of node features is performed to generate attention-enhanced graph feature data.
[0040] Preferably, the multi-stage predictive modeling comprises the following steps:
[0041] Preliminary grouping and modeling: cluster analysis of joint feature representations, cluster joint feature representations using K-means clustering algorithm, and generate preliminary group distribution models; establish preliminary prediction models for each preliminary group, and model the preliminary groups using regression analysis or classification models;
[0042] Segment division and state labeling: divide the preliminary group data into multiple segments, divide the preliminary group data into fixed time windows based on the time series; label each segment, use the hidden Markov model to label the segment data, and generate state sequence data;
[0043] State transition modeling: construct the state sequence data into graph structure data, take each state as a node in the graph, and the transition relationship between nodes as an edge; use graph neural network to model the state sequence data and generate a state transition graph model;
[0044] Reinforcement learning optimization, optimize the state transition graph model in the reinforcement learning environment, define the state, action and reward mechanism, use the deep Q network or policy gradient method to optimize the model, and generate the final optimized model.
[0045] Preferably, the preliminary grouping and modeling comprises the following steps:
[0046] Cluster the joint feature representation using the K-means clustering algorithm, input the joint feature representation into the K-means clustering algorithm, select the appropriate number of clusters, and generate a preliminary group distribution model;
[0047] A preliminary prediction model is established for each preliminary group, and each preliminary group is modeled using regression analysis or classification model to generate preliminary prediction model data.
[0048] Preferably, the state transition modeling comprises the following steps:
[0049] Use graph neural network to perform multi-layer graph convolution processing on state sequence data, construct state sequence data into graph structure data, use multi-layer graph convolution network to extract high-order features of nodes, and update node representation;
[0050] Generate a state transition graph model, integrate the high-order feature representations of all nodes, form a state transition graph model, and use it to calculate and predict state transition probabilities.
[0051] A system for implementing the method for predicting the surface quality of a workpiece, comprising:
[0052] A data acquisition module, used to respectively acquire interference image data, thermal imaging data and ultrasonic signal data of the workpiece by using phase shift interferometry, thermal imaging and ultrasonic phased array equipment;
[0053] A preprocessing module, connected to the data acquisition module, for performing noise removal and image enhancement on the interference image data, performing thermal image correction and removing thermal noise on the thermal imaging data, and performing signal filtering and feature extraction on the ultrasonic signal data to generate interference feature data, thermal feature data and ultrasonic feature data;
[0054] A data fusion module, connected to the preprocessing module, is used to perform standardization processing on the interference feature data, the thermal feature data and the ultrasonic feature data, and perform weighted averaging to generate comprehensive feature data;
[0055] A feature extraction and modeling module, connected to the data fusion module, includes a multimodal variational autoencoder, which is used to perform joint feature learning on the comprehensive feature data to generate a joint feature representation; and a graph neural network, which is used to construct the joint feature representation into graph structure data, and perform deep learning and optimization on the graph structure data to generate a graph structure state model;
[0056] A prediction modeling module, connected to the feature extraction and modeling module, is used to perform multi-stage prediction modeling on the graph structure state model, including preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization, so as to generate a final optimization model;
[0057] The real-time prediction and feedback module is connected to the prediction modeling module and is used to collect the interference image data, thermal imaging data and ultrasonic signal data of the workpiece in real time, perform preprocessing and fusion, and generate real-time comprehensive feature data; input the real-time comprehensive feature data into the final optimization model for prediction to generate surface quality prediction results; and feed back the surface quality prediction results to the processing equipment and operators through the distributed sensor network to adjust the processing parameters.
[0058] Compared with the prior art, the advantages and beneficial effects of the present invention are:
[0059] The present invention realizes comprehensive capture of workpiece surface features through multimodal data fusion and feature extraction technology, and solves the limitation of single attribute grouping of existing solutions;
[0060] The present invention realizes joint feature learning of comprehensive feature data through the multimodal variational autoencoder (MM-VAE) technology, thereby improving the accuracy of feature representation;
[0061] The present invention uses the graph neural network (GNN) technology to achieve deep learning and optimization of graph structure data, capture the complex relationship between features, and solve the problem of high computational complexity of the state transition probability matrix;
[0062] The present invention realizes adaptive adjustment of model parameters and improves the prediction accuracy of the model by means of reinforcement learning optimization technology;
[0063] The present invention realizes accurate prediction of workpiece surface quality through multi-stage prediction modeling technology and improves the single assumption of the Markov model. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] Figure 1 It is a schematic diagram of the process of the present invention;
[0065] Figure 2 Schematic diagram of data preprocessing steps in the present invention;
[0066] Figure 3 It is a schematic diagram of the data fusion steps in the present invention;
[0067] Figure 4 Schematic diagram of the joint feature learning steps in the present invention;
[0068] Figure 5 It is a schematic diagram of the steps of graph structure data modeling in the present invention;
[0069] Figure 6 It is a schematic diagram of the reinforcement learning optimization steps in the present invention;
[0070] Figure 7 It is a schematic diagram of the real-time prediction and feedback steps in the present invention;
[0071] Figure 8 It is a structural block diagram of the system of the present invention. DETAILED DESCRIPTION
[0072] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0073] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0074] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.
[0075] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0076] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some boxes or combinations thereof in the block diagrams and / or flow charts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions can create a device for implementing the functions / operations described in these block diagrams and / or flow charts when executed by the processor. The technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable storage medium storing instructions, which can be used by an instruction execution system or used in conjunction with an instruction execution system.
[0077] like Figure 1 As shown, a method for predicting the surface quality of a workpiece comprises the following steps:
[0078] like Figure 2-Figure 3 As shown, phase shift interferometry, thermal imaging and ultrasonic phased array equipment are used to respectively obtain interference image data, thermal imaging data and ultrasonic signal data of the workpiece, and pre-process them to generate interference feature data, thermal feature data and ultrasonic feature data, and the interference feature data, thermal feature data and ultrasonic feature data are fused to generate comprehensive feature data;
[0079] Preferably, the pretreatment comprises:
[0080] De-noise the interference image data and apply the median filter algorithm to remove the noise in the interference image;
[0081] Perform image enhancement on the interference image data and use the histogram equalization algorithm to enhance the image contrast;
[0082] Perform thermal image correction on thermal imaging data and apply multi-point temperature calibration method to correct thermal images;
[0083] Remove thermal noise from thermal imaging data and use low-pass filtering algorithm to remove noise from thermal imaging data;
[0084] Perform signal filtering on the ultrasonic signal data, and apply a bandpass filter to remove low-frequency and high-frequency noise in the ultrasonic signal;
[0085] The ultrasonic signal data is subjected to feature extraction, and the characteristic spectrum of the ultrasonic signal is extracted using the time-frequency analysis method.
[0086] The noise removal of the interference image data is mainly achieved through the median filtering algorithm. Median filtering is a nonlinear filtering technology that removes noise by replacing the value of each pixel with the median of its neighboring pixels. It is particularly effective in eliminating salt and pepper noise. Through this process, the random noise in the interference image can be significantly reduced, the image clarity and detail retention ability can be improved, and higher quality basic data can be provided for subsequent image enhancement and feature extraction. Next, the interference image data is enhanced by using the histogram equalization algorithm, which enhances the contrast of the image by adjusting the grayscale distribution of the image. Histogram equalization stretches the grayscale range of the image to make the details of the original dark and bright parts more obvious, improves the overall visual effect of the image, and helps to identify and analyze the details in the subsequent feature extraction process.
[0087] For the preprocessing of thermal imaging data, thermal image correction is first performed, and a multi-point temperature calibration method is applied. Multi-point temperature calibration ensures the accuracy and consistency of thermal imaging data by performing corrections at multiple known temperature points. This process can eliminate errors caused by ambient temperature changes or inaccurate equipment calibration, and improve the reliability of thermal imaging data. Subsequently, a low-pass filtering algorithm is used to remove thermal noise from the thermal imaging data. The low-pass filter can effectively remove high-frequency noise in thermal imaging data, retain key thermal features, improve data quality, and provide more accurate thermal distribution information for subsequent analysis.
[0088] For the preprocessing of ultrasonic signal data, signal filtering is first performed, and a bandpass filter is applied to remove low-frequency and high-frequency noise in the ultrasonic signal. The bandpass filter can selectively retain the intermediate frequency part of the signal, remove the low-frequency and high-frequency interference components, and enhance the effective components of the signal. Subsequently, the ultrasonic signal data is feature extracted, and the characteristic spectrum of the ultrasonic signal is extracted using the time-frequency analysis method. Time-frequency analysis methods, such as short-time Fourier transform (STFT), can analyze the signal in two dimensions of time and frequency, and extract the spectral characteristics of the signal at different time points. These characteristics are of great significance for identifying and analyzing internal defects of workpieces.
[0089] Through the above preprocessing steps, the interference image data, thermal imaging data and ultrasonic signal data generate high-quality interference feature data, thermal feature data and ultrasonic feature data respectively. After standardization and weighted averaging, these feature data will be fused into comprehensive feature data. The high quality and accuracy of the comprehensive feature data are crucial for the subsequent multimodal variational autoencoder joint feature learning and graph neural network deep learning, thereby ensuring the accuracy and reliability of the workpiece surface quality prediction.
[0090] Preferably, the data fusion includes:
[0091] The interference characteristic data, thermal characteristic data and ultrasonic characteristic data are standardized, and each characteristic data is standardized to zero mean and unit variance;
[0092] Performing weighted averaging on the standardized interference feature data, thermal feature data, and ultrasonic feature data, assigning a weight to each feature data, and calculating the weighted average to generate comprehensive feature data;
[0093] The weight determination method includes selecting the optimal weight combination through a cross-validation method based on the prediction performance of each feature data.
[0094] In the present invention, data fusion is a key step. By fusing multiple feature data, the accuracy and robustness of the prediction can be improved. Data fusion includes the following specific steps: First, the interference feature data, thermal feature data and ultrasonic feature data are standardized. Standardization refers to the zero mean unit variance standardization of each feature data, that is, the mean of the data is adjusted to 0 and the standard deviation is adjusted to 1. The principle of this processing method is to eliminate the dimensional difference of the data so that different types of feature data can be compared and fused on the same scale, avoiding a certain type of data from dominating the fusion process, thereby improving the effect of data fusion.
[0095] After the standardization process, the next step is to perform weighted averaging on the standardized interference feature data, thermal feature data, and ultrasonic feature data. In this step, a weight is assigned to each feature data, and the weighted average is calculated to generate comprehensive feature data. The principle of weighted averaging is to assign different weights according to the importance of each feature data in the prediction, so that the feature data that contributes more to the prediction result occupies a larger proportion in the comprehensive feature data. In a specific implementation, the method for determining the weight includes selecting the optimal weight combination based on the prediction performance of each feature data through a cross-validation method. The cross-validation method is a commonly used model validation technique. By dividing the data set into multiple subsets, training and validation are performed separately, the performance of the model on different data sets is evaluated, and finally the weight combination that optimizes the model prediction performance is selected.
[0096] For example, when predicting the surface quality of a workpiece, it may be found that the interference feature data has a higher accuracy in predicting surface roughness, while the thermal feature data has a higher accuracy in predicting surface thermal damage, and the ultrasonic feature data is more sensitive to internal defect detection. Through the cross-validation method, the optimal weight combination of interference feature data, thermal feature data, and ultrasonic feature data can be determined, so that the final generated comprehensive feature data can fully reflect the quality status of the workpiece surface. The generation of comprehensive feature data through the above-mentioned standardization and weighted average processing not only ensures the fusion of different feature data on the same scale, but also optimizes the data fusion method according to the prediction performance, thereby improving the accuracy and reliability of the workpiece surface quality prediction.
[0097] Through the data fusion step, feature data from different sources are integrated on the same scale, eliminating the dimensional differences between the data, and by optimizing the weight distribution, the integrated feature data can more accurately reflect the comprehensive quality status of the workpiece surface. This process lays a solid foundation for the subsequent multimodal variational autoencoder joint feature learning and graph neural network deep learning, ensuring the efficiency and accuracy of the entire prediction model.
[0098] like Figure 4-Figure 5 As shown, the comprehensive feature data is input into a multimodal variational autoencoder for joint feature learning to generate a joint feature representation; the joint feature representation is constructed as graph structure data, and a graph neural network is used to perform deep learning and optimization on the graph structure data to generate a graph structure state model;
[0099] In the present invention, inputting the comprehensive feature data into the multimodal variational autoencoder (MM-VAE) for joint feature learning and generating joint feature representation is an important step to improve prediction accuracy. The multimodal variational autoencoder extracts the potential feature representation of the high-dimensional space from the comprehensive feature data through the encoding and decoding process. Specifically, MM-VAE first inputs the comprehensive feature data into the encoder, and the encoder uses a multi-layer neural network structure to map the input data to a probability distribution in the latent space to obtain the latent feature representation. Then, the decoder extracts samples from this latent space and decodes them into reconstructed feature data. By calculating the reconstruction error between the original comprehensive feature data and the reconstructed feature data, the parameters of the encoder and decoder are optimized, thereby reducing the error and improving the accuracy of the joint feature representation.
[0100] The generated joint feature representation will be further constructed into graph-structured data. The construction of graph-structured data is to use graph neural networks (GNNs) for deep learning and optimization. The nodes in the graph-structured data represent different features, and the edges between the nodes represent the relationships and dependencies between the features. The graph neural network gradually extracts the high-order features of the nodes through multi-layer graph convolution operations and updates the representation of each node. Specifically, the graph convolution operation updates the feature representation of the target node by aggregating the information of neighboring nodes, thereby capturing the complex relationships and dependencies between features.
[0101] The graph neural network is used to perform deep learning and optimization on the graph structure data, and finally a graph structure state model is generated. The graph structure state model can effectively capture the high-dimensional relationship between the surface quality features of the workpiece and improve the prediction accuracy of the workpiece surface quality. The high-order features extracted by the graph convolutional network can more comprehensively reflect the quality changes of the workpiece surface under different processing stages and process parameters.
[0102] For example, in specific applications, by using MM-VAE to perform joint feature learning on comprehensive feature data, joint feature representations that can reflect the surface roughness, thermal distribution, and internal defects of the workpiece can be extracted. After these joint feature representations are constructed as graph structure data, the graph neural network extracts the high-dimensional relationship between different features on the workpiece surface through multi-layer graph convolution operations to form a graph structure state model. This model can accurately predict the surface quality of the workpiece under different processing conditions, thereby guiding the optimization and adjustment of the processing process and ensuring the stability and improvement of the workpiece surface quality.
[0103] In summary, by inputting the comprehensive feature data into the multimodal variational autoencoder for joint feature learning, generating a joint feature representation, and constructing it as graph structured data, deep learning and optimization are performed using a graph neural network to generate a graph structured state model. This process plays a key role in the prediction of workpiece surface quality and significantly improves the accuracy and reliability of the prediction.
[0104] Preferably, the multimodal variational autoencoder performs joint feature learning through the following steps:
[0105] The comprehensive feature data is input into the encoder, and a multi-layer neural network is used to encode the comprehensive feature data and map it to the latent space;
[0106] Decoding from latent space, using a multi-layer neural network to decode the latent representation into reconstructed feature data;
[0107] Calculate the reconstruction error, use the mean square error as the loss function, and calculate the error between the original comprehensive feature data and the reconstructed feature data;
[0108] Optimize model parameters and use the back-propagation algorithm to adjust the parameters of the encoder and decoder to reduce the reconstruction error and improve the accuracy of the joint feature representation.
[0109] The comprehensive feature data is input into the encoder, which encodes the comprehensive feature data using a multi-layer neural network and maps it to the latent space. Specifically, the encoder converts the input high-dimensional feature data into low-dimensional latent feature representations through a multi-layer neural network. These latent feature representations can capture the key features and patterns of the original data while reducing the redundancy and noise of the data.
[0110] Decoding from the latent space, using a multi-layer neural network to decode the latent representation into reconstructed feature data. The task of the decoder is to reconstruct feature data that is as similar as possible to the original data from the low-dimensional latent feature representation. The purpose of this step is to verify whether the latent feature representation extracted by the encoder can effectively represent the original data.
[0111] Calculate the reconstruction error, using the mean square error as the loss function to calculate the error between the original comprehensive feature data and the reconstructed feature data. The mean square error is a common method to measure the difference between the original data and the reconstructed data. It quantifies the degree of deviation between the reconstructed data and the original data by calculating the average of the squared errors of each feature. The smaller the reconstruction error, the more accurately the latent feature representation extracted by the encoder can reflect the structure and characteristics of the original data.
[0112] Optimize model parameters and use the back-propagation algorithm to adjust the parameters of the encoder and decoder to reduce reconstruction error and improve the accuracy of joint feature representation. The back-propagation algorithm is a commonly used optimization method that gradually adjusts the model parameters by calculating the gradient of the loss function, so that the reconstruction error continues to decrease, thereby improving the performance and stability of the model.
[0113] For example, in the prediction of workpiece surface quality, by inputting the comprehensive feature data after the fusion of interference feature data, thermal feature data and ultrasonic feature data into MM-VAE, the encoder maps these high-dimensional comprehensive feature data to a low-dimensional latent space. These low-dimensional latent feature representations can effectively capture the key features of workpiece surface quality. Then, the decoder reconstructs the feature data from these latent feature representations, compares them with the original comprehensive feature data, and calculates the reconstruction error. The parameters of the encoder and decoder are continuously optimized through the back-propagation algorithm, and the reconstruction error is finally minimized, thereby extracting an accurate joint feature representation. These joint feature representations can be used for subsequent graph neural network deep learning and optimization to improve the accuracy and reliability of workpiece surface quality prediction.
[0114] In summary, the multimodal variational autoencoder effectively extracts the potential feature representation of the comprehensive feature data through the steps of encoding, decoding, calculating the reconstruction error and optimizing the model parameters. This process not only improves the accuracy of the feature representation, but also provides high-quality input data for the subsequent prediction model, ensuring the efficiency and accuracy of the workpiece surface quality prediction.
[0115] Preferably, the graph neural network performs deep learning and optimization on the graph structure data through the following steps:
[0116] The joint feature representation is constructed as graph structure data, each feature representation is regarded as a node in the graph, and the relationship between the nodes is regarded as an edge;
[0117] Perform graph convolution processing, use a multi-layer graph convolutional network to extract high-order features of nodes, and update the representation of each node;
[0118] Apply the graph attention mechanism to calculate the attention weight of each node and enhance the focus on key feature nodes;
[0119] Generate a graph structure state model and integrate the high-order feature representations of all nodes to form a graph structure state model.
[0120] The joint feature representation is constructed as graph-structured data. The nodes in the graph-structured data represent each joint feature, while the edges between the nodes represent the relationship and dependency between the features. For example, in the prediction of workpiece surface quality, the nodes can represent different surface features (such as surface roughness, heat distribution, internal defects, etc.), while the edges represent the mutual influence and association between these features. Through this graph structure, the complex relationship between features can be captured more intuitively and systematically, providing a structured data foundation for subsequent deep learning.
[0121] Perform graph convolution processing. Graph convolution network (GCN) is a deep learning model that can perform convolution operations on graph structured data. Use multi-layer graph convolution networks to extract high-order features of nodes and update the representation of each node. Specifically, each layer of graph convolution network updates the feature representation of the target node by aggregating information from neighboring nodes, thereby capturing the complex relationships and dependencies between nodes layer by layer. For example, through multi-layer processing of GCN, more abstract high-order features can be gradually extracted from simple node features, reflecting the deep structure and pattern of the surface quality features of the workpiece.
[0122] In order to further improve the accuracy of the model, the graph attention mechanism is applied. The graph attention mechanism is a method to enhance the weights of key feature nodes. By calculating the attention weight of each node, the nodes that have an important impact on the prediction results are highlighted. Specifically, the graph attention mechanism calculates the correlation scores between nodes, then normalizes these scores through the Softmax function, and finally weighted sums the node features to generate attention-enhanced graph feature data. For example, in the prediction of workpiece surface quality, certain features (such as surface roughness or thermal damage at a specific location) may have a greater impact on the overall quality. The graph attention mechanism can highlight these key features, thereby improving the accuracy of the prediction.
[0123] Generate a graph structure state model. The graph structure state model integrates the high-order feature representations of all nodes to form a model that comprehensively reflects the quality state of the workpiece surface. For example, by integrating the high-order representations of multiple features such as surface roughness, heat distribution, and internal defects, the graph structure state model can comprehensively reflect the quality state of the workpiece surface under different processing conditions, providing a reliable basis for predicting and optimizing the processing process.
[0124] For example, in a specific application, the comprehensive feature data generated by fusing interference feature data, thermal feature data, and ultrasonic feature data is constructed into graph structure data, and the high-order features of the nodes are extracted layer by layer using a graph convolutional network to update the representation of each node. Subsequently, the graph attention mechanism is applied to calculate the attention weight of each node, highlight the key feature nodes, and generate attention-enhanced graph feature data. Finally, by integrating the high-order feature representations of all nodes, a graph structure state model is generated to predict the surface quality of workpieces under different processing conditions.
[0125] In summary, the graph neural network effectively captures the complex relationship between the surface quality features of the workpiece by constructing graph structure data, performing graph convolution processing, applying the graph attention mechanism and generating a graph structure state model, thereby improving the accuracy and reliability of the prediction. This process not only optimizes the feature representation, but also provides high-quality input data for subsequent prediction models, ensuring the efficiency and accuracy of the workpiece surface quality prediction.
[0126] Preferably, the graph attention mechanism includes:
[0127] Calculate the correlation between nodes and use the dot product attention mechanism to calculate the correlation score between nodes;
[0128] Normalize the correlation scores and use the Softmax function to normalize the correlation scores;
[0129] Weighted summation of node features, taking the normalized correlation score as the weight, weighted summation of node features is performed to generate attention-enhanced graph feature data.
[0130] First, the correlation between nodes is calculated, and the dot product attention mechanism is used to calculate the correlation score between nodes. In graph structure data, each node represents a feature, and the edges between nodes represent the relationship between features. The dot product attention mechanism measures the correlation between nodes by calculating the dot product between their feature vectors. Specifically, given the feature vectors h of node i and node j, i and h j , the correlation score e ij Calculated by the following formula:
[0131] e ij =h i ·h j
[0132] The dot product operation in this formula captures the similarity or correlation between nodes. A high correlation score indicates a strong correlation between the features of two nodes, which is very important in the prediction of workpiece surface quality because some features (such as surface roughness and thermal distribution) may have significant correlation and have a significant impact on the prediction results.
[0133] Next, normalize the relevance scores and use the Softmax function to normalize the relevance scores. The Softmax function converts the relevance scores between all nodes into probability distributions to ensure that the sum of all relevance scores is 1. Specifically, given a set of relevance scores e between a given node i and its neighboring nodes, ij , the normalized weight α ij Calculated by the following formula:
[0134]
[0135] in, Represents the set of all neighbor nodes of node i. Through Softmax normalization, the correlation score of each node with its neighbor nodes is converted into a weight between 0 and 1, which helps to maintain numerical stability during weighted summation.
[0136] Finally, the node features are weighted and summed, and the normalized relevance score is used as the weight to generate the attention-enhanced graph feature data. Specifically, the final feature representation h′ of node i is i Calculated by the following formula:
[0137]
[0138] This process enhances the focus on key feature nodes by aggregating the features of neighboring nodes and weighting them according to the relevance weights. For workpiece surface quality prediction, this means that the model can more accurately identify and utilize features that have an important impact on the prediction results. For example, if a node represents an important feature of the workpiece surface (such as roughness or thermal damage), the graph attention mechanism can effectively transfer and aggregate its feature information according to its high relevance weight, thereby improving the accuracy of the prediction.
[0139] For example, in practical applications, suppose there are three nodes representing the roughness, heat distribution, and internal defect characteristics of the workpiece surface. After calculating the correlation scores between them through the dot product attention mechanism, the Softmax function is used to normalize these scores to obtain the weight of each node. Then, through weighted summation, attention-enhanced graph feature data is generated, highlighting the features that have a significant impact on surface quality prediction. This process can significantly improve the accuracy of the prediction model for workpiece surface quality and provide reliable data support for subsequent processing and quality control.
[0140] In summary, the graph attention mechanism significantly enhances the focus on key feature nodes and improves the accuracy of workpiece surface quality prediction by calculating the correlation between nodes, normalizing the correlation score and weighted summing node features. This process not only optimizes the feature representation, but also provides high-quality input data for subsequent prediction models, ensuring the efficiency and accuracy of workpiece surface quality prediction.
[0141] like Figure 6 As shown, multi-stage predictive modeling is performed on the graph structure state model, including preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization, so as to generate a final optimization model;
[0142] In the present invention, multi-stage prediction modeling of the graph structure state model is an important step to improve prediction accuracy and robustness. This process includes preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization, and finally generates an optimized prediction model.
[0143] First, preliminary grouping and modeling is to perform cluster analysis on the joint feature representation. By using the K-means clustering algorithm, the joint feature representation is grouped to generate a preliminary group distribution model. Each preliminary group represents a subset of the workpiece surface features, which have similarities in the feature space. The principle of this step is to cluster similar features together through clustering analysis, thereby reducing the diversity of features and improving the efficiency and accuracy of subsequent modeling. For example, in the prediction of workpiece surface quality, surface roughness, thermal distribution, and internal defect features can be clustered into different groups respectively. The features in each group are similar to a certain extent, which helps to model and predict more accurately.
[0144] Next, the segment division and state labeling is to further subdivide the preliminary group data and label each segment with a state. Specifically, the preliminary group data is divided into multiple segments according to the time series to generate segment data. Then, the hidden Markov model (HMM) is used to label each segment with a state to generate state sequence data. The principle of this step is to capture the changes of features in the time dimension through time series analysis. The hidden Markov model can effectively identify and label different states through state transition probabilities. For example, during the workpiece processing process, the surface quality characteristics may change in different time periods. Segment division and state labeling can capture these changes and improve the timeliness and accuracy of the prediction.
[0145] State transition modeling is to construct state sequence data as graph structure data, and use graph neural network (GNN) to model it to generate a state transition graph model. Specifically, state transition modeling constructs a state transition graph by taking each state as a node in the graph and the transition relationship between nodes as an edge. The graph neural network extracts high-order features of nodes and updates node representations through multi-layer graph convolution operations. The principle of this step is to capture the complex relationships and dependencies between states through the state transfer of the graph structure. For example, in the prediction of workpiece surface quality, the state transition model can reflect the changes and transfers of features in different processing stages, providing more comprehensive input data for the prediction model.
[0146] Finally, reinforcement learning optimization is to optimize the state transition graph model in a reinforcement learning environment. By defining the state, action, and reward mechanism, the model is optimized using a deep Q network (DQN) or a policy gradient method to generate the final optimized model. Reinforcement learning continuously optimizes model parameters through experimentation and adjustment, so that the model can make optimal decisions in different environments. The principle of this step is that through the reinforcement learning algorithm, the model can adaptively adjust parameters to improve the accuracy and robustness of the prediction. For example, through reinforcement learning optimization, the adaptability of the model under different processing conditions can be improved, thereby more accurately predicting the surface quality of the workpiece.
[0147] For example, in a specific application, suppose there is a set of workpiece surface feature data, including roughness, heat distribution, and internal defects. Through preliminary grouping and modeling, these feature data are clustered into three groups, and then the data of each group is segmented and state labeled to generate state sequence data. Next, the state sequence data is constructed as graph structure data, and the state transition modeling is performed using a graph neural network to generate a state transition graph model. Finally, through reinforcement learning optimization, the model parameters are adjusted to improve the prediction accuracy and adaptability of the model. The final generated optimization model can accurately predict the surface quality of the workpiece under different processing conditions, providing reliable data support for the optimization and quality control of the workpiece processing process.
[0148] In summary, multi-stage predictive modeling of the graph structure state model can significantly improve the accuracy and robustness of workpiece surface quality prediction through preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization. This process not only optimizes the feature representation, but also provides high-quality input data for the prediction model, ensuring the efficiency and accuracy of workpiece surface quality prediction.
[0149] Preferably, the multi-stage predictive modeling comprises the following steps:
[0150] Preliminary grouping and modeling: cluster analysis of joint feature representations, cluster joint feature representations using K-means clustering algorithm, and generate preliminary group distribution models; establish preliminary prediction models for each preliminary group, and model the preliminary groups using regression analysis or classification models;
[0151] Segment division and state labeling: divide the preliminary group data into multiple segments, divide the preliminary group data into fixed time windows based on the time series; label each segment, use the hidden Markov model to label the segment data, and generate state sequence data;
[0152] State transition modeling: construct the state sequence data into graph structure data, take each state as a node in the graph, and the transition relationship between nodes as an edge; use graph neural network to model the state sequence data and generate a state transition graph model;
[0153] Reinforcement learning optimization, optimize the state transition graph model in the reinforcement learning environment, define the state, action and reward mechanism, use the deep Q network or policy gradient method to optimize the model, and generate the final optimized model.
[0154] Preliminary grouping and modeling is to cluster the joint feature representation, cluster the joint feature representation using the K-means clustering algorithm, and generate a preliminary group distribution model. The K-means clustering algorithm distributes data points into K clusters through iterative optimization, so that the data points in each cluster are closer in the feature space. The principle of this process is to cluster similar features together through cluster analysis, reduce the diversity of features, and improve the efficiency and accuracy of subsequent modeling. A preliminary prediction model is established for each preliminary group, and the preliminary group is modeled using regression analysis or classification models. For example, in the prediction of workpiece surface quality, the surface roughness, thermal distribution, and internal defect features can be clustered into different groups respectively. The features in each group are similar to a certain extent, which helps to model and predict more accurately.
[0155] Segment division and state labeling is to further subdivide the preliminary group data and label each segment with a state. Specifically, the preliminary group data is divided into multiple segments according to the time series to generate segment data. Then, the hidden Markov model (HMM) is used to label each segment with a state to generate state sequence data. The hidden Markov model can effectively identify and label different states through state transition probabilities. The principle of this step is to capture the changes in features in the time dimension through time series analysis. For example, during the workpiece processing process, the surface quality features may change in different time periods. Segment division and state labeling can capture these changes and improve the timeliness and accuracy of the prediction.
[0156] State transition modeling is to construct state sequence data as graph structure data, and use graph neural network (GNN) to model it to generate a state transition graph model. Specifically, state transition modeling constructs a state transition graph by taking each state as a node in the graph and the transition relationship between nodes as an edge. The graph neural network extracts high-order features of nodes and updates node representations through multi-layer graph convolution operations. The principle of this step is to capture the complex relationships and dependencies between states through the state transfer of the graph structure. For example, in the prediction of workpiece surface quality, the state transition model can reflect the changes and transfers of features in different processing stages, providing more comprehensive input data for the prediction model.
[0157] Reinforcement learning optimization is to optimize the state transition graph model in a reinforcement learning environment. By defining the state, action, and reward mechanism, the model is optimized using a deep Q network (DQN) or a policy gradient method to generate the final optimized model. Reinforcement learning continuously optimizes model parameters through experimentation and adjustment, so that the model can make optimal decisions in different environments. The principle of this step is that through the reinforcement learning algorithm, the model can adaptively adjust parameters to improve the accuracy and robustness of the prediction. For example, through reinforcement learning optimization, the adaptability of the model under different processing conditions can be improved, thereby more accurately predicting the surface quality of the workpiece.
[0158] For example, suppose there is a set of workpiece surface feature data, including roughness, heat distribution, and internal defects. Through preliminary grouping and modeling, these feature data are clustered into three groups, and then the data of each group is segmented and state labeled to generate state sequence data. Next, the state sequence data is constructed as graph structure data, and the state transition modeling is performed using a graph neural network to generate a state transition graph model. Finally, through reinforcement learning optimization, the model parameters are adjusted to improve the prediction accuracy and adaptability of the model. The final generated optimization model can accurately predict the surface quality of the workpiece under different processing conditions, providing reliable data support for the optimization and quality control of the workpiece processing process.
[0159] In summary, multi-stage predictive modeling gradually improves the accuracy and robustness of workpiece surface quality prediction through preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization. This process not only optimizes feature representation, but also provides high-quality input data for the prediction model, ensuring the efficiency and accuracy of workpiece surface quality prediction.
[0160] Preferably, the preliminary grouping and modeling comprises the following steps:
[0161] Cluster the joint feature representation using the K-means clustering algorithm, input the joint feature representation into the K-means clustering algorithm, select the appropriate number of clusters, and generate a preliminary group distribution model;
[0162] A preliminary prediction model is established for each preliminary group, and each preliminary group is modeled using regression analysis or classification model to generate preliminary prediction model data.
[0163] The joint feature representation is clustered using the K-means clustering algorithm. The K-means clustering algorithm is a commonly used unsupervised learning method that distributes data points into K clusters through iterative optimization, so that the data points in each cluster are closer in the feature space. Specifically, the joint feature representation is input into the K-means clustering algorithm, and similar feature representations are clustered together by selecting an appropriate number of clusters K. The core of the K-means algorithm is to optimize the distribution of clusters by minimizing the distance between the data points in the cluster and the cluster center. The principle of this process is to cluster similar features together through cluster analysis, reduce the diversity of features, and improve the efficiency and accuracy of subsequent modeling. For example, in the prediction of workpiece surface quality, the surface roughness, thermal distribution, and internal defect features can be clustered into different groups respectively. The features in each group are similar to a certain extent, which helps to model and predict more accurately.
[0164] A preliminary prediction model is established for each preliminary group. Specifically, each preliminary group is modeled using regression analysis or classification model to generate preliminary prediction model data. Regression analysis is used to deal with the prediction problem of continuous variables, while classification model is used to deal with the prediction problem of discrete variables. By establishing a special prediction model for each preliminary group, the relationship between the features in the group can be captured more accurately, thereby improving the accuracy of the prediction. For example, for each group after clustering, linear regression, logistic regression, support vector machine or decision tree model can be used to model separately to adapt to the characteristics and data distribution of different feature groups.
[0165] For example, suppose there is a set of workpiece surface feature data, including roughness, thermal distribution, and internal defects. Through preliminary grouping and modeling, these feature data are clustered into three groups, and then regression analysis and classification model modeling are performed on the data of each group. For the first group (such as surface roughness data), a linear regression model can be used for modeling; for the second group (such as thermal distribution data), a logistic regression model can be used for modeling; for the third group (such as internal defect data), a decision tree model can be used for modeling. In this way, the generation of preliminary prediction model data can more accurately reflect the relationship and dependency between different feature groups, thereby improving the accuracy of workpiece surface quality prediction.
[0166] In summary, preliminary grouping and modeling significantly improves the efficiency and accuracy of workpiece surface quality prediction by clustering the joint feature representation using the K-means clustering algorithm, generating a preliminary group distribution model, and establishing a preliminary prediction model for each preliminary group. This process not only optimizes the feature representation, but also provides high-quality input data for subsequent multi-stage prediction modeling, ensuring the efficiency and accuracy of workpiece surface quality prediction.
[0167] Preferably, the state transition modeling comprises the following steps:
[0168] Use graph neural network to perform multi-layer graph convolution processing on state sequence data, construct state sequence data into graph structure data, use multi-layer graph convolution network to extract high-order features of nodes, and update node representation;
[0169] Generate a state transition graph model, integrate the high-order feature representations of all nodes, form a state transition graph model, and use it to calculate and predict state transition probabilities.
[0170] Use graph neural networks to perform multi-layer graph convolution processing on state sequence data. State sequence data represents the quality status of workpieces in different time periods. These state data are constructed as graph structure data, where nodes represent different states and edges represent the transfer relationship between states. By constructing state sequence data as graph structure data, the complex relationship between features can be captured more systematically. Specifically, the multi-layer graph convolution network extracts high-order features of nodes by aggregating the information of neighbor nodes layer by layer and updates the representation of nodes. The principle of the graph convolution operation is to slide the convolution kernel on the graph structure, aggregate the neighbor information of each node, and thus capture deeper feature relationships. For example, in the prediction of workpiece surface quality, the surface roughness, thermal distribution, and internal defect characteristics in different time periods may have complex interdependencies. These high-order relationships can be extracted through the graph convolution network to improve the performance of the prediction model.
[0171] Generate a state transition graph model. The state transition graph model is formed by integrating the high-order feature representations of all nodes. The state transition graph model can effectively capture the dynamic changes of the surface quality characteristics of the workpiece at different processing stages. The principle of this process is to integrate the high-order features extracted by the multi-layer graph convolutional network to form a graph model that comprehensively reflects the quality state of the workpiece. This graph model can not only describe the characteristics of the current state, but also predict the transition probability between states, thereby providing the ability to predict future states. For example, in practical applications, the state transition graph model can predict the surface quality changes that may occur in the workpiece after different processing steps, thereby guiding the optimization and control of the processing process.
[0172] For example, suppose there is a set of state sequence data on the surface of a workpiece, including surface roughness, heat distribution, and internal defect characteristics. In the state transition modeling process, these data are first constructed as graph structure data, where each node represents the state of a time period, and the edges between nodes represent the transition relationship of the state in the time series. Multi-layer graph convolution processing is performed through the graph neural network to extract the high-order features of each node and update the representation of the node. For example, through a three-layer graph convolution network, the feature information of the first-order neighbors, second-order neighbors, and third-order neighbors of each node can be extracted layer by layer to capture deeper relationships. Finally, the high-order feature representations of all nodes are integrated to form a state transition graph model. This model can not only describe the current state of the workpiece surface quality, but also predict the trend of future state changes, providing an important basis for the optimization of the machining process.
[0173] In summary, state transition modeling uses graph neural networks to perform multi-layer graph convolution processing on state sequence data, extract high-order features of nodes, and integrate these features to form a state transition graph model, which effectively captures the complex relationship and dynamic changes between workpiece surface quality features and improves the accuracy and reliability of prediction. This process not only optimizes feature representation, but also provides high-quality input data for subsequent reinforcement learning optimization, ensuring the efficiency and accuracy of workpiece surface quality prediction.
[0174] like Figure 7 As shown, the interference image data, thermal imaging data and ultrasonic signal data of the workpiece are collected in real time, preprocessed and fused, and real-time comprehensive feature data are generated. The real-time comprehensive feature data is input into the final optimization model for prediction, and the surface quality prediction results are generated. The surface quality prediction results are fed back to the processing equipment and operators through the distributed sensor network to adjust the processing parameters.
[0175] Real-time collection of interference image data, thermal imaging data and ultrasonic signal data. Interference image data can capture the microscopic morphological changes on the workpiece surface, thermal imaging data can reflect the temperature distribution and thermal anomalies on the workpiece surface, and ultrasonic signal data can detect defects and unevenness inside the workpiece. By collecting these data in real time, comprehensive information on the surface and internal state of the workpiece during the processing can be obtained. These real-time data provide the basis for subsequent preprocessing and fusion.
[0176] The collected data is preprocessed and fused. The preprocessing steps include noise removal and image enhancement for interference image data, thermal image correction and thermal noise removal for thermal imaging data, and signal filtering and feature extraction for ultrasonic signal data. The purpose of these steps is to improve the quality and availability of data, remove noise and interference, and retain key features. For example, median filtering is performed on interference image data to remove noise, and histogram equalization is used to enhance image contrast; multi-point temperature calibration and low-pass filtering are performed on thermal imaging data to remove thermal noise; and band-pass filtering and time-frequency analysis are performed on ultrasonic signal data to extract feature spectra. After preprocessing, the interference feature data, thermal feature data, and ultrasonic feature data are fused to generate real-time comprehensive feature data. This step uses data standardization and weighted averaging to compare and fuse different types of feature data on the same scale, generating a more comprehensive and accurate comprehensive feature representation.
[0177] The real-time comprehensive feature data is input into the final optimization model for prediction. The final optimization model is generated through the aforementioned multi-stage predictive modeling and reinforcement learning optimization, and has the characteristics of high accuracy and high robustness. After the model receives the real-time comprehensive feature data, it generates the workpiece surface quality prediction results through deep learning and feature extraction processes. For example, the model can predict the surface roughness, heat distribution and internal defects of the workpiece under the current processing conditions. These prediction results provide an important basis for real-time monitoring and quality control of the processing process.
[0178] The surface quality prediction results are fed back to the processing equipment and operators through the distributed sensor network. The role of the distributed sensor network is to transmit and display the prediction results in real time, so that the processing equipment and operators can adjust the processing parameters in time according to the prediction results. By adjusting the processing parameters, the processing process can be optimized, the surface defects of the workpiece can be reduced, and the processing quality and efficiency can be improved. For example, when the prediction results show that the surface roughness of the workpiece exceeds expectations, the operator can adjust the processing speed or cutting depth to improve the surface quality; when the thermal imaging data shows that the surface temperature is too high, the cooling system can be adjusted to reduce the risk of thermal damage.
[0179] In summary, the interference image data, thermal imaging data and ultrasonic signal data of the workpiece are collected and processed in real time, and real-time comprehensive feature data is generated through preprocessing and fusion. These data are input into the final optimization model for prediction, and the surface quality prediction results are generated. The results are fed back to the processing equipment and operators through the distributed sensor network to adjust the processing parameters. This process ensures the efficiency and accuracy of the workpiece surface quality prediction, and provides strong technical support for the optimization and quality control of the processing process.
[0180] like Figure 8 As shown, a system for implementing the method for predicting the surface quality of a workpiece comprises:
[0181] The data acquisition module is used to obtain the interference image data, thermal imaging data and ultrasonic signal data of the workpiece using phase shift interferometry, thermal imaging and ultrasonic phased array equipment respectively; the data acquisition module is responsible for obtaining the interference image data, thermal imaging data and ultrasonic signal data of the workpiece using phase shift interferometry, thermal imaging and ultrasonic phased array equipment respectively. Phase shift interferometry can capture the microscopic morphological changes on the surface of the workpiece and provide high-precision surface roughness information; thermal imaging technology reveals thermal damage and abnormal heat sources by detecting the temperature distribution on the surface of the workpiece; ultrasonic phased array technology detects internal defects and structural inhomogeneities through the propagation and reflection of sound waves inside the workpiece. Through these three technical means, the system can comprehensively obtain the surface and internal feature data of the workpiece, providing a rich information basis for subsequent processing and prediction.
[0182] The preprocessing module is connected to the data acquisition module and is used to remove noise and enhance the interference image data, perform thermal image correction and remove thermal noise on the thermal imaging data, and perform signal filtering and feature extraction on the ultrasonic signal data to generate interference feature data, thermal feature data and ultrasonic feature data; the preprocessing module is connected to the data acquisition module and is responsible for preliminary processing of the collected data. For the interference image data, the preprocessing includes noise removal and image enhancement. Noise removal mainly uses the median filter algorithm to remove random noise; image enhancement uses histogram equalization to improve the contrast and detail resolution of the image. For the thermal imaging data, the preprocessing includes thermal image correction and thermal noise removal. Thermal image correction ensures the accuracy of temperature data through multi-point temperature calibration, and thermal noise removal uses low-pass filtering to remove high-frequency noise. For the ultrasonic signal data, the preprocessing includes signal filtering and feature extraction. Signal filtering uses a bandpass filter to remove low-frequency and high-frequency noise, and feature extraction extracts the characteristic spectrum of the ultrasonic signal through time-frequency analysis. The preprocessed data generates interference feature data, thermal feature data and ultrasonic feature data respectively, providing high-quality input for data fusion.
[0183] The data fusion module is connected to the preprocessing module and is used to standardize the interference feature data, thermal feature data and ultrasonic feature data, and perform weighted averaging to generate comprehensive feature data; the data fusion module is connected to the preprocessing module and is responsible for fusing feature data from different sources. First, the interference feature data, thermal feature data and ultrasonic feature data are standardized to eliminate the dimensional differences between different feature data. The standardization process usually adopts the method of zero mean unit variance so that each feature data has the same scale. Then, a weighted average process is performed to assign a weight to each feature data, and the optimal weight combination is determined according to the importance of each feature data in the prediction to generate comprehensive feature data. This step generates a more comprehensive and accurate feature representation through data fusion, providing a reliable foundation for subsequent feature extraction and modeling.
[0184] The feature extraction and modeling module is connected to the data fusion module, including a multimodal variational autoencoder for joint feature learning of the comprehensive feature data and generating a joint feature representation; and a graph neural network for constructing the joint feature representation into graph structure data, and performing deep learning and optimization on the graph structure data to generate a graph structure state model; the feature extraction and modeling module is connected to the data fusion module, including a multimodal variational autoencoder (MM-VAE) and a graph neural network (GNN). MM-VAE is responsible for joint feature learning of the comprehensive feature data, mapping high-dimensional feature data to latent space through the encoder, generating a joint feature representation, and then reconstructing the latent representation into feature data through the decoder, calculating the reconstruction error and optimizing the model parameters to ensure the accuracy of the feature representation. After the joint feature representation is constructed as graph structure data, a graph neural network is used for deep learning and optimization. GNN extracts high-order features of nodes through multi-layer graph convolution processing, updates node representations, and generates a graph structure state model. This process can capture the complex relationship between features and improve the accuracy and robustness of prediction.
[0185] The prediction modeling module is connected to the feature extraction and modeling module, and is used to perform multi-stage prediction modeling on the graph structure state model, including preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization to generate the final optimization model; the prediction modeling module is connected to the feature extraction and modeling module, and is responsible for performing multi-stage prediction modeling on the graph structure state model. First, through preliminary grouping and modeling, cluster analysis is performed on the joint feature representation, a preliminary group distribution model is generated using the K-means clustering algorithm, and a preliminary prediction model is established for each preliminary group. Next, segment division and state labeling are performed, the preliminary group data is divided into multiple segments according to the time series, and the state labeling of the segment data is performed using the hidden Markov model to generate state sequence data. Then, the state sequence data is constructed as graph structure data, and state transition modeling is performed using a graph neural network to generate a state transition graph model. Finally, the model parameters are optimized through reinforcement learning, and the deep Q network or policy gradient method is used to find the optimal model parameters and generate the final optimization model.
[0186] The real-time prediction and feedback module is connected to the prediction modeling module, and is used to collect the interference image data, thermal imaging data and ultrasonic signal data of the workpiece in real time, perform preprocessing and fusion, and generate real-time comprehensive feature data; input the real-time comprehensive feature data into the final optimization model for prediction, and generate surface quality prediction results; and feed back the surface quality prediction results to the processing equipment and operators through the distributed sensor network to adjust the processing parameters. The real-time prediction and feedback module is connected to the prediction modeling module, and is responsible for collecting the interference image data, thermal imaging data and ultrasonic signal data of the workpiece in real time, performing preprocessing and fusion, and generating real-time comprehensive feature data. The real-time comprehensive feature data is input into the final optimization model for prediction, and the surface quality prediction results are generated. The prediction results are fed back to the processing equipment and operators through the distributed sensor network. The operator can adjust the processing parameters in time according to the feedback results, such as adjusting the cutting speed, cutting depth or cooling system, to optimize the processing process, reduce the surface defects of the workpiece, and improve the processing quality and efficiency.
[0187] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0188] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0191] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0192] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0193] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0194] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0195] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for predicting the surface quality of a workpiece, characterized in that: The following steps are involved: Using phase-shift interferometry, thermal imaging and ultrasonic phased array equipment to respectively acquire interference image data, thermal imaging data and ultrasonic signal data of the workpiece, and pre-processing to generate interference feature data, thermal feature data and ultrasonic feature data, and fusing the interference feature data, thermal feature data and ultrasonic feature data to generate comprehensive feature data; Inputting the comprehensive feature data into a multimodal variational autoencoder for joint feature learning to generate a joint feature representation; constructing the joint feature representation as graph structure data, and using a graph neural network to perform deep learning and optimization on the graph structure data to generate a graph structure state model; Performing multi-stage predictive modeling on the graph structure state model, including preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization, to generate a final optimization model; The interference image data, thermal imaging data and ultrasonic signal data of the workpiece are collected in real time, preprocessed and fused to generate real-time comprehensive feature data, which are input into the final optimization model for prediction to generate surface quality prediction results, and the surface quality prediction results are fed back to the processing equipment and operators through a distributed sensor network to adjust the processing parameters.
2. The method according to claim 1, wherein the preprocessing comprises: De-noise the interference image data and apply the median filter algorithm to remove the noise in the interference image; Perform image enhancement on the interference image data and use the histogram equalization algorithm to enhance the image contrast; Perform thermal image correction on thermal imaging data and apply multi-point temperature calibration method to correct thermal images; Remove thermal noise from thermal imaging data and use low-pass filtering algorithm to remove noise from thermal imaging data; Perform signal filtering on the ultrasonic signal data, and apply a bandpass filter to remove low-frequency and high-frequency noise in the ultrasonic signal; The ultrasonic signal data is subjected to feature extraction, and the characteristic spectrum of the ultrasonic signal is extracted using the time-frequency analysis method.
3. The method according to claim 1, wherein the data fusion comprises: The interference characteristic data, thermal characteristic data and ultrasonic characteristic data are standardized, and each characteristic data is standardized to zero mean and unit variance; Performing weighted averaging on the standardized interference feature data, thermal feature data, and ultrasonic feature data, assigning a weight to each feature data, and calculating the weighted average to generate comprehensive feature data; The weight determination method includes selecting the optimal weight combination through a cross-validation method based on the prediction performance of each feature data.
4. According to the method of claim 1, the multimodal variational autoencoder performs joint feature learning by the following steps: The comprehensive feature data is input into the encoder, and a multi-layer neural network is used to encode the comprehensive feature data and map it to the latent space; Decoding from latent space, using a multi-layer neural network to decode the latent representation into reconstructed feature data; Calculate the reconstruction error, use the mean square error as the loss function, and calculate the error between the original comprehensive feature data and the reconstructed feature data; Optimize model parameters and use the back-propagation algorithm to adjust the parameters of the encoder and decoder to reduce the reconstruction error and improve the accuracy of the joint feature representation.
5. According to the method of claim 4, the graph neural network performs deep learning and optimization on the graph structure data through the following steps: The joint feature representation is constructed as graph structure data, each feature representation is regarded as a node in the graph, and the relationship between the nodes is regarded as an edge; Perform graph convolution processing, use a multi-layer graph convolutional network to extract high-order features of nodes, and update the representation of each node; Apply the graph attention mechanism to calculate the attention weight of each node and enhance the focus on key feature nodes; Generate a graph structure state model and integrate the high-order feature representations of all nodes to form a graph structure state model.
6. The method according to claim 5, wherein the graph attention mechanism comprises: Calculate the correlation between nodes and use the dot product attention mechanism to calculate the correlation score between nodes; Normalize the correlation scores and use the Softmax function to normalize the correlation scores; Weighted summation of node features, taking the normalized correlation score as the weight, weighted summation of node features is performed to generate attention-enhanced graph feature data.
7. The method according to claim 1, wherein the multi-stage predictive modeling comprises the following steps: Preliminary grouping and modeling: cluster analysis of joint feature representations, cluster joint feature representations using K-means clustering algorithm, and generate preliminary group distribution models; establish preliminary prediction models for each preliminary group, and model the preliminary groups using regression analysis or classification models; Segment division and state labeling: divide the preliminary group data into multiple segments, divide the preliminary group data into fixed time windows based on the time series; label each segment, use the hidden Markov model to label the segment data, and generate state sequence data; State transition modeling: construct the state sequence data into graph structure data, take each state as a node in the graph, and the transition relationship between nodes as an edge; use graph neural network to model the state sequence data and generate a state transition graph model; Reinforcement learning optimization, optimize the state transition graph model in the reinforcement learning environment, define the state, action and reward mechanism, use the deep Q network or policy gradient method to optimize the model, and generate the final optimized model.
8. The method according to claim 7, wherein the preliminary grouping and modeling comprises the following steps: Cluster the joint feature representation using the K-means clustering algorithm, input the joint feature representation into the K-means clustering algorithm, select the appropriate number of clusters, and generate a preliminary group distribution model; A preliminary prediction model is established for each preliminary group, and each preliminary group is modeled using regression analysis or classification model to generate preliminary prediction model data.
9. The method according to claim 7, wherein the state transition modeling comprises the following steps: Use graph neural network to perform multi-layer graph convolution processing on state sequence data, construct state sequence data into graph structure data, use multi-layer graph convolution network to extract high-order features of nodes, and update node representation; Generate a state transition graph model, integrate the high-order feature representations of all nodes, form a state transition graph model, and use it to calculate and predict state transition probabilities.
10. A system for implementing the workpiece surface quality prediction method according to any one of claims 1 to 9, characterized in that: include: A data acquisition module, used to respectively acquire interference image data, thermal imaging data and ultrasonic signal data of the workpiece by using phase shift interferometry, thermal imaging and ultrasonic phased array equipment; A preprocessing module, connected to the data acquisition module, for performing noise removal and image enhancement on the interference image data, performing thermal image correction and removing thermal noise on the thermal imaging data, and performing signal filtering and feature extraction on the ultrasonic signal data to generate interference feature data, thermal feature data and ultrasonic feature data; A data fusion module, connected to the preprocessing module, is used to perform standardization processing on the interference feature data, the thermal feature data and the ultrasonic feature data, and perform weighted averaging to generate comprehensive feature data; A feature extraction and modeling module, connected to the data fusion module, includes a multimodal variational autoencoder, which is used to perform joint feature learning on the comprehensive feature data to generate a joint feature representation; and a graph neural network, which is used to construct the joint feature representation into graph structure data, and perform deep learning and optimization on the graph structure data to generate a graph structure state model; A prediction modeling module, connected to the feature extraction and modeling module, is used to perform multi-stage prediction modeling on the graph structure state model, including preliminary grouping and modeling, segment division and state labeling, state transition modeling and reinforcement learning optimization, so as to generate a final optimization model; A real-time prediction and feedback module is connected to the prediction modeling module and is used to collect interference image data, thermal imaging data and ultrasonic signal data of the workpiece in real time, perform preprocessing and fusion, and generate real-time comprehensive feature data; The real-time comprehensive feature data is input into the final optimization model for prediction to generate surface quality prediction results; and the surface quality prediction results are fed back to the processing equipment and operators through a distributed sensor network to adjust the processing parameters.
Citation Information
Cited By
Multi-process manufacturing quality prediction method based on auto-encoder and graph attention network
CN119359156A
Multi-process manufacturing quality prediction method based on autoencoder and graph attention network
CN119359156B
Multi-process manufacturing process quality monitoring method and device and storage medium
CN119758928A
Industrial spraying workpiece defective product factor intelligent detection method based on graph neural network model
CN119807808A
An intelligent detection method for defective factors of industrial spraying workpieces based on graph neural network model
CN119807808B
