Industrial carbon emission data missing filling method and system based on process procedure and timing similarity knowledge migration
By employing knowledge transfer methods based on process flow and temporal similarity, and utilizing a large language model and joint loss function, the problem of filling in missing values in carbon emission data was solved, thereby improving data integrity and analytical accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2025-04-11
- Publication Date
- 2026-07-24
AI Technical Summary
When carbon emission data is missing, existing technologies struggle to accurately fill in the missing values, impacting the accuracy of data analysis and policy decisions. In particular, the selection of deep learning models is ineffective under small sample conditions.
By adding descriptive text related to process steps, feature vectors are extracted using a large language model to construct a vector database. A deep learning model is selected based on temporal similarity for knowledge transfer, and a joint loss function is used to train the model to fill in missing values.
In cases of insufficient data samples, this method improves the accuracy of missing value imputation, enhances the integrity of carbon emission data and the reliability of subsequent analysis, and reduces the difficulty of model training.
Smart Images
Figure CN120471330B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data science and data processing, specifically to a method and system for filling missing industrial carbon emission data based on knowledge transfer of process and time sequence similarity. Background Technology
[0002] Carbon emission data, as a typical type of time-series data, is crucial for monitoring carbon emission levels, assessing the effectiveness of emission reductions, and formulating sound policies. However, due to factors such as human error, equipment malfunction, or abnormal signal transmission, missing data in carbon emission data is unavoidable and can severely impact subsequent data analysis, trend forecasting, and policy decisions. Therefore, developing accurate and efficient data completion methods is essential to ensuring the integrity and reliability of carbon emission data.
[0003] Traditional missing value imputation methods mainly include statistical methods such as mean imputation, mode imputation, and median imputation. With the development of machine learning technology, methods such as Support Vector Machines (SVM), Random Forests, AdaBoost, and K-means clustering have been gradually applied to missing value imputation tasks, and have further evolved into deep learning methods based on Long Short-Term Memory Networks (LSTM) and diffusion models, significantly improving imputation accuracy. However, deep learning models rely on large-scale data to learn complex temporal features, and when data is insufficient, it is difficult to extract effective information, affecting the imputation effect. Therefore, knowledge transfer, by applying existing models or experience to related tasks, effectively improves learning efficiency and performance under small sample conditions.
[0004] In knowledge transfer, the choice of deep learning model has a significant impact on the transfer effect. Traditional methods typically use algorithms such as Dynamic Time Warping (DTW) to evaluate the similarity between the sequence to be imputed and the existing sequence, and select a deep learning model accordingly. However, when there are insufficient samples, the difference between the sequence to be imputed and the real sequence may lead to inaccurate similarity measurement, thus affecting the imputation effect. Therefore, a deep learning model selection method that does not completely rely on the existing sequence may further improve the imputation performance. Summary of the Invention
[0005] To overcome the shortcomings of existing technologies, this invention proposes a method for filling missing industrial carbon emission data based on knowledge transfer from process sequence and temporal similarity. First, descriptive text related to the process sequence is added to the input data, such as information on factory output, production process, procedures, and products. Then, a large language model is used to extract textual features related to industry, product, and process route, and construct vector representations. In a pre-established vector database, the top n most relevant data groups are matched based on industry, product, and process route similarity, and the similarity between their carbon emission-related data and the input carbon emission-related data is further calculated. Taking into account the similarity of industry, product, and process route similarity with the similarity of carbon emission-related data, a deep learning model is selected and trained using the input carbon emission-related data. Finally, the model with transferred knowledge is used to fill in the missing values.
[0006] The technical solution adopted by this invention to solve its technical problem is:
[0007] A method for filling missing industrial carbon emission data based on knowledge transfer from process steps and temporal similarity includes the following steps:
[0008] Step 1: Establish a vector database: For the input text, use the prompt word engineering to specify a large language model to extract process features from the descriptive text, generate text feature vectors, and build a vector database accordingly. For each carbon emission-related time series dataset, train the corresponding deep learning model, and associate and store the deep learning model with the text feature vectors and time series related data.
[0009] Step two, the input data includes descriptive text closely related to carbon emissions, and time series data related to carbon emissions X = {x1, x2, ..., x...} L}, where x t This represents the carbon emission data at time t, where t ranges from 1 to L, and L represents the length of the time series. The prompt words guide the large language model to extract the feature vectors of the process steps in the input descriptive text, generating the feature vector INPUT_E.
[0010] Step 3, deep learning model selection: In the pre-built vector database, using the feature vector INPUT_E as the query vector, retrieve the top n data groups with the highest text similarity. For the retrieved data groups, further calculate the similarity between their time series data and the input time series data. Employ a similarity measurement algorithm suitable for time series data, and comprehensively consider both text similarity and time series data similarity to select a deep learning model to achieve knowledge transfer.
[0011] Step 4, Model Training and Missing Value Imputation: On the selected deep learning model, train it using the input carbon emission-related time series data to complete the knowledge transfer process. Use the trained model to impute missing values in the time series data to improve data integrity and the accuracy of subsequent analysis.
[0012] Furthermore, the model in step one has the following input, output, and training methods:
[0013] 1.1 Define the model input and output: Input time series data Where B is the batch size, L is the sequence length, F is the number of features, and X = {x1, x2, ..., x} L Data x at each time step in} t It consists of two parts: carbon emission related data and time features. For the missing carbon emission data, zeros are used to fill in the missing parts. The time series data X is processed by a deep learning model and outputs the filling results for the missing carbon emission data.
[0014] 1.2 Training Loss Function: A joint loss function is constructed based on time domain and frequency domain information, with the time domain loss being the loss function. time The predicted value is measured using the SmoothL1 function. Compared with the true value y t Differences:
[0015]
[0016] Frequency domain loss FFT By performing Fourier transforms on the predicted and actual sequences, the frequency domain representation is obtained, and the mean absolute difference between the two is calculated:
[0017]
[0018] Where FFT(·) represents Fast Fourier Transform, and combining the above, the total loss function is Loss. total for
[0019] Loss total =Loss time +λ FFT *Loss FFT
[0020] Where, λ FFT is the scaling factor for the frequency domain loss;
[0021] 1.3 Model Training: In each training cycle, the middle portion of the input data X is first masked, and the mask window size is set to the maximum window W for the model to predict missing values. max The masked data is then fed into the model defined in step 1.1 to reconstruct the masked region and generate prediction results. Calculate the loss using the loss function defined in step 1.2, backpropagate the calculated loss, and update the model parameters using the optimizer. Repeat all steps in the training cycle until the pre-set training cycle is reached.
[0022] Furthermore, the method of extracting text feature vectors INPUT_E using a large-scale pre-trained model in steps one and two is as follows:
[0023] 2.1 Text Preprocessing and Tokenization: Given the input text Text, first use a tokenizer to split the text into n tokens.
[0024] Text→(token1,token2,…,token n )
[0025] Each token i This represents a token in the text and corresponds to a unique token ID;
[0026] 2.2 Token Embedding: Each token ID is mapped to a fixed-dimensional embedding vector, let the embedding matrix be... Where VL is the vocabulary size and d is the embedding dimension, then for a token... i Its embedding vector is
[0027]
[0028] 2.3 Add position code: Let the position code be p i Then the final input embedding is
[0029] z i =e i +p i
[0030] The entire sequence input is denoted as
[0031] Z = (z1, z2, ..., z n )
[0032] 2.4 Transformer Encoder Processing: The embedded sequence Z is input into the Transformer encoder of the large language model. Each encoder layer processes the input using a self-attention mechanism. Its core computation is as follows:
[0033]
[0034] In this context, the query Q, key K, and value V are derived from the input Z and the trainable weight matrix W. Q W K W VThe operation is performed by a linear transformation to obtain:
[0035] Q = ZW Q K = ZW K V=ZW V
[0036] After passing through multiple encoders, the hidden representation of each token is obtained.
[0037]
[0038] Among them, h i Refers to the hidden representation of the i-th token.
[0039] 2.5 Text Feature Vector Extraction: A pooling strategy is adopted to obtain a fixed-length representation of the entire text. Taking max pooling as an example, the maximum value of all hidden states of tokens is taken for each dimension.
[0040]
[0041] in, This is the final extracted text feature vector.
[0042] Furthermore, in step three, the process of selecting a deep learning model is as follows:
[0043] 3.1 Text Similarity Calculation: Suppose the input text is processed by a large language model to extract a feature vector INPUT_E, and the vector database stores the text feature vector E corresponding to each data group. i (i = 1, 2, ..., n), calculate the similarity between the input text and the text of each data set:
[0044] S text (i) = Similarity(INPUT_E, E i )
[0045] Select the top n data groups based on similarity values;
[0046] 3.2 Time series data similarity calculation: For each data group ranked in the top n, extract its corresponding time series X. i , and the input time series X in The two sequences are compared, and the DTW algorithm is used to calculate the distance D between them. DTW (X in ,X i ), convert the distance into a similarity score S time
[0047]
[0048] 3.3 Comprehensive Similarity Calculation and Deep Learning Model Selection: The comprehensive similarity score is calculated using the averaging method.
[0049]
[0050] Where S(i) represents the comprehensive similarity score of the i-th candidate data group, and the deep learning model corresponding to the group with the highest comprehensive score is finally selected for subsequent knowledge transfer training.
[0051] In step four, the model structure and training process remain consistent with the deep learning model constructed in step one. However, before training, the pre-trained weights of the selected deep learning model must be loaded to achieve effective knowledge transfer. After training, the model outputs W each time. max The missing data are used as candidate imputation data, and the corresponding prediction results are selected for imputation based on the location of the missing value.
[0052] An industrial carbon emission data missing completion system based on knowledge transfer from process steps and temporal similarity includes a deep learning model training module, a vector database module, a text feature vector extraction module, and a deep learning model selection module.
[0053] The deep learning model training module uses carbon emission-related time series data to train a deep learning model. The module performs masking processing on the input data and updates the model parameters based on the joint loss function, thereby enabling the model to reconstruct missing data. The module can choose whether to load weights to adapt to the different requirements of steps one and four.
[0054] The vector database module constructs and maintains a vector database that stores feature vectors extracted from descriptive text of each dataset and the corresponding pre-trained deep learning models. This module obtains data from open-source and private data sources to achieve efficient text feature storage and model association management, providing support for the rapid retrieval of subsequent deep learning models.
[0055] The text feature vector extraction module first embeds the input text according to a predefined prompt word template, and then uses a large-scale pre-trained language model to preprocess, segment, embed tokens, encode positions, and encode the text to obtain a fixed-dimensional text feature vector. This module is used to extract descriptive text feature vectors from various datasets when building a vector database, and also to generate feature vectors INPUT_E from the descriptive text of the input data, thereby providing a precise basis for the selection of deep learning models.
[0056] The deep learning model selection module uses the input text feature vector INPUT_E as the query vector. It retrieves the top n data groups with the highest text similarity from the vector database using cosine similarity, and further calculates temporal data similarity using methods such as DTW. The module then evaluates each candidate data group based on both text and temporal similarity, selecting the deep learning model with the highest overall score for knowledge transfer.
[0057] The technical concept of this invention is as follows: This invention utilizes a large-scale pre-trained model to extract feature vectors of industry and process information from descriptive text, and constructs a text vector database, associating carbon emission-related time-series data with corresponding descriptive text. For the data to be filled, the similarity between the text and the time-series data is comprehensively evaluated using cosine similarity and dynamic time warping algorithms. A corresponding deep learning model is then selected for knowledge transfer training, and the trained model is used to accurately fill in the missing values in the carbon emission-related data.
[0058] The beneficial effects of this invention are as follows: This invention fully integrates information from process descriptions and time-series data, enabling precise selection of deep learning models and knowledge transfer training; it improves the accuracy of missing value imputation in cases of insufficient data samples, significantly enhancing the completeness of carbon emission-related data and the reliability of subsequent analysis, while simultaneously reducing the difficulty of model training. By using a joint loss function, comprehensively considering both time-domain and frequency-domain losses, the training process can better capture subtle changes in carbon emission-related data. Attached Figure Description
[0059] Figure 1 This is a block diagram of the method of the present invention.
[0060] Figure 2 The input and output requirements for the deep learning model of this invention are specified.
[0061] Figure 3 These are the prompt words for the large language model in this invention. Detailed Implementation
[0062] The invention will now be further described with reference to the accompanying drawings.
[0063] Reference Figures 1-3 A method for filling missing industrial carbon emission data based on knowledge transfer from process steps and time sequence similarities includes the following steps:
[0064] Step 1, establish a vector database: Use prompt words for the input text, such as... Figure 3 As indicated by the prompt, a large language model is specified to extract process features from the descriptive text, generate text feature vectors, and build a vector database based on these vectors. For each carbon emission-related time series dataset, a corresponding deep learning model is trained, and the model is associated with and stored in relation to the text feature vectors and time series-related data.
[0065] The model in step one has the following input, output, and training method:
[0066] 1.1 Define the model input and output: Input time series data Where B is the batch size, L is the sequence length, F is the number of features, and X = {x1, x2, ..., x} L Each time step data x in} t It consists of two parts: carbon emission-related data and time features. For the missing parts of the carbon emission-related data, zeros are used for filling. The time series data X is processed by a deep learning model and outputs the filling results of the missing parts of the carbon emission-related data.
[0067] 1.2 Training Loss Function: A joint loss function is constructed based on time domain and frequency domain information, with the time domain loss being the loss function. time The predicted value is measured using the Smooth L1 function. Compared with the true value y t Differences:
[0068]
[0069] Frequency domain loss FFT By performing Fourier transforms on the predicted and actual sequences, the frequency domain representation is obtained, and the mean absolute difference between the two is calculated:
[0070]
[0071] Where FFT(·) represents Fast Fourier Transform, and combining the above, the total loss function is Loss. total for
[0072] Loss total =Loss time +λ FFT *Loss FFT
[0073] Where, λ FFT is the scaling factor for the frequency domain loss;
[0074] 1.3 Model Training: In each training cycle, firstly, the middle portion of the input data X is masked, and the mask window size is set to the maximum window W for the model to predict missing values. max The masked data is then fed into the model defined in step 1.1 to reconstruct the masked region and generate prediction results. Calculate the loss using the loss function defined in step 1.2, backpropagate the calculated loss, update the model parameters using the optimizer, and repeat all the steps in the training cycle until the pre-set training cycle is reached.
[0075] Step two, the input data includes descriptive text closely related to carbon emissions, such as information on factory output, process flow, and procedures, and time series data related to carbon emissions X = {x1, x2, ..., x...} L}, where x t This represents the carbon emission data at time t, where t ranges from 1 to L, and L represents the length of the time series. The prompt words guide the large language model to extract the feature vectors of the process steps in the input descriptive text, generating the feature vector INPUT_E.
[0076] The method for extracting the text feature vector INPUT_E using a large-scale pre-trained model in steps one and two is as follows:
[0077] 2.1 Text Preprocessing and Tokenization: Given the input text Text, first use a tokenizer to split the text into n tokens.
[0078] Text→(token1,token2,…,token n )
[0079] Each token i This represents a token in the text and corresponds to a unique token ID;
[0080] 2.2 Token Embedding: Each token ID is mapped to a fixed-dimensional embedding vector, let the embedding matrix be... Where VL is the vocabulary size and d is the embedding dimension, then for a token... i Its embedding vector is
[0081]
[0082] 2.3 Add position code: Let the position code be p i Then the final input embedding is
[0083] z i =e i +p i
[0084] The entire sequence input is denoted as
[0085] Z = (z1, z2, ..., z n )
[0086] 2.4 Transformer Encoder Processing: The embedded sequence Z is input into the Transformer encoder of the large language model. Each encoder layer processes the input using a self-attention mechanism. Its core computation is as follows:
[0087]
[0088] In this context, the query Q, key K, and value V are derived from the input Z and the trainable weight matrix W. Q W K W V The operation is performed by a linear transformation to obtain:
[0089] Q = ZW Q K = ZW K V=ZW V
[0090] After passing through multiple encoders, the hidden representation of each token is obtained.
[0091]
[0092] Among them, h i Refers to the hidden representation of the i-th token;
[0093] 2.5 Text Feature Vector Extraction: A pooling strategy is adopted to obtain a fixed-length representation of the entire text. Taking max pooling as an example, the maximum value of all hidden states of tokens is taken for each dimension.
[0094]
[0095] in, This is the final extracted text feature vector;
[0096] Step 3, deep learning model selection: In the pre-built vector database, using the feature vector INPUT_E as the query vector, the cosine similarity method is used to retrieve the top n data groups with the highest text similarity. For the retrieved data groups, the similarity between their time series data and the input time series data is further calculated. A similarity measurement algorithm suitable for time series data is adopted. Taking into account both text similarity and time series data similarity, a deep learning model is selected to achieve knowledge transfer.
[0097] In step three, the process of selecting the deep learning model is as follows:
[0098] 3.1 Text Similarity Calculation: Suppose the input text is processed by a large language model to extract a feature vector INPUT_E, and the vector database stores the text feature vector E corresponding to each data group. i For i = 1, 2, ..., n, cosine similarity is used to calculate the text similarity between the input text and each data group. The formula is as follows:
[0099]
[0100] Select the top n data groups based on similarity values;
[0101] 3.2 Time series data similarity calculation: For each data group ranked in the top n, extract its corresponding time series X. i , and the input time series X in The two sequences are compared, and the DTW algorithm is used to calculate the distance D between them. DTW (X in ,X i ), convert the distance into a similarity score S. time
[0102]
[0103] 3.3 Comprehensive Similarity Calculation and Deep Learning Model Selection: The comprehensive similarity score is calculated using the averaging method.
[0104]
[0105] Where S(i) represents the comprehensive similarity score of the i-th candidate data group, and the deep learning model corresponding to the group with the highest comprehensive score is finally selected for subsequent knowledge transfer training.
[0106] Step 4, Model Training and Missing Value Imputation: On the selected deep learning model, train it using the input carbon emission-related time series data to complete the knowledge transfer process. Use the trained model to impute missing values in the time series data to improve data integrity and the accuracy of subsequent analysis.
[0107] In step four, the model structure and training process remain consistent with the deep learning model constructed in step one. However, before training, the pre-trained weights of the selected deep learning model must be loaded to achieve effective knowledge transfer. After training, the model outputs W each time. max The missing data are used as candidate imputation data, and the corresponding prediction results are selected for imputation based on the location of the missing value.
[0108] This embodiment also provides an industrial carbon emission data missing filling system based on knowledge transfer of process steps and time sequence similarity, including a deep learning model training module, a vector database module, a text feature vector extraction module, and a deep learning model selection module.
[0109] The deep learning model training module uses carbon emission-related time series data to train a deep learning model. The module performs masking processing on the input data and updates the model parameters based on the joint loss function, thereby enabling the model to reconstruct missing data. The module can choose whether to load weights to adapt to the different requirements of steps one and four.
[0110] The vector database module constructs and maintains a vector database that stores feature vectors extracted from descriptive text of each dataset and the corresponding pre-trained deep learning models. This module obtains data from open-source and private data sources to achieve efficient text feature storage and model association management, providing support for the rapid retrieval of subsequent deep learning models.
[0111] The text feature vector extraction module first embeds the input text according to a predefined prompt word template, and then uses a large-scale pre-trained language model such as Qwen 2.5 to preprocess the text, perform word segmentation, token embedding, position encoding, and Transformer encoding to obtain a fixed-dimensional text feature vector. This module is used to extract descriptive text feature vectors from various datasets when building a vector database, and also to generate feature vector INPUT_E from the descriptive text of the input data, thereby providing a precise basis for the selection of deep learning models.
[0112] The deep learning model selection module uses the input text feature vector INPUT_E as the query vector, retrieves the top n data groups with the highest text similarity in the vector database using cosine similarity, and further calculates the temporal data similarity using methods such as DTW. It then evaluates each candidate data group by combining text and temporal similarity and selects the deep learning model with the highest comprehensive score for knowledge transfer.
[0113] As described above, the specific implementation steps of this patent make the present invention clearer. Any modifications and changes made to the present invention within the spirit and scope of the claims fall within the protection scope of the present invention.
Claims
1. A method for filling missing industrial carbon emission data based on knowledge transfer from process steps and temporal similarities, characterized in that, The method includes the following steps: Step 1: Establish a vector database: For the input text, use prompt word engineering, specify a large language model to extract process features from the descriptive text, generate text feature vectors, and build a vector database accordingly. For each carbon emission-related time series dataset, train the corresponding deep learning model, and associate and store the deep learning model with the text feature vectors and time series related data. Step two involves inputting descriptive text closely related to carbon emissions and time-series data related to carbon emissions. ,in express t Real-time carbon emission data t Take 1~ L , L The length of the time series is represented by prompt words. A large language model is used to extract feature vectors from descriptive text related to carbon emissions in the input data, specifically focusing on process steps. INPUT_E ; Step 3, Deep Learning Model Selection: From a pre-built vector database, select the deep learning model based on feature vectors. INPUT_ E For the query vector, a similarity method is used to retrieve the top-ranked texts based on their similarity. n For the retrieved data set, the similarity between its time-series data and carbon emission-related time-series data is further calculated. A similarity measurement algorithm suitable for time-series data is employed, comprehensively considering both textual similarity and time-series data similarity. A deep learning model is then selected to achieve knowledge transfer. The deep learning model selection process is as follows: 3.1 Text Similarity Calculation: Suppose the input text has feature vectors extracted using a large language model. INPUT_E The vector database contains text feature vectors corresponding to each data set. , Calculate the similarity between the input text and the text of each data set: ； Before selecting similarity values n Data group; 3.2 Time-series data similarity calculation: For the top-ranked data... n For each data set, extract its corresponding time series. , and the input time series The two sequences are compared, and the DTW algorithm is used to calculate the distance between them. Convert distance into similarity score : ； 3.3 Comprehensive Similarity Calculation and Deep Learning Model Selection: For each of the top n data groups, the average value of its text similarity and temporal similarity is calculated using the averaging method as the comprehensive similarity score of each data group. Finally, the deep learning model corresponding to the group with the highest comprehensive similarity score is selected for subsequent knowledge transfer training. Step 4, Model Training and Missing Value Imputation: On the selected deep learning model, train it using the input carbon emission-related time series data to complete the knowledge transfer process. Use the trained model to impute missing values in the time series data to improve data integrity and the accuracy of subsequent analysis.
2. The method for filling missing industrial carbon emission data based on knowledge transfer of process steps and time sequence similarity as described in claim 1, characterized in that, The model in step one has the following input, output, and training method: 1.1 Define the model input and output: Input time series data ,in B For batch size, L For sequence length, F For characteristic number, Data at each time step It consists of two parts: carbon emission-related data and time characteristics. Missing carbon emission data is filled with 0. Time series data X The output after processing by the deep learning model is the filling result for the missing parts of the carbon emission data; 1.2 Training Loss Function: A joint loss function is constructed based on time domain and frequency domain information, with time domain loss... Loss time use SmoothL 1. Function Measure Predicted Values Compared with the true value Differences: ； ； Frequency domain loss By performing Fourier transforms on the predicted and actual sequences, the frequency domain representation is obtained, and the mean absolute difference between the two is calculated: ； Where FFT(·) represents Fast Fourier Transform, and combining the above, the total loss function is... for ； in, is the scaling factor for the frequency domain loss; 1.3 Model Training: In each training cycle, the input data is first processed... X The middle part is masked, and the mask window size is set to the maximum window size for the model to predict missing values. The masked data is fed into the model defined in 1.1 to reconstruct the masked region and generate prediction results. The loss is calculated using the loss function defined in 1.
2. The calculated loss is backpropagated and the model parameters are updated using the optimizer. All steps in the training cycle are repeated until the pre-set training cycle is reached.
3. The method for filling missing industrial carbon emission data based on knowledge transfer of process steps and time sequence similarity as described in claim 1 or 2, characterized in that, Steps one and two involve using a large-scale pre-trained model to extract text feature vectors, as follows: 2.1 Text Preprocessing and Tokenization: Given the input text Text First, a word segmenter is used to split the text into segments. m One token, ； Each of them This represents a token in the text and corresponds to a unique token ID; 2.2 Token Embedding: Each token ID is mapped to a fixed-dimensional embedding vector, let the embedding matrix be... ,in VL It refers to the size of the vocabulary list. d If it is an embedded dimension, then for Its embedding vector is ； 2.3 Add location code: Let the location code be... Then the final input embedding is ； The entire sequence input is denoted as ； 2.4 Transformer Encoder Processing: Transforming the Embedded Sequence Z The input is fed into the Transformer encoder of the large language model. Each encoder layer processes the input using a self-attention mechanism, and its core computation is... ； Among them, query Q ,key K Sum V From input Z , For key K The vector length and the trainable weight matrix , , The operation is performed by a linear transformation to obtain: ； After passing through multiple encoders, the hidden representation of each token is obtained. ； in, Refers to the first j The hidden representation of each token; 2.5 Text Feature Vector Extraction: A pooling strategy is used to obtain a fixed-length representation of the entire text. For each dimension, the maximum value of all hidden states of the tokens is taken. ； in, INPUT_ This is the final extracted text feature vector.
4. The method for filling missing industrial carbon emission data based on knowledge transfer of process steps and time sequence similarity as described in claim 1 or 2, characterized in that, In step four, the model structure and training process remain consistent with the deep learning model constructed in step one. However, before training, the pre-trained weights of the selected deep learning model must be loaded to achieve effective knowledge transfer. After training, the model outputs... The missing data are used as candidate imputation data, and the corresponding prediction results are selected for imputation based on the location of the missing value.
5. A system implementing the industrial carbon emission data missing filling method based on knowledge transfer of process steps and temporal similarity as described in claim 1, characterized in that, The system includes a deep learning model training module, a vector database module, a text feature vector extraction module, and a deep learning model selection module. The deep learning model training module uses carbon emission-related time series data to train a deep learning model. The module performs masking processing on the input data and updates the model parameters based on the joint loss function, thereby enabling the model to reconstruct missing data. The module selects whether to load weights to adapt to the different requirements of steps one and four. The vector database module constructs and maintains a vector database that stores feature vectors extracted from descriptive text of each dataset and the corresponding pre-trained deep learning models. This module obtains data from open-source and private data sources to achieve efficient text feature storage and model association management, providing support for the rapid retrieval of subsequent deep learning models. The text feature vector extraction module first embeds the input text according to a predefined prompt word template, and then uses a large-scale pre-trained language model to preprocess the text, perform word segmentation, token embedding, positional encoding, and Transformer encoding to obtain a fixed-dimensional text feature vector. This module is used both to extract descriptive text feature vectors from various datasets when building a vector database, and to generate feature vectors from the descriptive text of the input data. INPUT_E This provides a precise basis for selecting deep learning models; The deep learning model selection module takes the input text feature vector as input. INPUT_E To query vectors, the top n data groups with the highest text similarity are retrieved from the vector database using cosine similarity. The DTW method is then used to calculate the temporal data similarity. The candidate data groups are evaluated by combining text and temporal similarity, and the deep learning model corresponding to the data group with the highest comprehensive score is selected for knowledge transfer.