A small-area data estimation method and system based on prior data fitting network

Abstract

The present application relates to the technical field of artificial intelligence data analysis, and more particularly to a small domain data estimation method and system based on prior data fitting network, wherein after the acquisition of the small domain data set, the multi-source heterogeneous characteristics in the small domain data set are first analyzed and aligned and converted into high-dimensional representation vectors, then the high-dimensional representation vectors are subjected to data missing detection, and according to the missing detection result, a learnable missing mask is embedded into the high-dimensional representation vectors to obtain context information; at the same time or before, the prior data fitting network in the target system is pre-trained; then the to-be-tested characteristics and the context information are spliced into a prompt sequence and input into the pre-trained prior data fitting network, and the posterior probability distribution of the to-be-tested characteristics in the preset discrete interval is output; finally, the posterior probability distribution is subjected to expectation decoding calculation to obtain the predicted value corresponding to the to-be-tested characteristics. The present application solves the problem of low prediction accuracy in the prior art for small domain estimation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence data analysis technology. More specifically, this invention relates to a small-domain data estimation method and system based on fitting a network with prior data. Background Technology

[0002] In the era of artificial intelligence, launching an AI prediction product for enterprises or public services often requires feeding and training with massive amounts of data samples. However, in many key sub-sectors, it is often impossible to obtain massive amounts of data samples in the initial launch or testing phase. This leads prediction products into the "small-domain estimation" problem. The "small domain" refers to scenarios where the sample size of the target subgroup is extremely scarce or even zero due to objective limitations (such as newly listed technology companies or newly commissioned microgrids). Existing technical solutions face the following significant technical bottlenecks when dealing with small-domain estimation:

[0003] Firstly, taking Chinese patent application CN120911819A as an example, its small-domain estimation mainly relies on the strict assumption of a normal distribution. This makes it difficult to capture complex nonlinear dynamic dependencies such as high-frequency fluctuations or sudden changes in energy load, thus limiting the expressive power of the parametric statistical model and leading to excessive bias or underfitting in the prediction results.

[0004] Secondly, tree ensemble models, such as XGBoost, are discriminative models. When faced with entirely new small-domain analysis tasks, they must collect data from scratch and retrain. Under extremely small sample constraints, they are still prone to overfitting and output meaningless inferences.

[0005] Third, in real-world applications, data is a mixture of "static attributes" and "dynamic time series," such as the financial statements and stock price curves of listed companies, and the transformer capacity and electricity consumption fluctuations of the power grid. Data in these scenarios is often accompanied by large areas of missing data. Existing technologies mostly use "mean interpolation" and "nearest neighbor interpolation" to preprocess missing values, which disrupts the underlying logic and causal relationships between heterogeneous data, leading to significant prediction errors.

[0006] In summary, existing technologies suffer from low prediction accuracy for small-domain estimation. Summary of the Invention

[0007] To address the technical problem of low prediction accuracy in small-domain estimation in the existing technologies, this invention discloses a small-domain data estimation method and system based on prior data fitting networks.

[0008] In a first aspect, the present invention discloses a small-domain data estimation method based on prior data fitting to a network, comprising: S10: In response to the acquisition of the small-domain dataset, parse the multi-source heterogeneous features in the small-domain dataset and align and convert them into high-dimensional representation vectors; S20: Perform missing data detection on the high-dimensional representation vector, and embed a learnable missing mask into the high-dimensional representation vector based on the missing detection results to obtain context information; S30: Pre-train the network that fits the prior data in the target system; S40: Concatenate the feature to be tested with the context information to form a prompt sequence, and input it into the pre-trained prior data fitting network to output the posterior probability distribution of the feature to be tested in the preset discrete interval; S50: Perform expectation decoding calculation on the posterior probability distribution to obtain the predicted value corresponding to the feature to be tested.

[0009] Through the above technical solution, the method of the present invention first aligns and converts multi-source heterogeneous features into high-dimensional representation vectors, and then embeds a learnable missing mask into the high-dimensional representation vectors. This replaces brute-force imputation with dynamic masking, maximizing the preservation of the true causal relationship of the original heterogeneous data and improving the information utilization rate and the accuracy of the final prediction result in the case of incomplete data. During or before this process, by pre-training the prior data fitting network, the cumbersome and high-risk problem of overfitting caused by the "collecting data-retraining" process in traditional small-domain estimation can be effectively overcome. Finally, the features to be tested and the context information are concatenated into a prompt sequence and input into the pre-trained prior data fitting network. The entire process mainly uses the posterior probability distribution to drive the generation of prediction values, avoiding the defects introduced by the normal distribution assumption in traditional schemes, thereby further improving the accuracy of the final prediction result.

[0010] Preferably, step S30 includes: The target system parameters are obtained by random sampling from a pre-defined prior hyperdistribution; A synthetic task with a synthetic dataset is generated using a pre-defined causal model; For each synthesis task, a support set is divided into a support set for simulating observation context information and a query set for simulating the target to be predicted; The support set and query set are concatenated and input into the prior data to fit the network. The classification cross-entropy loss between the network's predicted discrete interval probability and the true mapped label is calculated. The network parameters are updated using a preset optimization algorithm until training converges, at which point the network parameters are frozen.

[0011] The above technical solution enables the prior data fitting network to internalize the generalized causal reasoning ability and the inductive bias for handling extremely small samples into the weights of the network parameters, giving it the ability to perform posterior reasoning without fine-tuning.

[0012] Preferably, in step S40, the method of the present invention further includes: Dynamic pattern matching is performed by fitting prior data to the network's built-in alternating attention mechanism to compute the target conditional distribution; the target conditional distribution is used to constrain the posterior probability distribution.

[0013] Preferably, the alternating attention mechanism includes inter-feature self-attention and inter-sample self-attention; inter-feature self-attention and inter-sample self-attention are executed alternately during iterative training. The self-attention configuration between features is as follows: the input feature vector is mapped to a query, key, and value matrix, and the attention weights of each feature dimension within the matrix are calculated. The self-attention configuration between samples is as follows: combining the parameter matrix, the input feature vector is mapped to a query, key, and value matrix, and the attention weights of the sample dimension are calculated.

[0014] The method of this invention achieves the aggregation of information from features and samples by alternately performing inter-feature attention and inter-sample attention, and combines it with pre-trained prior data to fit the network, thereby performing context learning to promote the generation of more accurate prediction results.

[0015] Preferably, step S50 includes: The target value range of the network's final output is divided into several discrete intervals based on the prior data fitting. Obtain the predicted probability that the feature to be tested falls into each discrete interval; Extract the center value of each discrete interval, and sum the center value with the corresponding prediction probability to obtain the corresponding prediction value.

[0016] Preferably, the multi-source heterogeneous features include static attribute features and dynamic time-series signals; step S10 includes: Batch normalization is performed on continuous numerical features in static attribute features to obtain normalized vectors; embedding mapping is performed on discrete category features in static attribute features to obtain mapping vectors; a multi-harmonic Fourier transform is used to extract the frequency energy of dynamic time-series signals, construct high-frequency enhancement vectors, and compress the high-frequency enhancement vectors into fixed-length time-series vectors. Assign a corresponding learnable semantic vector to each feature dimension in the multi-source heterogeneous features; By concatenating the normalized vector, the mapping vector, and the time-series vector with the corresponding learnable semantic vector along the channel dimension and aligning them in a unified space, a high-dimensional representation vector is obtained.

[0017] The above technical solutions enable the prior data to fit the network and understand the actual business meaning behind the features, making the prediction results more accurate and reliable.

[0018] Preferably, step S20 includes: Construct a missing indicator matrix; Detect element features in high-dimensional representation vectors; Identify whether an element feature is a valid observation. If so, mark the corresponding element position in the missing indicator matrix as 1; otherwise, mark it as 0. Initialize a learnable missing mask to characterize the attributes in state 0 of the missing indication matrix; The learnable semantic vector and the learnable missing mask are concatenated and fused to obtain contextual information that preserves the deep features of the high-dimensional representation vector.

[0019] Compared with existing technologies, the above technical solution can overcome the technical defects of existing brute-force interpolation methods and preserve the true causal relationship of the original heterogeneous data to the greatest extent.

[0020] Preferably, the state vector used to represent context information is:

[0021] In the formula, Indicates the first In the nth sample A state vector of each feature; The `Encoder` directive represents the feature fusion operator; `Encoder` represents the feature alignment network. Indicates the missing indicator matrix; Represents the deep features of a high-dimensional representation vector; Indicates a learnable missing mask; This represents a learnable semantic vector.

[0022] Preferably, the method of the present invention is applicable to power grid data forecasting, financial data forecasting, or industrial Internet of Things (IoT) data forecasting.

[0023] Secondly, the present invention also provides a small-domain data estimation system based on a prior data fitting network, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the small-domain data estimation method based on a prior data fitting network described in the first aspect is implemented.

[0024] Compared with the prior art, the beneficial effects of the present invention are mainly reflected in the following aspects: First, the method of this invention overcomes the cumbersome and high-risk overfitting problem of traditional small-domain estimation's "data collection-retraining" process by establishing a prior data fitting network. During data processing, the above scheme can transform the inference paradigm into a contextual learning approach based on Bayesian posterior probability inference, effectively solving the problems of unreliable inference results and model collapse under extremely small sample sizes.

[0025] Secondly, the method of this invention achieves dynamic coupling of observed data through an alternating attention mechanism. During inference, it neither blindly applies global, macroscopic general laws nor is it affected by noise and fluctuations in individual small-domain data. Instead, it maximizes the exploration of correlations between data points, greatly improving the robustness of predictions in small-domain sample cases.

[0026] Third, the method of this invention learns by combining static attributes and dynamic time-series information. Then, by inserting a dynamic mask into the missing data to replace the brute-force imputation scheme in the prior art, this method preserves the true causal relationship of the original heterogeneous data to the greatest extent, improves the information utilization rate under incomplete data, and makes the prediction results more accurate. Attached Figure Description

[0027] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 This is a flowchart of the small-domain data estimation method based on prior data fitting network in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram illustrating the specific process of the small-domain data estimation method based on prior data fitting network in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the context learning process in Embodiment 1 of the present invention; Figure 4 This is a time-series data prediction graph used for microgrid prediction in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of the small-domain data estimation system based on prior data fitting network in Embodiment 3 of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] Example 1 like Figure 1 As shown, this embodiment discloses a small-domain data estimation method based on prior data fitting to a network, including: S10: In response to the acquisition of the small domain dataset, parse the multi-source heterogeneous features in the small domain dataset and align and convert them into high-dimensional representation vectors.

[0031] It should be noted that the aforementioned small-domain dataset can be understood as a small number of data samples, each consisting of original heterogeneous data, including static attribute features (numerical and categorical) and dynamic time-series signals. The term "small-domain dataset" is relative; the data samples required in this embodiment may be only one-twentieth to one-tenth of those required by traditional techniques.

[0032] Furthermore, step S10 specifically includes: S11: Perform batch normalization on the continuous numerical features in the static attribute features to obtain the normalized vector.

[0033] For continuous numerical features, in one embodiment, batch normalization can be performed along the feature dimension. Specifically, let n be the index of the sample dimension and m be the index of the feature dimension. Let m be the m-th original numerical feature in the n-th sample, and then batch normalization is achieved using the following formula:

[0034] In the formula, This represents the m-th normalized value in the n-th sample; This represents the sample mean of the m-th feature dimension; Let represent the variance of the m-th feature dimension; This represents a constant that maintains numerical stability.

[0035] S12: Perform an embedding mapping operation on the discrete category features in the static attribute features to obtain the mapping vector.

[0036] For discrete category features, in one embodiment, they can be directly mapped into feature vectors through a learnable embedding layer based on discrete category attributes, such as industry type name, analysis project name, or analysis node. .

[0037] S13: A multi-harmonic Fourier transform is used to extract the frequency energy of the dynamic time-series signal, construct a high-frequency enhancement vector, and compress the high-frequency enhancement vector into a time-series vector of fixed length.

[0038] To address the aforementioned dynamic time-series signals and ensure the network can capture high-frequency dynamic changes, in one embodiment, the data features corresponding to each timestamp can be mapped to a permutation-invariant high-dimensional orthogonal space. Then, a multi-harmonic Fourier transform is used to extract the frequency energy of the time series, thereby constructing a high-frequency enhancement vector. A more specific algorithm is as follows:

[0039] In the formula, Represents the high-frequency increment vector at time t; This represents the different signal frequencies obtained by Fourier transform in a time-series signal; k represents the number of frequencies. This represents a 2k-dimensional real vector space; the above formula compresses this variable-length frequency domain vector into a time-series vector of length 64 through multiple convolutional and linear layers.

[0040] S14: Assign a corresponding learnable semantic vector to each feature dimension in the multi-source heterogeneous features.

[0041] Specifically, in the attribute semantic encoding process, this step mainly assigns an independent and learnable semantic vector to each feature dimension, with a length defined as 64. This learnable semantic vector is used to represent the actual meaning represented by the feature dimension.

[0042] S15: The normalized vector, the mapping vector, and the time-series vector are concatenated along the channel dimension and aligned in a unified space with the corresponding learnable semantic vector to obtain a high-dimensional representation vector.

[0043] Specifically, the normalized vector obtained in step S11, the mapping vector obtained in step S12, and the temporal vector obtained in step S13 are mapped to a vector of length 64 through a linear layer. These vectors are then concatenated with their corresponding semantic vectors along the feature channel dimension to obtain a vector of length 128, which serves as the high-dimensional representation vector. Next, the concatenated 128-dimensional feature vector is compressed into a unified mapping space of length 64 through a 1×1 convolutional layer, denoted as the output deep feature representation. Finally, all feature representations corresponding to the same sample are aggregated to output a unified-dimensional feature vector representing the global information of the nth sample. This vector will be directly used as an input component for subsequent networks and context sequences.

[0044] Through the above steps S11-S15, the method of this embodiment can learn by combining static attributes and dynamic time-series information, and retain the true correlation of the original heterogeneous data by vector concatenation.

[0045] S20: Perform missing data detection on the high-dimensional representation vector, and embed a learnable missing mask into the high-dimensional representation vector based on the missing detection results to obtain context information.

[0046] Furthermore, step S20 above includes: S21: Construct the missing indicator matrix.

[0047] In this embodiment, the missing indicator matrix is ​​a Boolean missing indicator matrix type, and its symbol is represented as follows: .

[0048] S22: Detect element features in high-dimensional representation vectors.

[0049] In this embodiment, the main focus is on detecting whether there are missing values ​​in the matrix elements.

[0050] S23: Identify whether the element feature is a valid observation. If so, mark the corresponding element position in the missing indicator matrix as 1; otherwise, mark it as 0.

[0051] More specifically, when the m-th numerical feature of the n-th sample is a valid observation, it is marked as... Conversely, when data is missing or cannot be identified for other reasons, it is marked as... .

[0052] S24: Initialize the learnable missing mask to characterize the attributes in state 0 of the missing indication matrix.

[0053] In one embodiment, a globally shared, stochastic gradient-learnable missing mask can be initialized in the model. Here, C is set to 64, indicating that the mask is a vector of length 64. This learnable missing mask participates in attention calculations and is continuously updated during network iterations, representing the property that the data is in an unknown state.

[0054] S25: The learnable semantic vector and the learnable missing mask are concatenated and fused to obtain contextual information that retains the deep features of the high-dimensional representation vector.

[0055] The state vector used to represent context information is:

[0056] In the formula, Indicates the first In the nth sample A state vector of each feature; Represents the feature fusion operator; Encoder represents the feature alignment network, belonging to 1. 1 convolutional layer; Indicates the missing indicator matrix; Represents the deep features of a high-dimensional representation vector; Indicates a learnable missing mask; This represents a learnable semantic vector.

[0057] S30: Pre-train the network that fits the prior data in the target system.

[0058] It should be noted that, in combination Figure 2It can be seen that step S30 is not necessarily a sequential step of step S20. The execution of step S30 is not affected during the execution of steps S10-S20, or before the execution of step S20.

[0059] Furthermore, before performing context learning for specific small-domain tasks, the method in this embodiment needs to fit the network parameters to the prior data in a high-performance computing cluster. Offline pre-training is performed, a process that fully internalizes general causal reasoning and regularization logic into the model through meta-learning of synthetic data. Therefore, step S30 above should include: S31: Randomly sample the target system parameters from the preset prior hyperdistribution.

[0060] It should be explained that, in order to ensure the model has the ability to fit almost any distribution, It should cover as many families of mathematical models as possible, which may include Gaussian processes with different kernel functions, decision trees with random topological structures, neural networks with nonlinear activation functions, and heteroscedastic noise models, etc.

[0061] S32: Generate a synthetic task with a synthetic dataset using a pre-defined causal model.

[0062] The expression for the above synthesis task is as follows:

[0063] In the formula, This represents the k-th synthesis task set; Symbol indicating obedience; Represents input features; Indicates output features; Indicates a condition symbol.

[0064] S33: For each synthetic task, divide the support set for simulating observation context information and the query set for simulating the target to be predicted.

[0065] Specifically, in combination Figure 3 As shown, the method in this embodiment simulates the real inference scenario of small-domain estimation and randomly divides the above-mentioned synthetic task set into two independent sets.

[0066] Further, extraction The support set consists of 10 sample points, and its expression is:

[0067] In the formula, Indicates a support set; This represents the input vector corresponding to the i-th sample; This represents the output vector corresponding to the i-th sample.

[0068] Furthermore, extraction The query set consists of 10 sample points, and its expression is:

[0069] In the formula, Represents a query set; This represents the q-th query feature; This represents the true label of the q-th query feature.

[0070] S34: After concatenating the support set and query set, input them into the prior data to fit the network, and calculate the classification cross-entropy loss between the network's predicted discrete interval probabilities and the true mapped labels.

[0071] Specifically, the above steps involve concatenating the support set and query features and inputting them into the prior data fitting network. At this point, the prior data fitting network does not directly output a single, definitive value; instead, it divides the range of the predicted target's value into B discrete intervals. The prior data fitting network outputs the probability distribution of the predicted sample belonging to each discrete interval, and then uses the probability distribution and the true mapping label to calculate the classification cross-entropy loss.

[0072] In the formula, This indicates that the true label of the query feature is at the q-th sample point. The probability of each interval; B represents the upper bound of q; The upper bound; Indicates based on query features and support set The prediction network mapping function, which is used to characterize the network parameters under the current network conditions. Below, infer the predicted probability that the target value of the sample point falls into the b-th discrete interval; the two external expectations of the classification cross-entropy loss. This indicates that the loss function is based on different logical rules. The expected summation is performed on the combined synthetic task set.

[0073] S35: Update network parameters using a preset optimization algorithm until training converges, then freeze the network parameters.

[0074] Preferably, in step S35 above, the AdamW optimization algorithm can be used to calculate the gradient of the loss function with respect to the network parameters and continuously update the network parameters. Minimizing the expected cross-entropy loss on a large-scale, diverse set of synthetic tasks, the output distribution of the neural network will be optimized when training converges. This will mathematically approximate the true posterior Bayesian integral of the target:

[0075] Through steps S31-S35 above, the prior data fitting network successfully internalizes the generalized causal reasoning ability and the inductive bias for handling extremely small samples into the weights of the parameters, enabling it to perform posterior reasoning without fine-tuning.

[0076] S40: Concatenate the feature to be tested with the context information to form a prompt sequence, and input it into the pre-trained prior data fitting network to output the posterior probability distribution of the feature to be tested in the preset discrete interval.

[0077] In this process, to aggregate different pieces of information, a prior data fitting network with L layers of alternating attention is needed, performing orthogonal self-attention calculations sequentially at each layer. Therefore, the method in this embodiment also requires dynamic pattern matching using the alternating attention mechanism built into the prior data fitting network to calculate the target conditional distribution. The target conditional distribution is used to constrain the posterior probability distribution.

[0078] Furthermore, the aforementioned alternating attention mechanism includes feature-based self-attention and sample-based self-attention. It is important to note that feature-based self-attention and sample-based self-attention are performed alternately during iterative training.

[0079] The self-attention configuration between features is as follows: The input feature vector is mapped to a query, key, and value matrix, and the attention weights of each feature dimension within the matrix are calculated.

[0080] More specifically, computation is performed independently within each feature along the feature dimension. The input feature vector is mapped to a query (Q), key (K), value (V) matrix using a parameter matrix, and then attention weights for each feature dimension are calculated. This process essentially simulates causal reasoning, learning underlying logical connections by uncovering the dependencies between different features within a single sample.

[0081] The inter-sample self-attention configuration is as follows: By combining the parameter matrix, the input feature vector is mapped into a query, key, and value matrix, and the attention weights of the sample dimension are calculated.

[0082] More specifically, along the sample dimension Calculations are performed between different features using a parameter matrix. New Q, K, and V matrices are generated, and then the attention weights for the sample dimensions are calculated. This process essentially involves small-domain data integration, that is, learning the relationships between different samples to achieve information borrowing and global aggregation.

[0083] The above technical solution enables the aggregation of information between features and samples by alternately executing attention between features and attention between samples.

[0084] Returning to step S40, the above process and its network integration enable the model to perform forward inference directly using the frozen network when faced with a new, small-domain task with a small sample size, without any gradient updates. Specifically, this includes the following steps: S41: Perform the concatenation of the prompt sequence.

[0085] Specifically, this is achieved by collecting an extremely limited set of observed data from the current target domain as a context set. At the same time, obtain the current features to be tested. To adapt to the network sequence input format, they are concatenated along the sample dimension to form a Prompt sequence:

[0086] In the formula, Mask represents a specific missing value mask, that is, a learnable missing value mask for the actual input sample, which is used to indicate the current missing value of the model output. corresponding This is a state to be predicted.

[0087] S42: In response to the input of the cue sequence, perform alternating attention matching to obtain the posterior probability distribution.

[0088] Specifically, the assembled Prompt sequence is input into the model network. In the middle. Alternating attention within the network invokes internalized prior rules regarding the current context. Dynamic pattern matching is performed to calculate the target condition distribution, and the specific calculation formula is as follows:

[0089] Then, through forward computation, the network extracts hidden mapping rules from the context and integrates the information into the deep representation of the feature to be tested, thereby directly outputting the posterior probability distribution of the target value in the preset space.

[0090] S50: Perform expectation decoding calculation on the posterior probability distribution to obtain the predicted value corresponding to the feature to be tested.

[0091] Specifically, step S50 includes: S51: Divide the target value range of the network's final output into several discrete intervals.

[0092] It should be explained that at the end of the network, the classifier no longer outputs a single, definite value, but instead outputs a probability distribution sequence of the target falling within B discrete intervals.

[0093] S52: Obtain the predicted probability that the feature to be tested falls into each discrete interval.

[0094] In this embodiment, let the b-th discrete interval be... At this point, the probability of the target falling into this interval, as output by the model, is... And satisfy .

[0095] S53: Extract the center value of each discrete interval, and sum the center value with the corresponding prediction probability to obtain the corresponding prediction value.

[0096] Specifically, the formula for step S53 above is:

[0097] In the formula, the formula for calculating the center value is: ; This represents the predicted value.

[0098] Combining steps S10-S50 above, the system developed based on the method of this embodiment can achieve high-precision end-to-end prediction of small-domain estimation under zero-fine-tuning conditions. The above solution is particularly suitable for power grid data prediction, financial data prediction, or industrial IoT data prediction.

[0099] Unlike existing technologies, the method in this embodiment has the following technical advantages: Firstly, the method in this embodiment effectively avoids the cumbersome process and the risk of overfitting inherent in traditional small-domain estimation by constructing a prior data fitting network. In the data processing stage, this embodiment can convert the original inference model into a contextual learning mechanism based on Bayesian posterior probability, thereby properly resolving the problems of distorted inference conclusions and model instability when the sample size is extremely scarce.

[0100] Secondly, the method in this embodiment achieves dynamic correlation and integration of observation data by leveraging an alternating attention mechanism. When performing inference, the above method neither mechanically applies global macroscopic universal laws nor is it constrained by noise disturbances and random fluctuations within a single small data domain; on the contrary, it fully explores the inherent connections between various data, significantly enhancing the robustness of predictions under small-domain sample conditions.

[0101] Thirdly, the method in this embodiment combines static attributes with dynamic time-series information for learning, and adopts a strategy of applying dynamic masks to missing data, replacing the simple and crude imputation method in the prior art. The above scheme maintains the inherent true causal relationship of the original heterogeneous data to the greatest extent, improves the efficiency of information utilization under incomplete data conditions, and thus improves the accuracy of prediction results.

[0102] Fourthly, the method in this embodiment has superior prediction accuracy in multiple scenarios such as power grid prediction, industrial IoT prediction, and macro-level financial data prediction. It can achieve accurate estimation of small domains across different fields and help the decision-making system achieve standardization.

[0103] Example 2 Based on Example 1, this example is used for power prediction in microgrids.

[0104] Specifically, this embodiment selects a "microgrid in a newly built industrial park that has been in operation for two weeks" as the target microdomain. Because the park is newly built and operational, electricity consumption data is extremely scarce, with only a few short, sporadic records available, and data gaps exist during the initial meter sensor debugging phase. In such scenarios, using traditional time-series forecasting models (such as LSTM) can easily lead to severe overfitting due to the small sample size, and the discontinuous input sequence makes it difficult to provide a reliable electricity load forecast.

[0105] In contrast, this embodiment aims to utilize a "prior data fitting network" with frozen parameters to accurately predict the park's peak electricity consumption tomorrow through contextual learning, without zero fine-tuning or model training.

[0106] Based on the method provided in Example 1, for microgrid prediction applications, the first step is to perform feature processing and unified mapping on the collected microgrid context data and the target to be measured: (1) First, set the target to be measured as "the peak electricity load of the park tomorrow (unit kW)". When constructing the input Prompt sequence, the position corresponding to this target is marked as a missing value mask.

[0107] (2) In the static attribute preprocessing and semantic fusion stage, the static features of each park are first obtained, including numerical features, such as the total transformer capacity of 5000kVA, the park size of 500 people and output value of 200 million yuan, as well as categorical features, such as high energy consumption manufacturing industry.

[0108] (3) Perform adaptive normalization on numerical features to eliminate dimensional differences, and perform embedding encoding on categorical features, and then uniformly map them into vectors of length 64 through a linear layer.

[0109] (4) Assign an independent learnable semantic vector of length 64 to the business attribute of each of the above features. Concatenate the vector containing the actual numerical value with the semantic vector containing the attribute meaning by channel dimension, and then compress it back to 64 dimensions through a feature alignment network. This enables the model to understand both the magnitude of the numerical value and the actual meaning behind the numerical value.

[0110] (5) In the dynamic time series signal extraction stage, the intermittent "temperature fluctuation curve" and "load fluctuation curve" collected in the past 3 days are used. Through multi-harmonic semantic projection and combined with Fourier transform, these variable-length, high-frequency fluctuating time series curves are mapped into high-dimensional orthogonal vectors of uniform length, thereby accurately extracting the frequency energy characteristics of temperature and load linkage.

[0111] (6) During the embedding stage of the dynamic missing mask, for the blank electricity consumption data of a certain number of hours caused by the initial sensor debugging or failure in the historical data, the initialized learnable missing mask is called to perform state embedding and splicing. This mechanism enables the network to recognize the objective fact that "the data of this period is in an unknown state", thus perfectly preserving the true underlying logic and causal relationship of the time series data.

[0112] It should be explained that, as a preferred strategy, the system was trained on a large number of high-performance computing clusters during the offline meta-learning phase before deployment, enabling the network parameters to internalize the general causal laws of power load changes. When facing this novel microgrid task, the following online inference logic is executed: (1) Collect extremely limited observation samples of the target microgrid over the past 3 days, such as daytime characteristics and corresponding peak power, and forecast characteristics for the 4th day, and splice them into a Prompt sequence based on the above splicing mechanism: Prompt=[(x1,y1),(x2,y2),(x3,y3),(x q Mask)] Among them, x1, x2, and x3 contain feature vectors such as temperature curves for the first three days, transformer capacity, and output scale; y1, y2, and y3 are the measured peak loads, such as 1200kW, 1350kW, and 1290kW, respectively; x q [Mask] represents the forecast feature vector for tomorrow; [Mask] is used to indicate the load y corresponding to the model output. q .

[0113] (2) During the alternating attention mechanism process, the above sequence is input into the frozen network.

[0114] (3) In the process of the attention mechanism between features, the strong dependency between the temperature curve and the electricity consumption is quickly discovered within each x, and the potential rule that “due to the high temperature, the heat dissipation energy consumption of the equipment increases, thus significantly increasing the total electricity load” is identified.

[0115] (4) During the inter-sample attention mechanism processing stage, the model is in the process of x1 to x q By comparing the data horizontally, the power consumption habits of the power grid observed in the previous 3 days are matched with the internal prior rules of the pre-training to calculate the distribution of the target conditions.

[0116] Finally, in the segmented probability output and expected decoding stage: The classifier at the end of the network divides the electricity consumption prediction space into B discrete intervals, for example, each interval spanning 100kW. For x... q The model outputs the probability distribution sequence of the three core intervals into which the target falls: Interval 1: [1300, 1400] (center value c1=1350), probability 15%; Interval 2: [1400, 1500] (center value c2=1450), probability 75%; Interval 3: [1500, 1600] (center value c3=1550), probability 10%.

[0117] Then, by weighted summation of the probability density functions, the final exact continuous prediction value is calculated, and its application to the instance is expressed as follows: .

[0118] Ultimately, combining Figure 4 As shown in the forecast section to the right of the vertical dashed line, the system equipped with the method of Embodiment 1 will output a forecast report to the dispatch center of the park power grid. The report will predict the peak load for tomorrow to be 1445 kW, and will also include time-series data forecasts. Figure 1 Same as output.

[0119] Example 3 like Figure 5 As shown, this embodiment also provides a small-domain data estimation system based on prior data fitting network, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the small-domain data estimation method based on prior data fitting network described in the embodiment is implemented.

[0120] The system in this embodiment also includes other components well known to those skilled in the art, such as communication interfaces. Their settings and functions are known in the art, and therefore will not be described in detail here.

[0121] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A method for estimating small-domain data based on fitting a network with prior data, characterized in that, Includes the following steps: S10: In response to the acquisition of microgrid power data, the multi-source heterogeneous features in the microgrid power data are parsed and aligned into a high-dimensional representation vector; the multi-source heterogeneous features include static attribute features and dynamic time-series signals; step S10 includes: performing batch normalization on the continuous numerical features in the static attribute features to obtain a normalized vector; performing embedding mapping on the discrete category features in the static attribute features to obtain a mapping vector; for the temperature fluctuation curve and load fluctuation curve of the microgrid, using a multi-harmonic Fourier transform to extract the frequency energy of the dynamic time-series signal, constructing a high-frequency enhancement vector, and compressing the high-frequency enhancement vector into a fixed-length time-series vector; assigning a corresponding learnable semantic vector to each feature dimension in the multi-source heterogeneous features; concatenating the normalized vector, the mapping vector, and the time-series vector with the corresponding learnable semantic vector along the channel dimension and aligning them in a unified space to obtain the high-dimensional representation vector; S20: Perform data missing detection on the high-dimensional representation vector, and embed a learnable missing mask into the high-dimensional representation vector according to the missing detection result to obtain context information; step S20 includes: constructing a missing indicator matrix; detecting element features in the high-dimensional representation vector; identifying whether the element features are valid observations, if so, marking the corresponding element position in the missing indicator matrix as 1, otherwise marking it as 0; initializing a learnable missing mask to represent the attributes in the missing indicator matrix that are in state 0; concatenating the learnable semantic vector with the learnable missing mask to obtain context information that retains the deep features of the high-dimensional representation vector; S30: Pre-train the network that fits the prior data in the target system; S40: Concatenate the feature to be tested with the context information to form a prompt sequence, and input it into the pre-trained prior data fitting network to output the posterior probability distribution of the feature to be tested in a preset discrete interval; the feature to be tested is tomorrow's peak load. S50: Perform expectation decoding calculation on the posterior probability distribution to obtain the predicted value corresponding to the feature to be tested.

2. The small-domain data estimation method based on prior data fitting network according to claim 1, characterized in that, Step S30 includes: The target system parameters are obtained by random sampling from a pre-defined prior hyperdistribution; A synthetic task with a synthetic dataset is generated using a pre-defined causal model; For each synthesis task, a support set is divided into a support set for simulating the context information observed and a query set for simulating the target to be predicted; The support set and the query set are concatenated and input into the prior data fitting network to calculate the classification cross-entropy loss between the network's predicted discrete interval probability and the true mapped label. The network parameters are updated using a preset optimization algorithm until training converges, and then the network parameters are frozen.

3. The small-domain data estimation method based on prior data fitting network according to claim 1, characterized in that, In step S40, the method further includes: Dynamic pattern matching is performed by fitting the prior data to the network's built-in alternating attention mechanism to calculate the target conditional distribution; the target conditional distribution is used to constrain the posterior probability distribution.

4. The small-domain data estimation method based on prior data fitting network according to claim 3, characterized in that, The alternating attention mechanism includes inter-feature self-attention and inter-sample self-attention; the inter-feature self-attention and the inter-sample self-attention are executed alternately during iterative training. The self-attention configuration between features is as follows: the input feature vector is mapped into a query, key, and value matrix, and the attention weights of each feature dimension within the matrix are calculated. The inter-sample self-attention configuration is as follows: combining the parameter matrix, the input feature vector is mapped into a query, key, and value matrix, and the attention weights of the sample dimensions are calculated.

5. The small-domain data estimation method based on prior data fitting network according to claim 1, characterized in that, Step S50 includes: The target value range of the network's final output is divided into several discrete intervals based on the prior data fitting. Obtain the predicted probability that the feature to be tested falls into each discrete interval; Extract the center value of each discrete interval, and sum the center value with the corresponding prediction probability to obtain the corresponding prediction value.

6. The small-domain data estimation method based on prior data fitting network according to claim 1, characterized in that, The state vector used to represent contextual information is: In the formula, Indicates the first In the nth sample A state vector of each feature; Indicates feature fusion operators; Encoder represents the feature alignment network; This represents the missing indicator matrix; This represents the deep features of the high-dimensional representation vector; This represents the learnable missing mask; This represents the learnable semantic vector.

7. A small-domain data estimation system based on a prior data fitting network, characterized in that, It includes a processor and a memory, the memory storing computer program instructions, which, when executed by the processor, implement the small-domain data estimation method based on prior data fitting network as described in any one of claims 1-6.

Images