Small sample deep learning method based on mathematical expectation

Abstract

The invention provides a small sample deep learning method based on mathematical expectation. The method comprises the steps of obtaining a training set; wherein the training set comprises sample subset pairs and conditional expectations corresponding to the sample subsets; reconstructing the format of the to-be-predicted sample, and enabling the reconstructed format of the to-be-predicted sample to be the same as the structure of the sample subset pair; inputting the reconstructed prediction sample into the trained model to obtain a prediction result; the method at least comprises the following steps of obtaining a trained model: training a to-be-trained model based on a training set; according to the method, the sample subset and the condition expectation corresponding to the sample subset are used as the training set, the number of training samples is increased in this way, the problem of insufficient training data volume is solved, and therefore the performance and effect of a small sample data model are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and specifically provides a small sample deep learning method based on mathematical expectation. Background Art

[0002] Nowadays, artificial intelligence and deep learning have achieved significant results in academia and industry, but the three key elements required by current deep learning methods are algorithms, computing power, and data. Algorithms and computing power have developed rapidly in recent years, and a variety of neural network models have emerged, such as: MLP, CNN, RNN, GAN, Transformer, Diffusion Model. These models have achieved excellent results in various fields, but without exception, they all require a large amount of high-quality training data. Moreover, traditional methods fit the mapping relationship between samples, and the effect of small sample learning is very poor. For this reason, deep learning has not yet achieved very excellent results in fluid numerical simulation in the absence of labeled data.

[0003] Therefore, a new small sample training scheme is needed in this field to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects, the present invention is proposed to provide a solution or at least partially solve the problem of poor learning effect of small samples.

[0005] The present invention provides a small sample deep learning method based on mathematical expectation, comprising: obtaining a training set; wherein the training set includes sample subset pairs and conditional expectations corresponding to the sample subsets; reconstructing the format of the sample to be predicted so that the format of the reconstructed sample to be predicted is the same as the structure of the sample subset pairs; inputting the reconstructed prediction sample into a trained model to obtain a prediction result; the method comprises at least the following steps to obtain the trained model: training the model to be trained based on the training set.

[0006] In a technical solution of the above-mentioned small sample deep learning method based on mathematical expectation, the method includes at least the following steps to obtain sample subset pairs: generating several sample subsets according to the permutation and combination method of the selected small sample set, each sample subset containing several sample subset pairs.

[0007] In a technical solution of the above-mentioned small sample deep learning method based on mathematical expectation, the method includes at least the following steps to obtain the conditional expectation corresponding to the sample subset: determining the label set corresponding to the selected small sample set; calculating the conditional expectation of the sample subset to the corresponding label set in the label set.

[0008] In a technical solution of the above-mentioned small sample deep learning method based on mathematical expectation, the process of training the to-be-trained model based on the training set includes: obtaining the mapping relationship between the sample subset pairs and the conditional expectations corresponding to the sample subsets; based on the mapping relationship, taking the sample subset pairs as input and the conditional expectations corresponding to the sample subsets as output to train the to-be-trained model.

[0009] In a technical solution of the above-mentioned small sample deep learning method based on mathematical expectation, the process of obtaining the mapping relationship between the sample subset pairs and the conditional expectations corresponding to the sample subsets includes: obtaining the mapping relationship based on a fitting method.

[0010] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0011] In the technical solution for implementing the present invention, the present invention provides a small sample deep learning method based on mathematical expectation, including: obtaining a training set; wherein the training set includes sample subset pairs and conditional expectations corresponding to the sample subsets; reconstructing the format of the sample to be predicted so that the format of the reconstructed sample to be predicted is the same as the structure of the sample subset pairs; inputting the reconstructed prediction sample into the trained model to obtain a prediction result; the method at least includes the following steps to obtain the trained model: training the model to be trained based on the training set. Compared with the prior art, the beneficial effect of the small sample deep learning method based on mathematical expectation provided by the present invention is: the method uses sample subsets and conditional expectations corresponding to sample subsets as training sets, in this way increasing the number of training samples, solving the problem of insufficient training data, and thus improving the performance and effect of the small sample data model.

[0012] Furthermore, by generating sample subset pairs and calculating the corresponding conditional expectations, the model to be trained can access the distribution information of different parts of the data set. The combination of these sample subset pairs and conditional expectations represents various possible local conditions in the data set and the corresponding expected results, thereby indirectly incorporating the distribution characteristics of the data set into the training process. In addition, the mapping relationship between sample subset pairs and conditional expectations is learned during the training process. Since this mapping relationship is constructed based on training data containing distribution information, the neural network is actually learning how to make reasonable predictions based on the distribution characteristics of the data, thereby reducing the distribution uncertainty of the data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the drawings represent similar components, wherein:

[0014] Figure 1is a flowchart of main steps of a small sample deep learning method based on mathematical expectation according to an embodiment of the present invention;

[0015] Figure 2 It is a comparison diagram of loss functions when the original data is reconstructed using the method and then put into the same FCN model for training and testing according to an embodiment of the present invention;

[0016] Figure 3 This is a distribution diagram of prediction accuracy of a test set after the original data is reconstructed using the method according to an embodiment of the present invention and then put into the same FCN model for training. DETAILED DESCRIPTION

[0017] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0018] Embodiment 1

[0019] See attached Figure 1 , Figure 1 1 is a flow chart of the main steps of a small sample deep learning method based on mathematical expectation according to an embodiment of the present invention. Figure 1 As shown, the small sample deep learning method based on mathematical expectation in the embodiment of the present invention mainly includes the following steps S1 to S3.

[0020] Step S1, obtaining a training set; wherein the training set includes sample subset pairs and conditional expectations corresponding to the sample subsets.

[0021] In this embodiment, sample subset pairs can be constructed in a variety of ways. A common way is to divide based on the inherent logical relationship of the data. For example, in fluid mechanics simulation data, according to the physical process, the initial state parameters of the fluid (such as initial velocity, initial pressure, initial density, etc.) are taken as one part, and the state parameters after a certain time or a certain physical process (such as final velocity, final pressure, etc.) are taken as another part, thereby forming a sample subset pair. Another way is to form a sample subset pair through a data preprocessing method, such as grouping and sorting the data according to certain rules. The main purpose of constructing a sample subset pair is to better capture the relationship and change law between the data. By combining related data together, the model (such as a neural network) can learn the interaction between such data, rather than just focusing on the characteristics of a single data point. This can improve the model's ability to understand and predict data, especially for some complex data relationships and dynamically changing scenes. For each sample subset, under these initial conditions, the expected average value of certain physical quantities after the fluid has passed a period of time or a specific physical process is the conditional expectation corresponding to the sample subset.

[0022] Step S2: reconstruct the format of the sample to be predicted so that the format of the reconstructed sample to be predicted is the same as the structure of the sample subset pair.

[0023] In this embodiment, in order to ensure that the sample to be predicted and the training data are consistent in form and to avoid the model being unable to effectively use previously learned knowledge due to different data formats, the format of the sample to be predicted is reconstructed. Assume that in the training set of fluid mechanics data, the sample subset pair is a two-dimensional structure, in which one dimension contains the initial physical parameters of the fluid (such as initial flow rate, initial pressure, initial density), and the other dimension contains the physical parameters of the fluid after a certain process (such as final flow rate, final pressure).

[0024] For example, suppose a traditional deep learning model learns X i →y i , i∈1, 2, ..., n, and construct w by building data pairs i =(X i1 , …, X ik ) sample subset pairs, and then fit w i →E(y|w i )This mapping relationship.

[0025] in:

[0026]

[0027] The final result can be written as If you want to predict a new sample X iThen we only need to construct a subset w with the same components j =(X j , …, X j )

[0028] Step S3, input the reconstructed prediction sample into the trained model to obtain the prediction result; the method at least includes the following steps to obtain the trained model: training the model to be trained based on the training set.

[0029] In one embodiment, the method at least comprises the following steps to obtain the sample subset pairs: generating a plurality of sample subsets according to the permutation and combination of the selected small sample sets, each sample subset containing a plurality of sample subset pairs. i , 1≤i≤n}, generated according to permutations and combinations The sample subset w i , where w i is a subset pair containing k samples. Then the conditional expectation of the subset pair corresponding to the labeled data set is E(y i |ω i ), combined into a new training set (w i , E(y i |w i )).

[0030] Alternatively, in an alternative method, a random sampling method can be used to obtain sample subset pairs; or a cluster analysis method can be used, specifically, the sample set is first clustered to cluster similar samples into different categories. For example, in a fluid numerical simulation sample set, the simulation data is clustered into several different clusters according to fluid characteristics such as flow rate, pressure, and temperature through a clustering algorithm (such as K-Means clustering). Then, some samples are selected from each cluster to form a sample subset. The selection method can be random selection, or it can be selected based on the importance of the samples in the cluster (such as the degree of difference from the characteristic values ​​such as the average flow rate, pressure, and temperature in the cluster). The advantage of this method is that it can utilize the similarity between samples, and the homogeneity of the samples within the generated sample subset is high, which is conducive to special analysis and model training of sample groups with similar fluid characteristics. Of course, the method of obtaining sample subset pairs is not limited to the above, as long as the sample subset pairs can be obtained.

[0031] In one embodiment, the method comprises at least the following steps of obtaining the conditional expectation corresponding to the sample subset:

[0032] Determine the label set corresponding to the selected small sample set; and calculate the conditional expectation of the sample subset for the corresponding label set in the label set.

[0033] In this embodiment, in the data set, samples usually have corresponding "result descriptions" or "target attributes", which are labels. For the selected small sample set, there is also such a corresponding relationship. Each sample in the small sample set has its corresponding label, and all these labels together constitute the label set corresponding to the small sample set. If the data is stored in a structured form, such as a common table form, the data of the small sample set and its corresponding labels will generally be reflected in different columns in the same data table. Some identifiers (such as unique sample numbers, indexes, etc.) can be used to accurately match the samples in the small sample set with the corresponding labels one by one, and then sort out a complete set of labels. For each sample subset pair, there is a corresponding part of the labels in the label set, and the set composed of these labels is the label set corresponding to the sample subset pair.

[0034] In one embodiment, the process of training the model to be trained based on the training set includes:

[0035] Acquire a mapping relationship between the sample subset pair and the conditional expectation corresponding to the sample subset;

[0036] Based on the mapping relationship, the sample subset pair is taken as input, and the conditional expectation corresponding to the sample subset is taken as output to train the model to be trained.

[0037] In this embodiment, under such input and output settings, the model to be trained will continuously learn and adjust its internal parameters (such as weights in the neural network) based on a large number of sample subset pairs and their corresponding conditional expectation data. Through multiple iterations, the model tries to find an optimal parameter combination so that when a new sample subset pair is given as input, it can output the corresponding conditional expectation as accurately as possible, so that the model has the ability to learn and predict such data relationships.

[0038] In one embodiment, the process of obtaining the mapping relationship between the sample subset and the conditional expectation corresponding to the sample subset includes: obtaining the mapping relationship based on a fitting method.

[0039] Or, in an alternative approach, the mapping relationship can be obtained by building a classification or decision tree.

[0040] The following is an example of a two-dimensional single-phase steady-state Darcy flow problem:

[0041]

[0042] The seepage equation describes the relationship between the pressure and permeability field of the fluid, where K(x, y) is the permeability field of the space where the fluid is located, which is related to the spatial coordinates (x, y). After discretization using the finite volume method (FVM),

[0043]

[0044] The above formula satisfies the Dirichlet boundary condition, where T ij is the transmittance between two adjacent cells evaluated by the harmonic mean

[0045]

[0046] Given that the permeability field in this example has 5 strips, each strip satisfies a different uniform distribution, and the specific distribution parameters are as follows:

[0047] Table 1 Distribution parameters of permeability field

[0048] Strip number Uniform distribution parameters 1 [1,9] 2 [3,25] 3 [5,49] 4 [6,64] 5 [8,100]

[0049] The original spatial domain is discretized using a 20×20 Cartesian grid for subsequent numerical simulations. In order to represent the case where the training set is small due to the unavailability or high cost of measurements or forward simulators, the training set size in this example is set to 100. For traditional deep learning (sample learning), the training set contains 100 (m, d) pairs, where m is an instance of the permeability field and d is the corresponding pressure field. The test set contains another 100 pairs. The FCN network is trained, and the training input features are the vectors straightened from the permeability field [20,20] matrix. The DCNN structure is used for subsequent prediction of qwt production, and the input features are directly the images corresponding to the permeability field.

[0050] like Figure 2 As shown in the figure, the original data is reconstructed using this method and then put into the same FCN model for training and testing. Figure 3 After the original data is reconstructed using this method, it is put into the same FCN model for training and the prediction accuracy distribution in the test set. From the comparison in the above figure, it can be seen that the EL method can greatly improve the prediction accuracy of small sample learning.

[0051] Comparing the training effects of two different methods, in order to reduce the influence of other factors, other hyperparameters were kept unchanged as much as possible in the experiment. For example, both used the Adam optimizer, the number of iterations was 5000, the number of hidden layer neurons of MPL was the same, and the data batch size was 100. It can be seen that the original method and model predicted that the proportion of data with an error in [0,10%] was 28%, while the proportion of data with an error in [0,10%] using expected learning reached 90%, an increase of 71% compared to before. At the same time, the proportion of data with a relative error in [0,4%] increased from 14% to 81%, indicating that the new model training method has significantly improved the prediction accuracy.

[0052] Embodiment 2

[0053] Taking a two-phase flow problem as an example, the prediction indicator is changed from the original pressure to the predicted output qwt. In order to speed up the training, the CNN model is used for fitting. It can be found that for the 2-tuple expected learning method, the proportion of predicted relative errors falling in [0,4%] is 5.1% higher than that of the single CNN model, and the proportion of errors falling in the [0,10%] interval is also increased by 4.4%, which shows a significant improvement in accuracy.

[0054] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the original technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A small sample deep learning method based on mathematical expectation, characterized in that: include: Acquire a training set; wherein the training set includes sample subset pairs and conditional expectations corresponding to the sample subsets; Reconstructing the format of the sample to be predicted so that the format of the reconstructed sample to be predicted is the same as the structure of the sample subset pair; The reconstructed prediction samples are input into the trained model to obtain the prediction results; the method at least comprises the following steps to obtain the trained model: the model to be trained is trained based on the training set.

2. The method according to claim 1, characterized in that The method at least comprises the following steps to obtain the sample subset pair: The selected small sample set is used to generate several sample subsets according to the arrangement and combination method, and each sample subset contains several sample subset pairs.

3. The method according to claim 2, characterized in that The method at least comprises the following steps to obtain the conditional expectation corresponding to the sample subset: Determine a label set corresponding to the selected small sample set; The conditional expectation of the sample subset with respect to the corresponding label set in the label set is calculated.

4. The method according to claim 1, characterized in that: The process of training the training model based on the training set includes: Acquire a mapping relationship between the sample subset pair and the conditional expectation corresponding to the sample subset; Based on the mapping relationship, the sample subset pair is taken as input, and the conditional expectation corresponding to the sample subset is taken as output to train the model to be trained.

5. The method according to claim 4, characterized in that The process of obtaining the mapping relationship between the sample subset and the conditional expectation corresponding to the sample subset includes: obtaining the mapping relationship based on a fitting method.

Images