Agricultural germplasm resource data adaptive filling method based on FIML and DAE

Abstract

The invention discloses an agricultural germplasm resource data adaptive filling method based on FIML and DAE, and the method comprises the steps: firstly, employing clustering assistance to avoid the influence of a data abnormal value on an algorithm; a selective filtering layer is used for identifying high-quality estimation, and then a method (CFSM-DAE) for combining full information maximum likelihood (FIML) and a multi-pair resistance auto-encoder (DAE) by adopting an integrated framework is used for improving the estimation precision. Finally, simulation data and an actual rice germplasm resource data set are studied under three missing data mechanisms (MAR, MCAR and MNAR), and a CFSM-DAE method is compared with multiple common filling algorithms (FIML, DAE, KNN, RF and MICE); according to the method, observation data are fully utilized, a nonlinear relation is captured, the accuracy of a filling result and the robustness of the model are improved, an excellent filling effect is shown under different deletion mechanisms and deletion rates, the method is remarkably superior to common filling methods of FIML DAE, KNN, RF and MICE, and the method has remarkable superiority in filling of rice germplasm resource deletion values.

Description

Technical Field

[0001] The present invention relates to the technical field of data adaptive filling, and in particular to a method for adaptive filling of agricultural germplasm resource data. Background Art

[0002] In the digital age, large amounts of complex structured and unstructured data are accumulating at a high speed. In the process of agricultural data analysis, the quality of data will affect the effectiveness of data analysis. Filling missing values ​​in data is the key to ensuring the accuracy of analysis and the reliability of models. The main reason for missing data is the missing when collecting, processing or transmitting data sets. In addition, data quality is also related to many factors such as analysis methods, related attributes, sampling methods, data training, testing and verification. As an important food crop in China, rice germplasm resource data plays an important role in agricultural research. They usually contain multiple attributes such as genotype information, phenotypic characteristics, and environmental variables, and there may be nonlinear relationships and interactions between these attributes. Genotype information refers to the genetic characteristics of rice at the molecular level, which directly affects important phenotypic characteristics such as rice growth and disease resistance. In addition, phenotypic characteristics include rice yield, plant height, flowering time, etc., which are also significantly affected by environmental factors. Environmental variables such as climate conditions and soil types will have complex effects on rice growth. Therefore, choosing the right analysis method is crucial to fill the missing rice germplasm resources. The most direct way to deal with missing data is to delete incomplete values, but deleting incomplete data may lead to insufficient information and affect the performance of the model. To deal with missing data, it is usually necessary to consider the amount of missing data, the treatment of missing values, and the missing pattern of data values.

[0003] At present, common missing value filling (MVI) methods are divided into statistical methods and machine learning methods. Statistical methods (including mean interpolation, regression interpolation and multiple interpolation, etc.) often show limitations when facing large-scale and complex data sets. For example, mean interpolation may ignore important variability and complex structures in the data, while regression interpolation may rely too much on certain features, increase sensitivity to outliers, and thus affect the accuracy of the interpolation results. Multiple interpolation can handle complex data, but its computational complexity is high and it is sensitive to the choice of parameters. Machine learning methods such as k-nearest neighbor, random forest, chained multiple interpolation, VAE, etc. can handle large-scale and complex data and capture nonlinear relationships. However, these methods usually cannot make full use of observed data and perform poorly when processing small sample data. Therefore, it is urgent to propose a new MVI method to fill rice germplasm resource data, which should have the advantages of combining statistical methods and machine learning techniques while overcoming its own limitations, being able to handle large-scale data sets, capturing nonlinear relationships and maintaining robustness to outliers.

[0004] In view of the above, the present application provides an agricultural germplasm resource data adaptive filling method based on FIML and DAE to deal with the above problems. Summary of the invention

[0005] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an adaptive filling method for agricultural germplasm resource data based on FIML and DAE, namely, a clustered full information selective filtering encoder data filling algorithm (hereinafter referred to as CFSM-DAE). First, clustering is used to assist in avoiding the influence of data outliers on the algorithm, and then a selective filtering layer is used to identify high-quality estimates, and then an integrated framework is used to combine full information maximum likelihood (FIML) with a multi-adversarial autoencoder (DAE) (CFSM-DAE) to improve the estimation accuracy, which solves the problem of properly handling the missing attributes that affect crop growth in crop data collection. Due to its robustness in handling missing rice data and its wide application potential, it provides an important idea for solving complex missing value problems in practical applications.

[0006] The method for adaptively filling agricultural germplasm resource data based on FIML and DAE is characterized by comprising the following steps:

[0007] S1: Apply clustering strategies to cluster the data, add the clustering results as additional features to the original data set, and use clustering to assist in avoiding the impact of germplasm data outliers on the algorithm;

[0008] S2: Use FIML to estimate the parameters of the data based on the entire large data set containing clustering information, and use the initial complete data after parameter estimation as the training initialization data for the selective filtering layer DAE model, making full use of germplasm observation data, capturing nonlinear relationships, improving estimation accuracy and improving the robustness of the model;

[0009] S3: The selective filtering layer is used in the DAE model to adaptively output the final result.

[0010] The above technical solution has the following beneficial effects:

[0011] (1) The clustered full information selective filter encoder (CFSM-DAE) method proposed in this paper effectively solves the common missing value problem in rice germplasm resource datasets;

[0012] (2) Clustering assistance is used to avoid the influence of germplasm data outliers on the algorithm;

[0013] (3) The full information maximum likelihood (FIML) is combined with the multi-adversarial autoencoder (DAE) using an integrated framework to make full use of the observed data, capture nonlinear relationships, and improve the accuracy of the filling results and the robustness of the model;

[0014] (4) Experimental results show that this method (CFSM-DAE) performs well under different missing mechanisms and missing rates, and is significantly better than the common filling methods such as FIML DAE, KNN, RF, and MICE. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is the framework diagram of the CFSM-DAE algorithm of the present invention;

[0016] Figure 2 This is a comparison chart of RMSE of different methods in various missing mechanisms and missing rates in experiments according to a specific embodiment of the present invention;

[0017] Figure 3 This is a MAE comparison diagram of different methods in various deletion mechanisms and deletion rates in experiments according to a specific embodiment of the present invention;

[0018] Figure 4 It is a convergence curve diagram of rice germplasm resource data under different deletion mechanisms at a deletion rate of 30% in the experiment of the specific embodiment of the present invention. DETAILED DESCRIPTION

[0019] The aforementioned and other technical contents, features and effects of the present invention will be clearly presented in the following detailed description of the embodiments with reference to the drawings of the present application. The contents mentioned in the following embodiments are all based on the drawings of the specification as reference.

[0020] 1. Materials and Methods

[0021] 1.1 Experimental environment, data sources, and data preprocessing

[0022] The programming language used in the experiment of the present invention is python, version 3.8.0. A programming environment based on the Pytorch framework was built on the Windows 10 operating system, the framework version is 1.14.0, the development environment is PyCharm 2021.1.3amd64, the processor used in the experiment is Intel i7-8565U 1.80CHz, the graphics card is Intel UHD 620, and the memory is 16GB.

[0023] The data used are two types of data: simulated data and rice germplasm data. The simulated data uses Gaussian process based on radial basis function kernel to simulate 2000 actual samples with 30 variables, while the rice germplasm resources are collected from 11,000 data from China Crop Germplasm Information Network and Jiangsu Province Germplasm Resources Protection and Utilization Platform. Subsequently, the two types of data are processed using standardized preprocessing methods. First, the data are divided into training sets and test sets in a ratio of 8:2 for algorithm evaluation and verification. The study introduces an experimental design with artificial missing data, in which 10% to 50% of the data in the data set will be artificially deleted for subsequent research.

[0024] 1.2 Overall Framework of CFSM-DAE

[0025] In order to solve the impact of anomalies on the algorithm model, the present invention uses the clustering full information selective filtering encoder (CFSM-DAE) data filling algorithm, applies the clustering strategy to cluster the data, and adds the clustering results as additional features to the original data set; then, based on the clustering, FIML is used to estimate the parameters of the data, and the parameter estimation results are used as the training initialization of the selective filtering layer DAE model; finally, the selective filtering layer is used in the selective filtering layer DAE model to adaptively output the final result. The specific process is as follows Figure 1 shown.

[0026] 1.3 Clustering Strategy

[0027] When there are heterogeneous or outlier samples in the data set, it may cause the estimation of missing values ​​to have higher variance or error. To deal with this problem, the clustering information of the samples is used to more accurately estimate the missing values. First, the median of the univariate is used to fill the data set with missing values, and then the clustering algorithm is applied to group the data samples. The clustering algorithm assigns each sample to one of the clusters and calculates the centroid (mean) of each cluster. A distance-based clustering algorithm is used to update the centroid by minimizing the sum of the squared distances between the samples in each cluster and their centroids, as shown in formula (1). Each sample is assigned a cluster label reflecting the cluster to which it belongs:

[0028]

[0029] in is the centroid of the pth cluster, and the association between the ith sample and the pth cluster is shown in formula (2):

[0030]

[0031] In the formula: if x i belongs to the pth cluster, then the indicator variable w ip =1, otherwise, w ip =0.

[0032] In order to integrate clustering information into the training data, three methods are adopted: first, cluster labels are added to the dataset as new features; second, cluster labels are converted into one-hot encoded vectors and added to the dataset; third, the centroid vector (mean vector) of each cluster is added to the dataset as an additional variable column. Then FIML estimation is performed based on the added clustering information.

[0033] 1.4 FIML estimation

[0034] FIML infers the parameters of missing data by maximizing the likelihood function of the complete data set. For the linear regression model, we use FIML's parameter estimation to solve a system of equations containing all observable data and parameters as shown in formula (3):

[0035]

[0036] Among them, X ij is the observed value of the jth feature of the ith sample, Y ij is the observed value of the corresponding response variable, is the mean of the i-th sample, and β is the model parameter. The initial complete data estimated by FIML is used as the initialization data for the training of the selective filtering layer DAE model, so that the selective filtering layer DAE model can utilize the prior knowledge provided by FIML during the training process.

[0037] 1.5 Selective filtration layer

[0038] When we process a data matrix X, containing n variables and s samples, we ensure that we use the mask matrix M to handle missing values. We calculate the selective filter layer L according to formula (4):

[0039]

[0040] Make it closer to the actual data X. The key role of this layer is to generate sample sets Select the sample that is most similar to the actual data X. By evaluating the Euclidean distance between samples and setting a threshold δ, L can select the required sample by judging whether the distance is less than the threshold δ. If the threshold is not met, continue to use FIML estimation.

[0041] Based on the selective filtering layer, the input As shown in formula (5), the generated samples that are more similar to the actual data will be selected as Then, based on the adjusted mask matrix M * , the corresponding actual data X * will with Combined with after, Can be used for further processing to update the entire model

[0042]

[0043] 1.6 Multi-DAE Collaboration

[0044] Based on the selective filtering layer, we can obtain high-quality samples. However, due to the existence of the threshold δ, the samples The number is far less than Since the actual data should be The number of samples is consistent, so it cannot be guaranteed that all actual samples in X can be selected. In addition, due to the reduction in the number of samples, the diversity of samples sent to DAE is also limited. In order to increase the number of selected actual samples and improve the diversity of filled samples, we adopt a multi-DAE mechanism.

[0045] In the process of filling data in multi-DAE, k groups of filling samples will be generated in each iteration. Since the selective filtering layer does not contain any neural network parameters, a universal selective filtering layer can be applied to all DAEs. The input is the FIML padded sample X. Based on the selective layer, is converted to Then, by comparing with the corresponding actual samples Combined, based on formula (6), a sample combining the actual value and the filling value is obtained

[0046]

[0047] Afterwards, the k filled data matrices obtained from DAE are The filling data matrix for the model can be obtained by calculating the element mean of each filling matrix As shown in formula (7), in order to handle discrete elements, the calculated value will be rounded to an integer:

[0048]

[0049] In the proposed selective filtering layer DAE model, δ and k are determined by parameter adjustment, and then δ and k are determined by cross-validation. The model is trained under different δ and k.

[0050] 1.7 Evaluation Indicators

[0051] When evaluating the padded dataset, we used root mean square error (RMSE), mean absolute relative error (MAE), coefficient of determination (R2 ) is used as the evaluation index. The calculation formula is shown in formula (8-10) 1) Root mean square error (RMSE): This index measures the overall error between the interpolated value and the actual value.

[0052]

[0053] Where n is the number of samples, y i is the actual value, are the corresponding interpolated values.

[0054] 2) Mean Absolute Error (MAE): The error is measured by calculating the average of the absolute errors between the predicted value and the actual value:

[0055]

[0056] 3) Coefficient of determination (R 2 ): is a statistical indicator used to measure the explanatory power of a model. It indicates the degree of fit between the predicted value and the actual value, providing an intuitive understanding of the model performance. 2 Values ​​range from 0 to 1, and may sometimes be negative:

[0057]

[0058] 2 Results and analysis

[0059] 2.1RMSE results and analysis

[0060] In this study, we used CFSM-DAE, DAE, KNN, RF, MICE and FIML methods for imputation, and evaluated the imputation effect by the root mean square error (RMSE). Figure 2 The RMSE performance of each method on simulated data and real data sets under different missing rates is shown. Note: Figure 2 (a), (b), and (c) are the RMSE of simulated data under the three missing mechanisms, and (d), (e), and (f) are the RMSE of rice germplasm resource data under the three missing mechanisms, where FIML corresponds to the secondary coordinate axis.

[0061] Under the MAR missing mechanism, CFSM-DAE simulates data among all interpolation algorithms. Figure 2 (a) and rice germplasm resource data Figure 2(d) have the lowest RMSE values, averaging 0.0533 and 0.04. In the simulated data, as the missing rate increases, the RMSE value of CFSM-DAE remains basically stable. Relatively speaking, the performance of DAE, KNN, RF, and MICE decreases significantly when the missing rate increases, especially in the rice germplasm resource data, which indicates that these algorithms may be more sensitive to specific features or missing patterns of this dataset. FIML RMSE is above 100, which is a poor performance.

[0062] Figure 2 (b, e) The results show that under the MNAR missing mechanism, CFSM-DAE performs relatively well in simulated data and rice germplasm data, with an average of 0.0256 and 0.0416, although the performance is lower than that of MAR. Its RMSE increase in simulated data is smaller than that of other algorithms, indicating that it is less dependent on unobserved data. In rice germplasm data, the performance of DAE and RF decreases significantly, indicating that these algorithms may have limitations in dealing with scenarios that are highly dependent on unobserved data. KNN and MICE perform poorly under MNAR, especially in high missing rate scenarios, because these methods may over-rely on observable data when dealing with missing data. The highest RMSE value of FIML remains above 100, and the performance is still poor.

[0063] Figure 2 (c, f) The results show that under the MCAR missing mechanism, CFSM-DAE is still relatively good in the simulation data and rice germplasm data, with an average of 0.0489 and 0.0443. Under high missing rates, its performance fluctuates the least, indicating that its algorithm can adapt well to and interpolate data under completely random missing data. The performance of DAE, RF, and KNN is improved compared with MAR and MNAR, but worse than CFSM-DAE, indicating that these algorithms may still have limitations when facing completely random missing data. Although MICE's performance is close to CFSM-DAE in some cases, it has a certain gap in stability and accuracy overall. The FIML RMSE value is still the highest at 111, and the performance is still poor compared with other algorithms.

[0064] 2.2 MAE Results and Analysis

[0065] In order to comprehensively evaluate the effectiveness of different data interpolation methods in dealing with different types of missing data, we use the MAE indicator for analysis, because it can directly reflect the average error between the predicted value and the actual value, and is a key indicator for measuring interpolation accuracy. Figure 3 It can be seen that in the evaluation of simulation data and rice germplasm resource data, the CFSM-DAE method showed the best filling effect under the MCAR, MAR or MNAR mechanism. Note: Figure 3 (a), (b), and (c) are the MAEs of simulated data under three missing mechanisms. Figure 3 (d), (e), and (f) are the MAEs of rice germplasm resource data under three missing mechanisms, where FIML corresponds to the secondary coordinate axis.

[0066] The specific results are as follows: Under the MAR mechanism, the MAE of CFSM-DAE in the simulated data and rice germplasm data is close to zero; under the MNAR mechanism, the MAE of CFSM-DAE in the simulated data remains at a low level (about 0.1), and in the rice germplasm data, it remains only about 0.22 at a missing rate of 50%; under the MCAR mechanism, the MAE of CFSM-DAE in the simulated data is 0.265 and the MAE in the rice germplasm data is 0.0105. The performance is highly consistent, which shows that CFSM-DAE can effectively handle different types of missing data, provide high-precision interpolation results, and is a reliable choice for dealing with missing data problems.

[0067] In contrast, among the three missing mechanisms, the DAE method performs generally well at low missing rates, but its MAE increases slowly, showing a certain degree of stability. The RF, KNN, and MICE methods perform well at low missing rates, but as the missing rate increases, the MAE increases significantly, reflecting the limitations of these methods at high missing rates. In addition, the FIML method performs poorly in all cases, and the MAE is always high, showing its disadvantage in dealing with missing data. Overall, CFSM-DAE performs better than other methods under different missing mechanisms and missing rates, demonstrating its superior performance in data interpolation tasks.

[0068] 2.3R 2 Results and Analysis

[0069] In order to study the R performance of different data imputation methods under simulated data and real rice germplasm resource datasets, 2 Performance, plotting simulation generated data R 2 (Table 1) and rice germplasm resources R 2 (Table 2).

[0070] Table 1 Simulation generated data R 2

[0071]

[0072]

[0073] Table 2 Rice germplasm resources R 2

[0074]

[0075] As shown in Table 1, when evaluating the simulated data under different missing mechanisms (MAR, MNAR, MCAR), under the MAR mechanism, the R 2 The values ​​are close to 1 (0.9986 to 0.982 at 10% to 50% missing rate, respectively), which is significantly better than FIML, DAE, KNN, RF and MICE algorithms. Similarly, under the MCAR mechanism, CFSM-DAE still performs well, with its R 2 The value remains between 0.9984 and 0.992 under the missing rate of 10% to 50%. Under the MNAR mechanism, the R 2 Although the value has decreased, it still remains at a high level (0.9998 to 0.9864 at 10% to 50% missing rate), far exceeding other comparison algorithms. This consistent excellent performance shows that the CFSM-DAE algorithm has extremely high robustness and stability when dealing with different types of missing data.

[0076] The results in Table 2 show that for the evaluation of rice germplasm data, the R of CFSM-DAE was 2 The values ​​are 0.9972 to 0.8384, which are significantly better than other algorithms. Under the MCAR mechanism and the missing rate is 10% to 50%, the R 2 The values ​​remain between 0.9975 and 0.7206. In the case of MNAR mechanism and missing rate of 10% to 50%, the R 2 The value remains between 0.998 and 0.687, which is significantly better than FIML, DAE, KNN, RF and MICE algorithms. Therefore, the CFSM-DAE algorithm can not only effectively process simulated data, but also provide high-precision interpolation results for complex missing mechanisms in practical applications, making it a reliable choice for dealing with missing data problems.

[0077] 2.4R 2 Model convergence

[0078] This study will analyze the model convergence performance of rice germplasm resource data under three missing mechanisms: MAR, MNAR, and MCAR, with a missing rate of 30%. The analysis results are as follows Figure 3 As shown in the figure: Under the three missing mechanisms, the training and test RMSE of the model both drop rapidly in the early stage and tend to be stable, and the final value is close to zero, indicating that the model can converge quickly and maintain a stable filling effect under different missing mechanisms. Note: Figure 4(a), (b), and (c) show the convergence of rice germplasm resource data under the MAR, MNAR, and MCAR missing mechanisms, respectively.

[0079] 2.5 Ablation Experiment

[0080] By comparing the complete model containing clustering features with the control model without clustering features, the actual utility of clustering information in the missing value filling task can be evaluated. This study designed a cluster ablation experiment of rice germplasm resources under three missing mechanisms (MAR, MNAR, and MCAR) (30% missing rate), as shown in Table 3.

[0081] Table 3 Clustering feature effectiveness

[0082]

[0083] The results show that the data filled with cluster features under the MAR mechanism show a lower root mean square error (RMSE) of 0.0434 (clustering) than 0.0714 (no clustering), and a lower mean absolute error (MAE) of 0.0103 (clustering) than 0.0188 (no clustering). 2 ) is 0.7941 (clustering) which is significantly higher than 0.6921 (no clustering). Similarly, it is the same under MNAR and MCAR mechanisms. This shows that clustering features are effective in processing different types of missing data and verify its wide applicability in the field of data filling.

[0084] In summary, combined with the technical solution of the present invention and the verification results, it can be seen that in the actual data collection process, data missing is often encountered. Processing these missing values ​​is an important part of the research because they may affect the accuracy of data analysis and the reliability of the results. The clustered full information selective filtering encoder (CFSM-DAE) method proposed in the present invention effectively solves the common missing value problem in rice germplasm resource data sets. The CFSM-DAE method uses clustering assistance, selective filtering layers, and an integrated framework to combine full information maximum likelihood (FIML) with multi-adversarial autoencoders (DAE), making full use of observed data, capturing nonlinear relationships, and improving the accuracy of the filling results and the robustness of the model. The experimental results also show that the method shows excellent filling effects under different missing mechanisms and missing rates, and is significantly better than the common filling methods of FIMLDAE, KNN, RF, and MICE. At the same time, the ablation experiment further verifies the actual utility of clustering information in the missing value filling task, and proves the effectiveness of clustering methods in data filling. Therefore, the CFSM-DAE method shows significant superiority in filling missing values ​​of rice germplasm resources.

[0085] After combining FIML with deep learning, R2 Both have been significantly improved, which effectively improves the filling algorithm of the rice germplasm resource dataset. At the same time, the results show that by increasing the threshold of the filter layer in deep learning, the accuracy of filling can be further screened and improved.

[0086] The above description is only for illustrating the present invention. It should be understood that the present invention is not limited to the above embodiments, and various variations that conform to the concept of the present invention are within the protection scope of the present invention.

Claims

1. The method for adaptive filling of agricultural germplasm resource data based on FIML and DAE is characterized by: The following steps are involved: S1: Apply clustering strategies to cluster the data, add the clustering results as additional features to the original data set, and use clustering to assist in avoiding the impact of germplasm data outliers on the algorithm; S2: Use FIML to estimate the parameters of the data based on the entire large data set containing clustering information, and use the initial complete data after parameter estimation as the training initialization data for the selective filtering layer DAE model, making full use of germplasm observation data, capturing nonlinear relationships, improving estimation accuracy and improving the robustness of the model; S3: The selective filtering layer is used in the DAE model to adaptively output the final result.

2. The method for adaptively filling agricultural germplasm resource data based on FIML and DAE according to claim 1, characterized in that: The clustering strategy in S1 specifically includes: first, using the median of a single variable to fill in the data set with missing values, then applying a clustering algorithm to group the data samples, the clustering algorithm assigns each sample to one of the clusters, and calculates the centroid of each cluster, i.e., the average value; using a distance-based clustering algorithm, the centroid is updated by minimizing the sum of the squares of the distances between the samples in each cluster and their centroids, as shown in formula (1): in, is the centroid of the pth cluster. Each sample is assigned a cluster label, reflecting the cluster to which it belongs. The association between the i-th sample and the p-th cluster is shown in formula (2): Among them: if x i belongs to the pth cluster, then the indicator variable w ip =1, otherwise, w ip =0.

3. The method for adaptively filling agricultural germplasm resource data based on FIML and DAE according to claim 2, characterized in that: The specific process of integrating clustering information into training data in S2 includes: First, cluster labels are added to the dataset as new features; Secondly, the cluster labels are converted into one-hot encoded vectors and added to the dataset; thirdly, the centroid vector of each cluster, i.e. the mean vector, is added to the dataset as an additional variable column.

4. The method for adaptively filling agricultural germplasm resource data based on FIML and DAE according to claim 2, characterized in that: The specific process of S2 includes solving a set of equations including all observable data and parameters through parameter estimation of FIML as shown in formula (3): Among them, X ij is the observed value of the jth feature of the ith sample, Yij is the observed value of the corresponding response variable, is the mean of the i-th sample, β is the model parameter, and the result of FIML estimation is used as the initialization data for the training of the selective filtering layer DAE model, so that the selective filtering layer DAE model can utilize the prior knowledge provided by FIML during the training process.

5. The method for adaptively filling agricultural germplasm resource data based on FIML and DAE according to claim 4, characterized in that: The specific process of S3 includes calculating the selective filtering layer L according to formula (4) so ​​that it is closer to the actual data X: The key role of this layer is to generate sample sets Select the sample that is most similar to the actual data X; by evaluating the Euclidean distance between samples and setting a threshold δ, L can select the required sample by judging whether the distance is less than the threshold δ. If it does not meet the threshold, continue to use FIML estimation; based on the selective filtering layer, the input As shown in formula (5): Generated samples that are more similar to the actual data will be selected Then, based on the adjusted mask matrix M*, the corresponding actual data X* will be Combined with after, Can be used in further processing to update the entire model.

6. The method for adaptively filling agricultural germplasm resource data based on FIML and DAE according to claim 5, characterized in that: The DAE model adopts a multi-DAE mechanism, specifically including: in the process of multi-DAE filling data, each iteration will generate k groups of filling samples Since the selective filtering layer does not contain any neural network parameters, a universal selective filtering layer can be applied to all DAE models. The input is the sample X padded by FIML. Based on the selection layer, is converted to Then, by comparing with the corresponding actual samples Combined, based on formula (6), a sample combining the actual value and the filling value is obtained Afterwards, the k filled data matrices obtained from the DAE model The filling data matrix for the model can be obtained by calculating the element mean of each filling matrix As shown in formula (7): In order to process discrete elements, the calculated values ​​will be rounded off and converted to integers. In the proposed selective filtering layer DAE model, δ and k are determined by adjusting parameters, and then δ and k are determined by cross-validation. The model is trained under different δ and k, and the optimal model is selected by evaluation indicators.

Images