A method, system, device and medium for enhancing effective nutrient data of feed raw materials
By integrating the effective nutrient data of feed raw materials, filling and enhancing the synthetic data using a variety of deep learning models, and evaluating the quality, the problems of small data sample size and low model fitting accuracy are solved, and data quality and sample size are improved, providing strong data support for the effective nutrient prediction model of feed raw materials.
Patent Information
- Application Number
- CN202411718163.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-03-14
- Estimated Expiration
- 2044-11-28
AI Technical Summary
The prior art is difficult to effectively collect and enhance the effective nutrient data of feed raw materials, resulting in small data sample size, low model fitting accuracy, and lack of key nutritional indicator factors in the predictors.
By obtaining the initial sample data set, structural integration and data filling are performed, data subsets are divided, and data augmentation is enhanced by using conditional generation of adversarial neural networks, variational autoencoders and diffusion models, synthetic data is generated, and qualified data are screened through quality evaluation and merged into large sample data sets.
Improve the integrity and quality of the data, expand the sample size, and provide a solid data foundation for establishing a high-performance feed raw material effective nutrient prediction model.
Smart Images

Figure CN119202559B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of data processing, and in particular relates to a method, system, equipment and medium for enhancing effective nutrient data of feed raw materials. Background Art
[0002] The problem of food security is very serious, and the problem of food security is concentrated in the problem of feed grain security. The traditional feed formula structure is mainly based on soybeans and corn, and the type is single, which has led to a high dependence on soybean imports and a long-term tight balance of corn. In today's situation of tight feed resources, we need to accurately grasp the nutritional value of feed raw materials, especially the variation of effective nutrients, in order to make good use of existing feed raw material resources.
[0003] The effective nutrients of feed mainly include effective energy (such as digestible energy, metabolizable energy and net energy) and digestible amino acids (such as digestible lysine, digestible methionine, digestible threonine, etc.). The existing data sources of effective nutrients in feed raw materials are all from animal experiments, but animal experiments consume a lot of manpower and material resources, and the collection of corresponding data is relatively difficult. In addition, the data that can be collected from a single animal experiment is limited, which leads to only about 20 original samples when predictive modeling is carried out, resulting in poor prediction performance and low generalization ability of the model. Data collection is relatively difficult, so the original sample size used in the traditional linear regression model is generally small, which leads to low fitting accuracy of the traditional linear regression model, and the linear prediction equation also has the problem that the key nutritional index factors are not included in the prediction factors. The coefficients of many prediction factors are contrary to the nutritional logic and cannot reflect the correlation variation law of key influencing factors. The establishment of a new model with good prediction performance and strong generalization ability is inseparable from the support of data. This requires not only good data quality and the best possible reflection of the real situation, but also a large amount of data so that the model can learn the deep structural relationship between the data. However, there are relatively few sources of effective nutrient data such as effective energy of feed raw materials, and data collection is relatively difficult. How to expand a large data sample set is the primary issue in establishing an accurate prediction model for effective nutrients in feed raw materials. Summary of the invention
[0004] In order to solve the problem that it is difficult to collect effective nutrient data of feed raw materials, the present invention provides a method, system, device and medium for enhancing effective nutrient data of feed raw materials.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A method for enhancing effective nutrient data of feed raw materials comprises the following steps:
[0007] Acquire an initial sample data set of feed raw materials; structurally integrate the features of the initial sample data set to obtain structured data with a structural standard; fill the structured data with data by a composite data filling method to obtain a complete data set; divide the complete data set according to the types of feed raw materials to obtain a subset of effective nutrient data of the feed raw materials;
[0008] Setting weights for the conditional generative adversarial neural network, the variational autoencoder and the diffusion model respectively to form an enhanced data model, inputting the effective nutrient data subset of the feed raw material into the enhanced data model for data enhancement, and obtaining synthetic data;
[0009] By judging whether the synthesized data is reasonable and the similarity between the synthesized data and the data in the initial sample data set, the quality of the synthesized data is evaluated, qualified data is screened out, and the qualified data is merged with the initial sample data to obtain a large sample data set.
[0010] Preferably, the obtaining of the initial sample data set of feed raw materials comprises the following steps: obtaining a plurality of effective nutrient data of feed raw materials by keyword retrieval and screening in various databases through a meta-analysis method, and obtaining the initial sample data set of feed raw materials based on the plurality of effective nutrient data of feed raw materials; the characteristics of the initial sample data set of feed raw materials are divided into three categories: basic information of feed raw materials, basic information of experimental animals and effective nutrient information of feed raw materials.
[0011] Preferably, the method for compound data filling specifically comprises the following steps:
[0012] A k-nearest neighbor filling operation is used to obtain a first estimated value for each missing value of the structured data;
[0013] Taking the missing features as the target variables and the remaining features as the predictors, we selected machine learning algorithms such as decision tree, random forest and gradient boosting to establish the prediction model, optimized the hyperparameters of each model, selected the model with the best performance as the prediction model, and predicted the second estimated value;
[0014] Perform data filling based on the average of the first estimate and the second estimate;
[0015] After data filling, the effective nutrient data set of feed raw materials is obtained. This data set is divided into effective nutrient data subsets of each feed raw material according to the type of feed raw materials.
[0016] Preferably, the judging whether the synthesized data is reasonable is specifically judged by detecting whether the characteristic value of the synthesized data is within a range interval of the characteristic value, and characteristic values exceeding the range are directly eliminated.
[0017] Preferably, the similarity between the synthetic data and the data in the initial sample data set is determined by the following steps:
[0018] Setting evaluation criteria for the data similarity, wherein the evaluation criteria take into account data structure, data distribution, and data performance, and setting weights for each evaluation criterion according to importance;
[0019] The data structure is evaluated through principal component analysis and correlation matrix analysis. The principal component analysis compresses the high-dimensional space data set into a low-dimensional space for analysis, and according to the value range of the real data in the two-dimensional space, the proportion of the synthetic data after dimensionality reduction is statistically analyzed; the correlation matrix analysis establishes a correlation matrix of the real data and the synthetic data, compares the difference of the correlation matrix, calculates the absolute value of the difference between the two correlation matrices, and takes the average of the absolute value of the difference;
[0020] The evaluation of data distribution is carried out through statistical analysis, EM distance analysis and KL divergence analysis;
[0021] The evaluation of data performance is achieved by using the synthetic data as a test set and evaluating the performance of the model constructed from the real data on the synthetic data.
[0022] The present invention also provides a feed raw material effective nutrient data enhancement system, which specifically includes:
[0023] The sample processing module is used to obtain an initial sample data set of feed raw materials; structurally integrate the features of the initial sample data set to obtain structured data with structural standards; fill the structured data with data by a composite data filling method to obtain a complete data set; divide the complete data set according to the type of feed raw materials to obtain a subset of effective nutrient data of the feed raw materials.
[0024] The data enhancement module is used to set weights for the conditional generative adversarial neural network, the variational autoencoder and the diffusion model to form an enhanced data model, and input the effective nutrient data subset of the feed raw material into the enhanced data model for data enhancement to obtain synthetic data.
[0025] The evaluation and expansion module is used to evaluate the quality of the synthesized data by judging whether the synthesized data is reasonable and the similarity between the synthesized data and the data in the initial sample data set, screen out qualified data, and merge the qualified data with the initial sample data to obtain a large sample data set.
[0026] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps described in the method for enhancing effective nutrient data of feed raw materials.
[0027] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is loaded by a processor, the steps described in the method for enhancing effective nutrient data of feed raw materials can be executed.
[0028] The method for enhancing effective nutrient data of feed raw materials provided by the present invention has the following beneficial effects:
[0029] The present invention integrates the acquired initial sample data set of feed raw materials, fills the integrated data to obtain a complete data set, and divides the data subsets according to the types of feed raw materials. The obtained data is complete and has a unified structure. Weights are set for different models to form an enhanced data model, and the data subsets are input to obtain enhanced data samples, and the enhanced data samples are evaluated for quality to improve the availability of data. For data with qualified data quality or above, the enhanced data set is merged with the real data to obtain a larger sample data volume, which provides a data basis for the subsequent establishment of a machine learning prediction model related to the nutritional value of animal feed. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0031] Figure 1 The present invention is a flowchart of a method for enhancing effective nutrient data of feed raw materials in an embodiment of the present invention.
[0032] Figure 2 This is a data filling diagram in an embodiment of the present invention.
[0033] Figure 3 is a schematic diagram of the model in the present invention, wherein Figure 3 (a) is the principle diagram of conditional generation of adversarial neural network; Figure 3 (b) is the principle diagram of variational autoencoder; Figure 3 (c) is the schematic diagram of the composite model; Figure 3 (d) is the schematic diagram of the diffusion model.
[0034] Figure 4 This is a similarity evaluation diagram between synthetic data and real data in an embodiment of the present invention. DETAILED DESCRIPTION
[0035] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.
[0036] Example
[0037] The present invention provides a method for enhancing effective nutrient data of feed raw materials, such as Figure 1 As shown, the specific steps include:
[0038] S1. Obtain the initial sample data set of feed raw materials. Use the meta-analysis method to collect the effective nutrient data of feed raw materials published in the past ten years. Search the databases by keywords: growing pigs, nutritional value, effective energy, digestible energy, metabolizable energy, net energy, digestible amino acids, corn, soybean meal, corn gluten meal, wheat bran, rapeseed meal, rice bran, etc. After the search, screen them to obtain the initial sample data set that meets the conditions.
[0039] S2. Integrate the initial sample data set to obtain structured data with structural standards. Since the nutritional indicators measured in various studies are not necessarily the same, the characteristics of this initial data set should be integrated to unify the structure of each data sample, so as to obtain structured data with structural standards. According to the characteristics collected by meta-analysis and relevant knowledge of animal nutrition, the characteristic values are divided into three categories: basic information of feed raw materials, basic information of experimental animals, and effective nutrient information of feed raw materials. As shown in Table 1, a total of 19 characteristic values are included. Some of the characteristic values are discrete, and some are continuous.
[0040] Table 1 Structured data samples
[0041]
[0042] S3. Fill in the structured data using the composite data filling method to obtain a complete data set. The data feature information collected from each study is different, and the structured data obtained by integration must have missing data. For the missing data, a data filling operation is performed, such as Figure 2 Common data filling operations include mean filling, mode filling, and median filling. These filling operations are generally applicable to situations where only a small number of characteristic values are missing. However, for the characteristics of effective nutrient data of feed raw materials, such as net energy, there may be multiple missing data. This is also because the test determination of effective nutrients such as net energy is time-consuming and labor-intensive and highly dependent on large-scale professional measurement equipment. Therefore, there are very few net energy data. It is obviously inappropriate to use conventional data filling operations for characteristic values such as net energy that are missing a lot.
[0043] The composite data filling method of the present invention specifically first uses a k-nearest neighbor filling operation to obtain an estimated value for each missing value; then takes the missing feature as the target variable and the remaining features as the predictor, selects a decision tree, random forest, gradient enhancement and other machine learning algorithms to establish a prediction model, performs hyperparameter tuning on each model, selects a model with the best performance as the prediction model, and predicts the estimated value of the missing value; finally, the average value of the estimated values obtained by the two methods is obtained to fill the missing value. When all the missing data are filled, a feed raw material effective nutrient data set with a unified structure and complete data is obtained. The data set is divided into each feed raw material effective nutrient data subset according to the type of feed raw material.
[0044] The keywords such as growing pig, corn, effective energy, etc. were searched in public databases such as CNKI and Web of Science, and the retrieved relevant articles were screened. After screening, the digestible energy data of corn for growing pigs in the corresponding papers were collected, and the features were sorted to obtain an initial data set, as shown in Table 2. The initial data set contains 33 real data on the digestible energy of corn for growing pigs.
[0045] Table 2 Actual data on corn digestibility for growing pigs
[0046]
[0047] S4. An enhanced data model is constructed by different data synthesis models, and weights are set for different data synthesis models according to their performance; a subset of effective nutrient data of feed raw materials is input into the enhanced data model for data enhancement to obtain synthetic data. Data enhancement operations are performed on each obtained data subset. A single model will inevitably learn a bias towards the original data, resulting in the distribution or pattern of the synthesized data being different from the real data. Therefore, multiple data synthesis models will be used, and weights will be set according to the performance of the synthesis models. The use of multiple data synthesis models can also enrich the diversity of the synthesized data. The synthesis model selection conditions generate adversarial neural networks, variational autoencoders, and diffusion models, such as Figure 3 (a) is the principle diagram of conditional generation of adversarial neural network; Figure 3 (b) is the schematic diagram of the variational self-encoder; Figure 3 (c) is a schematic diagram of the composite model of conditional generative adversarial neural network and variational autoencoder; Figure 3 (d) is the schematic diagram of the diffusion model.
[0048] The conditional generative adversarial neural network takes a randomly generated noise vector as the input of the generator, and after mapping through multiple nonlinear hidden layers, when it reaches the output layer, the number of neurons in the output layer is set to be equal to the number of real data features. The output layer finally uses a sigmoid activation layer to map all outputs to (0,1), and then uses different linear mappings to map different features to the corresponding interval range to obtain synthetic data. The discriminator then discriminates between real data and synthetic data. The ultimate ideal effect is that the discriminator cannot accurately distinguish between real data and synthetic data. One problem with the conditional generative adversarial neural network is that the similarity between synthetic data is too high, so its weight setting is low. The variational autoencoder passes the real data to the variational autoencoder, which compresses the high-order vector of the real data to a low-dimensional intermediate vector, finds the probability distribution of the intermediate vector, and then samples and decompresses it from the probability distribution to the original dimension to obtain new data. The training process of the variational autoencoder is unsupervised, so its weight setting is also low. The variational autoencoder is combined with the conditional generative adversarial neural network. The probability distribution of the latent vector found by the variational autoencoder is used as the distribution of the noise vector, and samples are taken from it as the input of the conditional generative adversarial neural network generator, thereby combining the two, and the weight can be appropriately increased. The diffusion model usually includes a forward diffusion process and a reverse denoising process. Starting from the real data distribution, noise is gradually added to make the data close to a simple distribution. Then the reverse process starts from the simple distribution and gradually removes the noise through the learned denoising network, and finally generates realistic data samples. The diffusion model has high rigor and strong generation ability in theory, and its weight is set to the highest.
[0049] The initial data set processed by the enhanced data model generated 165 synthetic data. Some of the synthetic data are shown in Table 3. The synthesis quality evaluation of this synthetic data set can be continued later.
[0050] Table 3 Partial synthesis data of corn digestible energy for growing pigs
[0051]
[0052] S5. By judging whether the enhanced data samples are reasonable and the similarity between the data samples and the real data, the synthetic data is evaluated for synthesis quality, and qualified and above data are selected as high-quality synthetic data sets. The quality of the high-quality synthetic data set is combined with the real data to obtain a large sample data set.
[0053] First, to judge whether the synthetic data conforms to common sense, it is mainly to detect whether the eigenvalues of the synthetic data are within the range of the eigenvalues. The eigenvalues that exceed the range are directly eliminated. The range of eigenvalues is set according to the accumulated data and nutritional experience. For the nutritional value data of feed raw materials, there may be interactions between the various features. The value of a feature may be affected by multiple other eigenvalues, so the eigenvalues do not exist in isolation. Only by constraining each eigenvalue individually can some obviously unreasonable values be eliminated first. At the same time, since each feed raw material has different characteristics and nutritional values, the range of the eigenvalues of each raw material is different. Therefore, the judgment of whether the synthetic data is reasonable should also be distinguished according to the type of raw material, and obviously unreasonable data can be eliminated.
[0054] Secondly, the similarity test between synthetic data and real data is completed with the help of the table-evaluator library in Python. The TableEvaluator function is called to obtain a visualization result, but this visualization result cannot concretely evaluate the data similarity. The present invention sets an evaluation standard to specifically quantify the data similarity. This evaluation standard will focus on three aspects: data structure, data distribution, and data performance. The weights of the three aspects will be divided according to their importance. Figure 4 As shown, the evaluation results of data quality can also be fed back to adjust the weight of the enhanced data model.
[0055] The first is data structure evaluation, which includes principal component analysis and correlation matrix analysis. Principal component analysis compresses high-dimensional space data sets into low-dimensional space for analysis. To facilitate visualization, the data sets are reduced to two-dimensional space, thereby obtaining the value range of real data in two-dimensional space. Then, statistics are counted to see how much of the synthetic data set (x) is within this value range after dimensionality reduction. The ratio (x) * 100% multiplied by the weight of the principal component analysis is the contribution of the principal component analysis to the similarity. The correlation matrix analysis is to establish the correlation matrix of the real data and the synthetic data, compare the difference of the correlation matrix, calculate the absolute value of the difference between the two correlation matrices, and then take the mean y of the absolute value of the difference. Since the correlation coefficient between the features is in [-1,1], if the mean of the absolute value of the difference is 0, it means that the two correlation matrices are no different. If the mean of the absolute value of the difference is 2, it means that the two correlation matrices are completely different. The similarity of the two correlation matrices is quantified as (-0.5y+1) * 100%, (-0.5y+1) * 100% multiplied by the weight of the correlation matrix is the contribution of the correlation matrix analysis to the similarity.
[0056] The second is data distribution evaluation, which includes statistical analysis, EM distance analysis and KL divergence analysis. Statistical analysis is to analyze and compare the statistics of the mean, standard deviation and median of each eigenvalue of the real data and synthetic data. In order to avoid the greater impact of extreme values, the logarithmic mean, logarithmic standard deviation and logarithmic median of each eigenvalue are used to draw the logarithmic mean fitting graph, logarithmic standard deviation fitting graph and logarithmic median fitting graph of the synthetic data for the real data, respectively, and perform regression analysis to calculate the fitting degree R 2 , log mean fit R 2 Recorded as z1, logarithmic standard deviation fitting R 2 Recorded as z2, log median fitting R 2 Denoted as z3, (z1 * 1 / 3+z2 * 1 / 3+z3 * 1 / 3) * 100% multiplied by the weight of the statistical analysis is the contribution of the statistical analysis to the weight. EM distance analysis, EM distance measures the minimum "cost" required to transform one probability distribution into another probability distribution. The "cost" here usually refers to the amount of work required to move the probability mass in the probability space. KL divergence analysis, KL divergence measures the degree of information loss when one probability distribution is used to approximate another probability distribution. Both EM distance and KL divergence analysis use Gaussian-like kernel functions, namely negative exponential functions, for quantification. When the EM distance and KL divergence values are 0, it means that there is no difference between the two probability distributions, and the negative exponential function value is 1; when the EM distance and KL divergence values gradually increase, it means that the difference between the two probability distributions is gradually increasing, and when the difference between the two probability distributions is large enough, the value of the negative exponential function will approach 0.
[0057] The third is data performance evaluation, which is achieved by using part of the synthetic data as a test set and evaluating the performance of the model built from the real data on the synthetic data. Specifically, the real data is divided into a training set and a test set, and an optimal model is built and trained through the training set, and the model performance is verified on the test set. The goal is to make regression predictions, so the root mean square error is selected as the performance evaluation indicator, and then the synthetic data is sampled n times without repetition, and the number of samples each time is the sample size of the real data test set. For these n subsets of synthetic data, they are used as test sets in turn to verify the performance of the model built from the real data, and the root mean square errors of the obtained n synthetic data are averaged. Then, the Gaussian kernel function is used to measure the difference between the root mean square error of the real data and the mean root mean square error of the synthetic data. If the two are equal, the output of the Gaussian kernel function is 1. If the mean root mean square error of the synthetic data is infinitely far from the root mean square error of the real data, the output of the Gaussian kernel function is 0. The output of the Gaussian kernel function is recorded as the score c, and c*100% multiplied by the data performance weight is the contribution of the data performance analysis to the similarity. The contribution values are added together to get the similarity between the synthetic data and the real data. If the final similarity value is between [60%, 75%), the synthetic data is considered to be of qualified quality; if the similarity value is between [75%, 90%), the synthetic data is considered to be of good quality; if the similarity value is between [90%, 100%], the synthetic data is considered to be of excellent quality. Finally, the synthetic data with qualified quality or above is merged with the real data to obtain a large sample data set.
[0058] The present invention also provides a feed raw material effective nutrient data enhancement system, which specifically includes:
[0059] The sample processing module is used to obtain the initial sample data set of feed raw materials; structurally integrate the features of the initial sample data set to obtain structured data with structural standards; fill the structured data with data through a composite data filling method to obtain a complete data set; divide the complete data set according to the type of feed raw materials to obtain a subset of effective nutrient data of the feed raw materials.
[0060] The data enhancement module is used to set weights for the conditional generative adversarial neural network, variational autoencoder and diffusion model to form an enhanced data model, and input the subset of effective nutrient data of feed raw materials into the enhanced data model for data enhancement to obtain synthetic data.
[0061] The evaluation and expansion module is used to evaluate the quality of the synthesized data by judging whether the synthesized data is reasonable and the similarity between the synthesized data and the data in the initial sample data set, screen out qualified data, and merge the qualified data with the initial sample data to obtain a large sample data set.
[0062] Each module in the above feed raw material effective nutrient data enhancement system can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.
[0063] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in an embodiment of a method for enhancing effective nutrient data of feed raw materials. The specific implementation method can be found in the method embodiment, which will not be repeated here.
[0064] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, a non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in an embodiment of a method for enhancing effective nutrient data of feed raw materials can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.
[0065] It should be understood by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0066] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0067] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0068] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0069] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.
Claims
1. A method for enhancing effective nutrient data of feed raw materials, characterized in that: The following steps are involved: Obtaining an initial sample data set of feed raw materials by meta-analysis; structurally integrating the features of the initial sample data set to obtain structured data with structural standards; The structured data is filled with data by a composite data filling method to obtain a complete data set; the composite data filling method specifically includes: using a k-nearest neighbor filling operation for each missing value of the structured data to obtain a first estimated value; using the missing feature as the target variable and the remaining features as the prediction factor, selecting a machine learning algorithm to establish a prediction model, performing hyperparameter tuning on the model, and predicting to obtain a second estimated value; performing data filling according to the average of the first estimated value and the second estimated value; dividing the complete data set according to the type of feed raw materials to obtain a subset of effective nutrient data of the feed raw materials; Setting weights for the conditional generative adversarial neural network, the variational autoencoder and the diffusion model respectively to form an enhanced data model, inputting the effective nutrient data subset of the feed raw material into the enhanced data model for data enhancement, and obtaining synthetic data; By judging whether the synthesized data is reasonable and the similarity between the synthesized data and the data in the initial sample data set, the quality of the synthesized data is evaluated, qualified data is screened out, and the qualified data is merged with the initial sample data to obtain a large sample data set; wherein the similarity of the data is specifically evaluated by setting weights for data structure, data distribution and data performance.
2. A method for enhancing effective nutrient data of feed raw materials according to claim 1, characterized in that: The method of obtaining the initial sample data set of feed raw materials by meta-analysis is specifically to obtain multiple effective nutrient data of feed raw materials by keyword retrieval and screening in each database, and obtain the initial sample data set of feed raw materials based on the multiple effective nutrient data of feed raw materials; the characteristics of the initial sample data set of feed raw materials are divided into three categories: basic information of feed raw materials, basic information of experimental animals and effective nutrient information of feed raw materials.
3. A method for enhancing effective nutrient data of feed raw materials according to claim 1, characterized in that: The determination of whether the synthesized data is reasonable is specifically performed by detecting whether the characteristic value of the synthesized data is within a range of the characteristic value, and directly eliminating the characteristic value that exceeds the range.
4. A method for enhancing effective nutrient data of feed raw materials according to claim 1, characterized in that: The similarity of the data is specifically evaluated by setting weights on the data structure, data distribution and data performance, including the following steps: The data structure is evaluated through principal component analysis and correlation matrix analysis. The principal component analysis compresses the high-dimensional space data set into a low-dimensional space for analysis, and the proportion of the synthetic data after dimensionality reduction is statistically analyzed according to the value range of the real data in the two-dimensional space. The correlation matrix analysis is performed by establishing correlation matrices of real data and synthetic data, comparing the differences of the correlation matrices, calculating the absolute values of the differences between the two correlation matrices, and taking the average of the absolute values of the differences; The evaluation of data distribution is carried out through statistical analysis, EM distance analysis and KL divergence analysis; The evaluation of data performance is achieved by using the synthetic data as a test set and evaluating the performance of the model constructed from the real data on the synthetic data.
5. A feed raw material effective nutrient data enhancement system, characterized in that: include: A sample processing module, used for obtaining an initial sample data set of feed raw materials through a meta-analysis method; Performing structural integration on the features of the initial sample data set to obtain structured data with structural standards; The structured data is filled with data by a composite data filling method to obtain a complete data set; the composite data filling method specifically includes: using a k-nearest neighbor filling operation for each missing value of the structured data to obtain a first estimated value; using the missing feature as the target variable and the remaining features as the prediction factor, selecting a machine learning algorithm to establish a prediction model, performing hyperparameter tuning on the model, and predicting to obtain a second estimated value; performing data filling according to the average of the first estimated value and the second estimated value; dividing the complete data set according to the type of feed raw materials to obtain a subset of effective nutrient data of the feed raw materials; A data enhancement module, used for setting weights for the conditional generative adversarial neural network, the variational autoencoder and the diffusion model to form an enhanced data model, inputting the effective nutrient data subset of the feed raw material into the enhanced data model for data enhancement, and obtaining synthetic data; The evaluation and expansion module is used to evaluate the quality of the synthesized data by judging whether the synthesized data is reasonable and the similarity between the synthesized data and the data in the initial sample data set, screen out qualified data, and merge the qualified data with the initial sample data to obtain a large sample data set; wherein the similarity of the data is specifically evaluated by setting weights for data structure, data distribution and data performance.
6. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded into a processor, the computer program can execute the steps of the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Oil tea tree inherent frequency data enhancement method based on improved CTGAN network
CN118395859A
Rural highway condition index data quality restoration method and system based on Stacking ensemble learning
CN118568079A