Method and system for acquiring training data of biochar yield prediction model

Abstract

The invention discloses a method and a system for acquiring training data of a biochar yield prediction model, and relates to the field of biomass pyrolysis, the method comprises the following steps: dividing n groups of original data of raw material characteristics of target biomass into m clusters, and calculating the mass center and error sum of squares of each cluster; comparing the sum of squares of errors of all the clusters, combining the two clusters with the minimum sum of squares of errors, and recalculating the centroid and the sum of squares of errors of the combined clusters; repeating the steps until three clusters remain; calculating n pieces of comprehensive index data of the remaining three clusters based on the normalized data in the remaining three clusters and the weight of each element index; and designing data sets in multiple combination forms, pre-training the biochar yield prediction model by using the data sets in different combination forms, and determining the combination form of the corresponding data set when the fitting precision is maximum and the error is minimum as the optimal data combination form. The complexity of obtaining the training data of the random forest model is reduced.

Description

Technical Field

[0001] The present application relates to the field of biomass pyrolysis technology, and in particular to a method and system for acquiring training data for a biochar yield prediction model. Background Art

[0002] Biomass pyrolysis technology refers to the process in which biomass raw materials are heated in an anaerobic or low-oxygen environment to cause internal decomposition of molecules to form biochar, bio-oil and non-condensable gas products. The formation of biomass pyrolysis products (i.e. biochar output) is easily affected by factors such as the raw material characteristics of biomass (carbon, hydrogen, oxygen, nitrogen, cellulose, hemicellulose, lignin, ash, volatile matter, fixed carbon content), pyrolysis process parameters (temperature, heating rate, holding time), etc., so machine learning models can be used to predict and evaluate biochar output.

[0003] In terms of biochar production prediction, the best machine learning model is the random forest model. In the existing random forest model training process, the elemental analysis, industrial analysis, and chemical analysis data sets of biomass raw materials are usually required as input. Through continuous training of the model, the close relationship between them and biochar production is found, so as to achieve the goal of predicting biochar production. However, in the training process of the random forest model, since there are many raw material characteristics that affect biochar production, the amount of raw material characteristics data that need to be collected and sorted is also large and more complex, which also leads to the low training efficiency of the random forest model. Summary of the invention

[0004] The purpose of this application is to provide a method and system for obtaining training data for a biochar yield prediction model, which reduces the complexity of obtaining training data for a random forest model, thereby improving the training efficiency of the random forest model.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] In a first aspect, the present application provides a method for obtaining training data for a biochar yield prediction model, the method comprising: determining the raw material characteristics of a target biomass and obtaining n groups of original data of the raw material characteristics; the raw material characteristics include m element indicators; n>m>3; using the Ward strategy in an agglomerative hierarchical clustering algorithm, the n groups of original data of the raw material characteristics are divided into m clusters, and the centroid and the sum of squared errors of each cluster are calculated; one cluster corresponds to one element indicator; one element indicator corresponds to n raw data; comparing the sums of squared errors of all clusters, Merge the two clusters with the smallest sum of squared errors, and recalculate the centroid and sum of squared errors of the merged cluster; repeat the step of "comparing the sum of squared errors of all clusters, merging the two clusters with the smallest sum of squared errors, and recalculating the centroid and sum of squared errors of the merged cluster", and continue to merge clusters until three clusters remain; the remaining three clusters are the first cluster, the second cluster, and the third cluster; the element indicators in the first cluster, the second cluster, and the third cluster all meet the first set condition; the first set condition is that the number of element indicators is greater than or equal to 1 and less than m-2; the original data in the remaining three clusters are Normalization, and based on the normalized data in the remaining three clusters, calculate the weight of each element index in the remaining three clusters; based on the normalized data in the remaining three clusters and the weight of each element index in the remaining three clusters, calculate n comprehensive index data of the first cluster, n comprehensive index data of the second cluster, and n comprehensive index data of the third cluster; design data sets in multiple combinations, use data sets in different combinations to pre-train the biochar yield prediction model, and determine the combination form of the data sets corresponding to the maximum fitting accuracy and the minimum error as the optimal data combination form; the biochar yield prediction model is a random forest model; the data sets in multiple combinations are respectively n comprehensive index data of the first cluster, n comprehensive index data of the second cluster, n comprehensive index data of the third cluster, n comprehensive index data of the first cluster and the second cluster, n comprehensive index data of the first cluster and the third cluster, n comprehensive index data of the second cluster and the third cluster, and n comprehensive index data of the first cluster, the second cluster and the third cluster; the optimal data combination form is used for the subsequent training data determination of the biochar yield prediction model.

[0007] In a second aspect, the present application also provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the training data acquisition method for the biochar yield prediction model described in the first aspect.

[0008] According to the specific embodiments provided in this application, this application discloses the following technical effects.

[0009] In order to reduce the amount of training data for the random forest model, the application clusters the original data of each element index in the raw material characteristics of traditional biomass, and continuously merges and clusters the m clusters initially divided, so as to find the close connection between the element indexes in the raw material characteristics, and finally uses the remaining three clusters to characterize the three categories of raw material characteristics. In order to further simplify the amount of data, the application also calculates the weights of the element indexes in the remaining three clusters, and uses the calculated weights to perform weighted summation on the normalized data of each element index to obtain the comprehensive index data, which reduces the complexity of the index data of each element to a certain extent. Finally, the comprehensive index data of the remaining three clusters are combined in a variety of ways, and an optimal data combination form is found. In the subsequent training process of the random forest model, it is only necessary to obtain the training data according to the optimal data combination form and train the random forest model. Therefore, the application reduces the complexity of obtaining the training data of the random forest model, thereby improving the training efficiency of the random forest model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 A flowchart of a method for acquiring training data for a biochar yield prediction model provided in an embodiment of the present application.

[0012] Figure 2 This is a diagram of the internal structure of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The purpose of this application is to provide a method and system for obtaining training data for a biochar yield prediction model, which reduces the complexity of obtaining training data for a random forest model, thereby improving the training efficiency of the random forest model.

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0016] Example 1 This embodiment provides a method for obtaining training data for a biochar yield prediction model. Figure 1 As shown, the method for obtaining training data for the biochar yield prediction model is as follows.

[0017] Step S1: determine the raw material characteristics of the target biomass, and obtain n groups of original data of the raw material characteristics; the raw material characteristics include m element indicators; n>m>3.

[0018] In this embodiment, n=300, m=10, and the element indicators in the raw material characteristics are carbon, hydrogen, oxygen, nitrogen, cellulose, hemicellulose, lignin, ash, volatile matter, and fixed carbon content. The 300 groups of raw data of the raw material characteristics can be recorded in an Excel table, that is, each column of the Excel table corresponds to an element indicator, and each row corresponds to a group of raw data of the element indicator. The entire Excel table has 300 rows and 10 columns.

[0019] Step S2: using the Ward strategy in the agglomerative hierarchical clustering algorithm, the n groups of raw material characteristics data are divided into m clusters, and the centroid and the error square sum of each cluster are calculated; one cluster corresponds to one element index; one element index corresponds to n raw data.

[0020] In this embodiment, 300 groups of raw data of raw material characteristics are divided into 10 clusters, one cluster corresponds to one element index, and one cluster contains 300 raw data corresponding to its element index. It can be understood that the Excel table is split into 10 groups by column, and each group corresponds to 300 rows of raw data.

[0021] Furthermore, the centroid and the sum of squared errors of each cluster are calculated as follows.

[0022] .

[0023] .

[0024] In the formula, Cluster C The center of mass, n Cluster C The amount of raw data in Cluster C The i The original data, Cluster C The sum of squared errors.

[0025] Step S3: Compare the sum of squared errors of all clusters, merge the two clusters with the smallest sum of squared errors, and recalculate the centroid and sum of squared errors of the merged cluster.

[0026] Among them, the calculation formula of the centroid of the merged cluster is as follows.

[0027] .

[0028] In the formula, For the i Cluster and j The centroid of the clusters after merging, For the i The amount of data in a cluster, For the j The amount of data in a cluster, For the i The centroid of the cluster, For the j The centroid of a cluster.

[0029] The calculation formula for the sum of squared errors of the merged clusters is as follows.

[0030] .

[0031] .

[0032] In the formula, For the i Cluster and j The sum of squared errors after merging clusters is For the i The sum of squared errors of clusters, For the j The sum of squared errors of clusters, For the i Cluster and j The error sum of squares after merging clusters.

[0033] Step S4: Repeat the above step S3, and continuously merge clusters until three clusters remain; the remaining three clusters are the first cluster, the second cluster and the third cluster; the element indices in the first cluster, the second cluster and the third cluster all meet the first setting condition; the first setting condition is that the number of element indices is greater than or equal to 1 and less than m-2.

[0034] In this embodiment, by comparing the size of the sum of squared errors of each cluster, the two clusters with the smallest sum of squared errors are continuously merged. The entire merging process is the process of classifying the 10 element indicators. Through continuous practice, the 10 element indicators will eventually be divided into three categories, each corresponding to a cluster, that is, the first category (the first cluster) is lignin, hemicellulose, ash, fixed carbon, hydrogen and nitrogen; the second category (the second cluster) is cellulose, carbon and oxygen; the third category (the third cluster) is volatile matter.

[0035] The first cluster, the second cluster, and the third cluster of the remaining three clusters can be understood as splitting the original Excel table into three independent small tables, that is, the first table has 6 columns and 300 rows of original data, the second table has 3 columns and 300 rows of original data, and the third table has 1 column and 300 rows of original data.

[0036] Step S5: normalize the original data in the remaining three clusters, and calculate the weight of each element index in the remaining three clusters based on the normalized data in the remaining three clusters.

[0037] Among them, the formula for normalizing the original data in the remaining three clusters is as follows.

[0038] .

[0039] In the formula, For a cluster j The element index i Normalized data, For a cluster j The element index i The original data, For a cluster j The minimum raw data of element indicators, For a cluster j The original data of the first cluster, the second cluster, and the third cluster in the remaining three clusters all need to be normalized according to the above formula.

[0040] Furthermore, the weight calculation formulas for each element index in the remaining three clusters are as follows.

[0041] .

[0042] .

[0043] .

[0044] In the formula, For a cluster j The weight of the element index, For a cluster j The entropy of the element index, m is the number of element indices in a cluster, For a cluster j The element index i The proportion of normalized data, n For a cluster j The normalized data volume of element indicators, For a cluster jThe element index i Normalized data.

[0045] Step S6: Based on the normalized data in the remaining three clusters and the weights of the element indicators in the remaining three clusters, calculate n comprehensive indicator data of the first cluster, n comprehensive indicator data of the second cluster, and n comprehensive indicator data of the third cluster.

[0046] Among them, the calculation formula for the comprehensive indicator data is as follows.

[0047] .

[0048] In the formula, For a cluster i Comprehensive indicator data, The first index of the first element in a cluster. i Normalized data, is the weight of the first element index in a cluster, The index of the second element in a cluster. i Normalized data, is the weight of the second element index in a cluster, For a cluster j The element index i Normalized data, In a cluster j The weight of the element indicator.

[0049] Furthermore, the process of calculating the comprehensive index data can be understood as the process of weighted summation of all element indicators in a cluster. For example, the first table after splitting has 6 columns and 300 rows of normalized data. It can be seen that there are 6 element indicators in the cluster. In order to comprehensively evaluate these 6 element indicators, it is necessary to use the weight of each element indicator to weighted sum each row of normalized data, and the calculation result is the comprehensive index data. Because each cluster has 300 rows of normalized data, each row of normalized data is weighted and summed to obtain a comprehensive index data, so each cluster will eventually obtain 300 comprehensive index data.

[0050] Step S7: design data sets in various combinations, and use data sets in different combinations to pre-train the biochar yield prediction model, and determine the combination of data sets corresponding to the maximum fitting accuracy and the minimum error as the optimal data combination; the biochar yield prediction model is a random forest model; the data sets in various combinations are respectively the n comprehensive indicator data of the first cluster, the n comprehensive indicator data of the second cluster, the n comprehensive indicator data of the third cluster, the n comprehensive indicator data of the first cluster and the n comprehensive indicator data of the second cluster, the n comprehensive indicator data of the first cluster and the n comprehensive indicator data of the third cluster, the n comprehensive indicator data of the second cluster and the n comprehensive indicator data of the third cluster, and the n comprehensive indicator data of the first cluster, the n comprehensive indicator data of the second cluster and the n comprehensive indicator data of the third cluster.

[0051] In this embodiment, in order to determine the best data combination form, so as to determine the training data of the subsequent random forest model, a total of 7 data sets in the above combination forms are designed. The random forest model is an integrated learning method, which performs classification and regression through the voting mechanism of multiple decision trees. The raw material characteristics and process parameters are x values ​​(input values), and the biochar yield is y value (output value). The raw material characteristics are respectively substituted into the data sets in the above 7 combinations, and the process parameters remain unchanged. 80% of each data set is used as a training set and 20% as a prediction set. The number of decision trees in the random forest model is set to 400, and the maximum depth is not limited. The average of the prediction results of all decision trees is used as the final prediction value.

[0052] .

[0053] In the formula, is the final prediction value of the random forest model, T is the number of decision trees, For the t The predicted value of a decision tree. After many experiments, the fitting accuracy R 2 The error RMSE is shown in the following table.

[0054] Table 1 Fitting accuracy and error of different combinations of data sets

[0055] As shown in Table 1, when the random forest model is trained using the comprehensive indicator data of the first cluster alone, the RMSE on the test set is lower than 5.5, and the R 2 It exceeds 0.87, so the training data of the subsequent random forest model can be obtained based on the element index and weight in the first cluster. Through the above optimization of the data set, the prediction accuracy of the random forest model can be effectively improved.

[0056] Example 2 This embodiment provides a computer system, which may be a server or a terminal, and its internal structure diagram may be as follows: Figure 2 As shown. The computer system includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer system is used to provide computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer system is used to exchange information between the processor and an external device. The communication interface of the computer system is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training data acquisition method for a biochar yield prediction model is implemented.

[0057] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0058] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0059] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0060] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0061] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for obtaining training data for a biochar yield prediction model. It is characterized in that The training data acquisition method of the biochar yield prediction model includes: Determine the raw material characteristics of the target biomass, and obtain n groups of original data of the raw material characteristics; the raw material characteristics include m element indicators; n>m>3; Using the Ward strategy in the agglomerative hierarchical clustering algorithm, the n groups of raw data of the raw material characteristics are divided into m clusters, and the centroid and the error square sum of each cluster are calculated; one cluster corresponds to one element index; one element index corresponds to n raw data; Compare the sum of squared errors of all clusters, merge the two clusters with the smallest sum of squared errors, and recalculate the centroid and sum of squared errors of the merged cluster; Repeat the step of "comparing the sum of squared errors of all clusters, merging the two clusters with the smallest sum of squared errors, and recalculating the centroid and sum of squared errors of the merged cluster", and continue merging clusters until three clusters remain; the remaining three clusters are the first cluster, the second cluster, and the third cluster; the element indices in the first cluster, the second cluster, and the third cluster all meet the first setting condition; the first setting condition is that the number of element indices is greater than or equal to 1 and less than m-2; Normalize the original data in the remaining three clusters, and calculate the weights of the element indicators in the remaining three clusters based on the normalized data in the remaining three clusters; Based on the normalized data in the remaining three clusters and the weights of the element indicators in the remaining three clusters, calculate n comprehensive indicator data of the first cluster, n comprehensive indicator data of the second cluster, and n comprehensive indicator data of the third cluster; Design data sets in various combinations, and use the data sets in different combinations to pre-train the biochar yield prediction model, and determine the combination of data sets corresponding to the maximum fitting accuracy and the minimum error as the optimal data combination; the biochar yield prediction model is a random forest model; the data sets in various combinations are respectively the n comprehensive index data of the first cluster, the n comprehensive index data of the second cluster, the n comprehensive index data of the third cluster, the n comprehensive index data of the first cluster and the second cluster, the n comprehensive index data of the first cluster and the third cluster, the n comprehensive index data of the second cluster and the third cluster, and the n comprehensive index data of the first cluster, the second cluster and the third cluster; the optimal data combination is used for determining the training data of the subsequent biochar yield prediction model.

2. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that m=10。 3. The method for obtaining training data for the biochar yield prediction model according to claim 2, It is characterized in that The element indicators in the raw material characteristics are carbon, hydrogen, oxygen, nitrogen, cellulose, hemicellulose, lignin, ash, volatile matter and fixed carbon content.

4. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that The centroid of the merged cluster is calculated as: ； In the formula, For the i Cluster and j The centroid of the clusters after merging, For the i The amount of data in a cluster, For the j The amount of data in a cluster, For the i The centroid of the cluster, For the j The centroid of a cluster.

5. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that The formula for calculating the sum of squared errors of the merged clusters is: ； ； In the formula, For the i Cluster and j The sum of squared errors after merging clusters is For the i The sum of squared errors of clusters, For the j The sum of squared errors of clusters, For the i Cluster and j The error sum of squares after merging clusters is increased. For the i The amount of data in a cluster, For the j The amount of data in a cluster, For the i The centroid of the cluster, For the j The centroid of a cluster.

6. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that The element indicators in the first cluster include: lignin, hemicellulose, ash, fixed carbon, hydrogen and nitrogen; the element indicators in the second cluster include: cellulose, carbon and oxygen; the element indicators in the third cluster are volatile matter.

7. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that The formula for normalizing the original data in the remaining three clusters is: ； In the formula, For a cluster j The element index i Normalized data, For a cluster j The element index i The original data, For a cluster j The minimum raw data of element indicators, For a cluster j The maximum raw data of element indicators.

8. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that The weight calculation formula for each element index in the remaining three clusters is: ； ； ； In the formula, For a cluster j The weight of the element index, For a cluster j The entropy of the element index, m is the number of element indices in a cluster, For a cluster j The element index i The proportion of normalized data, n For a cluster j The normalized data volume of element indicators, For a cluster j The element index i Normalized data.

9. The method for obtaining training data for the biochar yield prediction model according to claim 1, It is characterized in that The calculation formula of the comprehensive indicator data is: ； In the formula, For a cluster i Comprehensive indicator data, The first index of the first element in a cluster i Normalized data, is the weight of the first element index in a cluster, The index of the second element in a cluster. i Normalized data, is the weight of the second element index in a cluster, For a cluster j The element index i Normalized data, A cluster j The weight of the element indicator.

10. A computer system, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the training data acquisition method for the biochar yield prediction model described in any one of claims 1 to 9.

Images