Biochar adsorption heavy metal prediction system and method based on multi-element machine learning framework

Abstract

The invention relates to the technical field of biochar prediction, in particular to a biochar adsorption heavy metal prediction system and method based on a multi-element machine learning framework, and the system comprises a client which is developed by adopting a go language, achieves the operation of a user on a platform, and enables a plurality of users to carry out learning training at the same time; the task scheduler is used for managing and distributing learning tasks, receiving sample data reported by the client, storing the sample data in a database and generating corresponding data IDs; the database is used for uniformly storing the training data uploaded by the user and the trained model; and the computing node is developed by adopting a python language, executes training on experimental data, and reports a training result and a generation model to the task scheduling unit. The model training time is shortened through parallel learning of multiple models, so that the learning efficiency is improved, meanwhile, the system also provides good interactive design to facilitate user operation, and the use difficulty of machine learning is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of biochar prediction, and in particular to a system and method for predicting heavy metal adsorption by biochar based on a multivariate machine learning framework. Background Art

[0002] Heavy metals are pollutants that are seriously toxic to water and soil, and have a negative impact on human health and the ecological environment. There are many methods currently used to treat heavy metals, including chemical precipitation, ion exchange, and membrane technology, but these methods have problems such as high cost and unsatisfactory results. Compared with these traditional methods, adsorption is considered to be a more promising method for the treatment of heavy metals due to factors such as cost and efficiency. At present, biochar prepared from biomass materials has attracted much attention. Biochar plays an important role in heavy metal removal because it is environmentally friendly, cheap and has a high removal efficiency. Biochar has a variety of uses due to the presence of oxygen and nitrogen functional groups. The strong physical and chemical properties of biochar make it an ideal adsorbent for treating heavy metals. Machine learning algorithms are often used for prediction and classification. The use of machine learning prediction models can reduce the workload, cost, and time consumption in experiments.

[0003] The existing prediction methods for biochar by machine learning use a single model and adopt a manual parameter adjustment method. Machine learning models usually have many parameters, and the change of parameters also has an important impact on the effect of the model. It takes a lot of time to get a better model by manual parameter adjustment. The process of model parameter adjustment has no strict regularity, so each parameter can only be adjusted over a large range. At the same time, the model parameters are not universal for different prediction samples and prediction targets, so the model parameters need to be re-tuned for different experimental scenarios.

[0004] In machine learning, the choice of model has a great impact on the prediction results. Therefore, in order to obtain a better model, it is necessary to try a lot of new models, which is accompanied by the process of model parameter tuning, making model training extremely time-consuming.

[0005] In response to the existing problems, the present invention proposes a system and method for predicting heavy metal adsorption by biochar based on a multivariate machine learning framework, provides multiple machine learning frameworks, efficiently and quickly obtains the efficiency of multiple models in predicting heavy metals on biochar, uses machine learning to better understand the effectiveness of biochar, and reduces energy, time and cost. Summary of the invention

[0006] The purpose of the present invention is to provide a prediction system and method for heavy metal adsorption by biochar based on a multivariate machine learning framework, providing a variety of machine learning frameworks, including: Auto-Sklearn, TPOT, Auto-Keras, FeatureTools, MLBox, Lightwood, Pycaret, XGBoost, etc. to achieve automatic model parameter tuning, and perform parallel learning on multiple models, thereby improving learning efficiency. At the same time, the system also provides good interactive design to facilitate user operation, greatly reducing the difficulty of using machine learning.

[0007] The present invention shortens the model training time by learning multiple models in parallel. The existing solution is to achieve higher model accuracy by manually debugging parameters. There is no obvious pattern in parameter debugging of machine learning for any scenario. Manual debugging only sets a parameter and manually checks the model accuracy. The present invention can automatically implement this process in batches through code by building a platform.

[0008] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0009] A prediction system for heavy metal adsorption by biochar based on a multivariate machine learning framework, comprising:

[0010] The client is developed in Go language to enable users to operate the platform. The platform can be used by multiple users for learning and training at the same time.

[0011] The task scheduler is responsible for managing and distributing learning tasks, receiving sample data reported by the client and saving it in the database, and generating corresponding data IDs;

[0012] Database, which uniformly stores the training data uploaded by users and the trained models;

[0013] The computing node, developed in Python, performs training on the experimental data and reports the training results and generated models to the task scheduling unit.

[0014] Furthermore, the client processing flow specifically includes:

[0015] S1: The client receives the Excel file uploaded by the user. The collected values ​​of the data samples may not be standard computer-recognizable values ​​due to manual entry. Therefore, the data format needs to be verified first, and some convertible non-standard data needs to be converted into standard data. At the same time, for some non-convertible data, the user needs to be prompted to modify it.

[0016] S2: After the data sample is converted, the sample data is uploaded to the task scheduling module and saved in the database.

[0017] S3: The user determines the input parameters and prediction parameters from the fields of the uploaded sample through the interface.

[0018] S4: Setting training parameters, which include the framework used and parameters corresponding to the framework.

[0019] S5: The user clicks the start button on the interface to start training.

[0020] S6: After waiting for the training to be completed, the interface outputs the training results of each model, and the user can select a model to save based on the evaluation parameters of the output results.

[0021] Furthermore, the requirements for the content of Excel files include: no merged cells, no multi-level headers, excel should only have one sheet in use, columns of numerical type must be filled in with data using the standard counting method, otherwise data recognition errors will occur, and there must be no missing items in the data.

[0022] Furthermore, the data items in the client processing flow include:

[0023] Data sample: Extract the header data and the first two rows of data from the Excel data for display.

[0024] Label column: The client code converts the first row of data into floating-point numbers. If the conversion is successful, the column is considered to be a numeric type. If the conversion fails, the column is considered to be a label column. The numerical conversion method corresponding to the label column is: the data is converted to a numerical value in the order in which the rows appear. For example: a column is A, C, A, B, D, B from top to bottom. Then A appears first, and the value corresponding to the A label is 0. C appears second, and the value corresponding to the C label is 1. The third A appears, because the A label has been converted to 0, so the third A is converted to 0. The fourth one appears is B, and the value corresponding to the B label is 2. By analogy, the converted numerical sequence is 0, 1, 0, 2, 3, 2.

[0025] Unrecognized items: For unrecognized numerical items, their locations need to be indicated to facilitate user modification.

[0026] Total number of data rows: Displays the total number of execl rows read, excluding the table header.

[0027] Data name: The user sets a name for the data.

[0028] Furthermore, it also includes display items:

[0029] Data sample: Displays the execl table header and the first two rows of data.

[0030] Select training fields: Users select fields for training via checkboxes

[0031] Select test field: Users select test fields (prediction fields) through radio buttons

[0032] Set the training data ratio: Set the data ratio used for training, and the remaining data is used as test data.

[0033] Furthermore, the task scheduler processing flow specifically includes:

[0034] S1: First receive the sample data reported by the client and save it in the database.

[0035] S2: Receive tasks reported by the client and save them in the database.

[0036] S3: Split the task into multiple subtasks according to different model frameworks.

[0037] S4: Send the subtask to the computing node according to the busy status of the computing node.

[0038] S5: Receive the training results of each computing node and save them to the database.

[0039] The computing node receives the task information sent by the task scheduler; starts the conda corresponding environment according to the task information, then starts the script corresponding to the task, and passes the task data into the script process to execute model training; after the training is completed, the computing node reads the training results and reports them to the scheduling node for storage.

[0040] Furthermore, the task information of the task scheduler includes:

[0041] Task ID: The self-increment ID in the database main task table is used as the task ID;

[0042] Data ID: the data ID generated in association 1;

[0043] Data name: the data name set by the user when uploading data;

[0044] Framework parameters: a summary of all framework parameters enabled by the user in this task;

[0045] Framework name: the names of all frameworks started by the user in this task;

[0046] Training data ratio: the ratio of data used for training;

[0047] Training field: the field index value used for training, index value in the order of Excel columns;

[0048] Prediction field: the field index value used for prediction, index value in the order of Excel columns;

[0049] Task status: The initial status of the task is "running";

[0050] Best R 2 : The initial optimal R of the task 2 value, the initial value is 0;

[0051] Task creation time: The timestamp of task creation, used to detect whether the task has timed out;

[0052] Task completion time: the timestamp of task completion;

[0053] At the same time, the scheduler maintains a main task cache table in memory to determine whether the main task has ended or timed out based on the reported information of the subtask. The main task cache table has the following fields:

[0054] Task ID: main task ID;

[0055] Number of subtasks: the total number of subtasks split according to the framework;

[0056] Number of completed subtasks: the number of subtasks whose results have been reported;

[0057] Task creation time: main task creation timestamp;

[0058] Best R 2 : The initial optimal R of the task 2 value;

[0059] Furthermore, the task scheduler splits the task into multiple subtasks according to different model frameworks. The subtask information includes:

[0060] Subtask ID: The self-increment ID in the database subtask table is used as the subtask ID;

[0061] Main task ID: The task ID generated in 2, used to associate the subtask with the main task;

[0062] Data ID: the data ID generated in association 1;

[0063] Subtask parameters: parameters set by the user for this framework;

[0064] Framework name: the name of the framework;

[0065] Subtask status: The initial status of a subtask is "running";

[0066] Subtask creation time: The timestamp when the subtask was created.

[0067] After receiving the data reported by the subtask, the task scheduler parses the result data reported by each subtask and extracts R 2 field, determine the "best R 2" is less than this value, if so, replace it with this value. At the same time, "Number of completed subtasks" is increased by 1. If "Number of completed subtasks" is equal to "Number of subtasks", the main task is considered to have been completed, and the "Task Status" of the main task in Update 2 is "Completed", and "Best R 2 " is the "best R" in the main task cache table 2 ”, set the “Task completion time” to the current timestamp.

[0068] If a subtask crashes during operation, it may result in the subtask not reporting any result information. The task scheduler will determine whether the main task has timed out every 10 minutes. The timeout is set to 1 hour. If the main task runs for more than 1 hour, the "Task Status" of the main task in Update 2 is set to "Timed Out" and the "Best R 2 " is the "best R" in the main task cache table 2 ”, set “Task completion time” to the current timestamp.

[0069] Finally, delete the task from the main task cache table.

[0070] Furthermore, the computing node specifically includes:

[0071] The computing node receives the task information sent by the task scheduler. It first parses the subtask usage framework name, main task id, and subtask id from the received json task information, creates a directory to save the subtask result file, and the directory structure is: " / tmp / $subtask_id / ", where $subtask_id is the id value of the subtask. Finally, the original task information is written to ". / task / $subtask_id" ($subtask_id is the same as above).

[0072] The computing node starts the conda environment corresponding to the framework name in 1 and runs the corresponding python script. The python script first reads the task parameters from ". / task / $subtask_id" in 1, then sets the framework parameters according to the task parameters and runs.

[0073] After the training is completed, the Python script writes the training results to the files in the " / tmp / $subtask_id / " directory in 1, including: trained model data, model performance indicator data, model performance comparison chart, etc. Finally, the computing node reads the data in each file and converts it into JSON data and reports it to the task scheduler through the TCP connection.

[0074] On the other hand, the present invention proposes a method for predicting heavy metal adsorption by biochar based on the above system, comprising the following steps:

[0075] S1: The client receives the biochar characteristic data sample uploaded by the user. After the user uploads the file and selects the target Excel file, the file data is parsed on the client and the summary information of the file is displayed;

[0076] S2: The client code converts the first row of data into floating-point numbers. If the conversion is successful, the column is considered to be a numeric type. If the conversion fails, the column is considered to be a label column. After the user modifies the unrecognized items and sets the data name, the data is submitted.

[0077] S3: After the data sample is converted, the sample data is uploaded to the task scheduling module and saved in the database. The user enters the training parameter setting page to select the training field, select the test field, and set the training data ratio.

[0078] S4: Set training parameters, including the framework used and the parameters corresponding to the framework, submit to the task scheduler, and create a training task;

[0079] S5: The task scheduler splits the task into multiple subtasks according to different model frameworks, converts the subtask data into JSON format and transmits it to the computing node via TCP connection, then receives the training results of each computing node and saves them to the database;

[0080] S6: The user views the training results through the task result page.

[0081] Furthermore, the biochar characteristics in step S1 include: pH of the biochar in water (pHH2O), specific surface area (SA, m 2 / g), cation exchange capacity (CEC, coml(+) / kg), ash content (ash,%), biochar particle size (PS, mm), mass fraction of total carbon in biochar (C%), molar ratio of oxygen, nitrogen and carbon (O+N) / C, oxygen and carbon molar ratio (O / C) and hydrogen and carbon molar ratio (H / C); adsorption conditions include: solution pH (pHsol), adsorption temperature (T, ℃), initial concentration ratio of heavy metals to biochar (C0, mmol / g).

[0082] Beneficial effects of the present invention:

[0083] The present invention significantly improves the model training efficiency by introducing multiple machine learning frameworks (such as Auto-Sklearn, TPOT, XGBoost, etc.) and implementing automatic parameter tuning. The parallel learning mechanism of the multi-framework enables multiple models to be trained at the same time, thereby significantly shortening the training time of the model. Traditional methods often rely on a single model and manually tune parameters, which is time-consuming and prone to human bias. The present invention splits the training task into multiple subtasks through a task scheduler, which can be processed in parallel for different models. The inherent mechanism behind it is to maximize the utilization of computing nodes through the optimal configuration of resources, thereby effectively improving the ability of data processing.

[0084] The user-friendly interactive design greatly reduces the threshold for using machine learning, allowing more users with non-professional backgrounds to smoothly upload data, select models, and set parameters. The client performs strict format verification and data processing on Excel files uploaded by users to ensure data standardization and consistency. This standardized data processing mechanism not only improves data quality, but also avoids errors in the training process caused by non-standard data formats, thereby enhancing the stability of the model and prediction accuracy. For example, the system will automatically convert non-standard values ​​into recognizable formats and prompt users to adjust non-convertible data to ensure the rigor of each step of processing.

[0085] The system automation and intelligent combination of the present invention makes model selection and parameter tuning not only efficient but also intelligent. Through the automatic tuning of the algorithm, the system can intelligently select the model that best suits the data features and task requirements in a variety of frameworks. The inherent mechanism of this process is that the machine learning algorithm can self-optimize and continuously adjust parameters based on previous training results to achieve the best output. This intelligent process not only reduces the user's workload in model selection and tuning, but also enables accurate model prediction through efficient algorithms.

[0086] The high efficiency of the task scheduler in resource management enables the system to adapt to different computing needs. By dynamically monitoring the status of computing nodes, the task scheduler can allocate and optimize computing resources in real time to adapt to different user needs and computing loads. This mechanism ensures good response and processing capabilities even in high concurrency situations, reducing task delays or failures caused by system overload. In addition, the timeout management function of the task scheduler can also promptly identify and handle tasks that have not responded for a long time, further improving the reliability and availability of the system.

[0087] In summary, the present invention not only improves the efficiency and accuracy of the prediction of heavy metal adsorption by biochar, but also greatly reduces the user's operation complexity through automation and intelligent means, making it more practical and flexible in practical applications.

[0088] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0090] Figure 1 It is a schematic diagram of the overall structure of the prediction system of the present invention;

[0091] Figure 2 This is a schematic diagram of the client processing flow of the present invention;

[0092] Figure 3 A schematic diagram of the task scheduler processing logic of the present invention;

[0093] Figure 4 A logical schematic diagram of a computing node of the present invention;

[0094] Figure 5 This is a working diagram of the machine learning platform of the present invention. Figure 1 ;

[0095] Figure 6 This is a working diagram of the machine learning platform of the present invention. Figure 2 ;

[0096] Figure 7 This is a working diagram of the machine learning platform of the present invention. Figure 3 ;

[0097] Figure 8 This is a working diagram of the machine learning platform of the present invention. Figure 4 ;

[0098] Fig. 9 This is a working diagram of the machine learning platform of the present invention. Figure 5 ;

[0099] Fig.10 This is a working diagram of the machine learning platform of the present invention. Figure 6 ;

[0100] Fig.11 This is a working diagram of the machine learning platform of the present invention. Figure 7 . DETAILED DESCRIPTION

[0101] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0102] Example 1

[0103] A biochar heavy metal adsorption prediction system based on a multivariate machine learning framework as described in this embodiment includes:

[0104] The client is developed in Go language to enable users to operate the platform. The platform can be used by multiple users for learning and training at the same time.

[0105] The task scheduler is responsible for managing and distributing learning tasks, receiving sample data reported by the client and saving it in the database, and generating corresponding data IDs;

[0106] Database, which uniformly stores the training data uploaded by users and the trained models;

[0107] The computing node, developed in Python, performs training on the experimental data and reports the training results and generated models to the task scheduling unit.

[0108] In this embodiment, the client receives an Excel file uploaded by the user. The collected values ​​of the data samples may not be standard computer-recognizable values ​​due to manual entry. Therefore, the data format needs to be verified first, and some convertible non-standard data needs to be converted into standard data. At the same time, the user needs to be prompted to modify some non-convertible data.

[0109] The requirements for the content of Excel files include: no merged cells, no multi-level headers, only one sheet should be used in excel, and columns of numerical type must be filled in with data using the standard counting method, otherwise data recognition errors will occur, and there cannot be missing items in the data.

[0110] like Figure 5 As shown, after the user clicks on file upload and selects the target Excel file, the file data is parsed on the client and the summary information of the file is displayed.

[0111] like Figure 6 As shown, the data items include:

[0112] Data sample: Extract the header data and the first two rows of data from the Excel data for display.

[0113] Label column: The client code converts the first row of data into floating-point numbers. If the conversion is successful, the column is considered to be a numeric type. If the conversion fails, the column is considered to be a label column. The numerical conversion method corresponding to the label column is: the data is converted to a numerical value in the order in which the rows appear. For example: a column is A, C, A, B, D, B from top to bottom. Then A appears first, and the value corresponding to the A label is 0. C appears second, and the value corresponding to the C label is 1. The third A appears, because the A label has been converted to 0, so the third A is converted to 0. The fourth one appears is B, and the value corresponding to the B label is 2. By analogy, the converted numerical sequence is 0, 1, 0, 2, 3, 2.

[0114] Unrecognized items: For unrecognized numerical items, their locations need to be indicated to facilitate user modification.

[0115] Total number of data rows: Displays the total number of execl rows read, excluding the table header.

[0116] Data name: The user sets a name for the data.

[0117] After the user has modified the unrecognized items and set the data name, submit the data.

[0118] like Figure 7 As shown in the figure, a row of new data is added to the "Data List" page.

[0119] After the data sample is converted, the sample data is uploaded to the task scheduling module and saved in the database.

[0120] The user clicks the "Train" button to enter the training parameter setting page, such as Figure 8 shown.

[0121] The exhibits include:

[0122] Data sample: Displays the execl table header and the first two rows of data.

[0123] Select training fields: Users select fields for training through check boxes;

[0124] Select test field: Users select test fields (prediction fields) through radio buttons;

[0125] Set the training data ratio: Set the data ratio used for training, and the remaining data is used as test data.

[0126] like Fig. 9 As shown, the user opens the training framework to be used and sets the training parameters of each training framework.

[0127] The user clicks the "Submit" button to create the training task.

[0128] Task scheduler processing logic:

[0129] The task scheduler first receives the sample data reported by the client and saves it in the database, and generates the corresponding data ID.

[0130] The task scheduler receives the tasks reported by the client and saves them in the database.

[0131] The task information includes:

[0132] Task ID: The self-increment ID in the database main task table is used as the task ID;

[0133] Data ID: The data ID generated in association 1.

[0134] Data name: the data name set by the user when uploading data;

[0135] Framework parameters: a summary of all framework parameters enabled by the user in this task;

[0136] Framework name: the names of all frameworks started by the user in this task;

[0137] Training data ratio: the ratio of data used for training;

[0138] Training field: the field index value used for training, index value in the order of Excel columns;

[0139] Prediction field: the field index value used for prediction, index value in the order of Excel columns;

[0140] Task status: The initial status of the task is "running";

[0141] Best R 2 : The initial optimal R of the task 2 value, the initial value is 0;

[0142] Task creation time: The timestamp of task creation, used to detect whether the task has timed out;

[0143] Task completion time: the timestamp of task completion;

[0144] At the same time, the scheduler maintains a main task cache table in memory to determine whether the main task has ended or timed out based on the reported information of the subtask. The main task cache table has the following fields:

[0145] Task ID: main task ID;

[0146] Number of subtasks: the total number of subtasks split according to the framework;

[0147] Number of completed subtasks: the number of subtasks whose results have been reported;

[0148] Task creation time: main task creation timestamp;

[0149] Best R 2 : The initial optimal R of the task 2 value;

[0150] The task scheduler splits the task into multiple subtasks according to different model frameworks.

[0151] Subtask information includes:

[0152] Subtask ID: The database subtask table automatically increments the ID as the subtask ID

[0153] Main task ID: The task ID generated in 2, used to associate the subtask with the main task. Data ID: The data ID generated in 1.

[0154] Subtask parameters: parameters set by the user for this framework

[0155] Framework Name: The framework name

[0156] Subtask status: The initial status of the subtask is "running"

[0157] Subtask creation time: The timestamp when the subtask was created.

[0158] The task scheduler sends the subtasks to the computing nodes.

[0159] The scheduler converts the subtask data into JSON format and transmits it to the computing node through a TCP connection.

[0160] The task scheduler receives the training results of each computing node and saves them to the database. Fig.10 shown.

[0161] After receiving the data reported by the subtask, the task scheduler parses the result data reported by each subtask and extracts R 2 field, determine the "best R 2 " is less than this value, if so, replace it with this value. At the same time, "Number of completed subtasks" is increased by 1. If "Number of completed subtasks" is equal to "Number of subtasks", the main task is considered to have been completed, and the "Task Status" of the main task in Update 2 is "Completed", and "Best R 2 " is the "best R" in the main task cache table 2 ”, set the “Task completion time” to the current timestamp.

[0162] If a subtask crashes during operation, it may result in the subtask not reporting any result information. The task scheduler will determine whether the main task has timed out every 10 minutes. The timeout is set to 1 hour. If the main task runs for more than 1 hour, the "Task Status" of the main task in Update 2 is set to "Timed Out" and the "Best R 2 " is the "best R" in the main task cache table 2 ”, set “Task completion time” to the current timestamp.

[0163] Finally, delete the task from the main task cache table.

[0164] Users can view the training results through the task result page, such as Fig.11 shown.

[0165] Example 2

[0166] The method for predicting heavy metal adsorption by biochar as described in this embodiment comprises the following steps:

[0167] S1: The client receives the biochar characteristic data sample uploaded by the user. After the user uploads the file and selects the target Excel file, the file data is parsed on the client and the summary information of the file is displayed;

[0168] S2: The client code converts the first row of data into floating-point numbers. If the conversion is successful, the column is considered to be a numeric type. If the conversion fails, the column is considered to be a label column. After the user modifies the unrecognized items and sets the data name, the data is submitted.

[0169] S3: After the data sample is converted, the sample data is uploaded to the task scheduling module and saved in the database. The user enters the training parameter setting page to select the training field, select the test field, and set the training data ratio.

[0170] S4: Set training parameters, including the framework used and the parameters corresponding to the framework, submit to the task scheduler, and create a training task;

[0171] S5: The task scheduler splits the task into multiple subtasks according to different model frameworks, converts the subtask data into JSON format and transmits it to the computing node via TCP connection, then receives the training results of each computing node and saves them to the database;

[0172] S6: The user views the training results through the task result page.

[0173] Furthermore, the biochar characteristics in step S1 include: pH of the biochar in water (pHH2O), specific surface area (SA, m 2 / g), cation exchange capacity (CEC, coml(+) / kg), ash content (ash,%), biochar particle size (PS, mm), mass fraction of total carbon in biochar (C%), molar ratio of oxygen, nitrogen and carbon (O+N) / C, oxygen and carbon molar ratio (O / C) and hydrogen and carbon molar ratio (H / C); adsorption conditions include: solution pH (pHsol), adsorption temperature (T, ℃), initial concentration ratio of heavy metals to biochar (C0, mmol / g).

[0174] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A prediction system for heavy metal adsorption by biochar based on a multivariate machine learning framework, characterized in that: include: The client is developed in Go language to enable users to operate the platform. The platform can be used by multiple users for learning and training at the same time. The task scheduler is responsible for managing and distributing learning tasks, receiving sample data reported by the client and saving it in the database, and generating corresponding data IDs; Database, which uniformly stores the training data uploaded by users and the trained models; The computing node, developed in Python, performs training on the experimental data and reports the training results and generated models to the task scheduling unit.

2. The biochar adsorption heavy metal prediction system based on a multivariate machine learning framework as claimed in claim 1, characterized in that: The client processing flow specifically includes: S1: The client receives the Excel file uploaded by the user. The collected values ​​of the data sample are not standard computer-recognizable values. Therefore, the data format needs to be verified first, and some convertible non-standard data needs to be converted into standard data. At the same time, the user needs to be prompted to modify some non-convertible data. S2: After data sample conversion, the sample data is uploaded to the task scheduling module and saved in the database; S3: The user determines the input parameters and prediction parameters from the fields of the uploaded sample through the interface; S4: Setting training parameters, which include the framework used and parameters corresponding to the framework; S5: The user clicks the start button on the interface to start training; S6: After waiting for the training to be completed, the interface outputs the training results of each model, and the user can select a model to save based on the evaluation parameters of the output results.

3. The biochar adsorption heavy metal prediction system based on a multivariate machine learning framework as claimed in claim 1, characterized in that: The data items in the client processing flow include: Data sample: extract the header data and the first two rows of data from the Excel data for display; Label column: The client code converts the first row of data into floating-point numbers. If the conversion is successful, the column is considered to be a numeric type. If the conversion fails, the column is considered to be a label column. The numerical conversion method corresponding to the label column is: the data is converted into numerical values ​​in the order in which the rows appear. Unrecognized items: For unrecognized numerical items, their locations need to be indicated to facilitate user modification; Total number of data rows: Displays the total number of execl rows read, excluding the table header; Data name: The user sets the name for the data; Also included are the following display items: Data sample: Displays the execl table header and the first two rows of data; Select training fields: Users select fields for training through check boxes; Select test field: Users select test fields through radio buttons; Set the training data ratio: Set the data ratio used for training, and the remaining data is used as test data.

4. The biochar adsorption heavy metal prediction system based on a multivariate machine learning framework as claimed in claim 1, characterized in that: The task scheduler processing flow specifically includes: S1: First, receive the sample data reported by the client and save it in the database; S2: Receive tasks reported by the client and save them in the database; S3: Split the task into multiple subtasks according to different model frameworks; S4: Send the subtask to the computing node according to the busy status of the computing node; S5: Receive the training results of each computing node and save them to the database; The computing node receives the task information sent by the task scheduler; starts the conda corresponding environment according to the task information, then starts the script corresponding to the task, and passes the task data into the script process to execute model training; after the training is completed, the computing node reads the training results and reports them to the scheduling node for storage.

5. The biochar adsorption heavy metal prediction system based on a multivariate machine learning framework as claimed in claim 1, characterized in that: After receiving the data reported by the subtask, the task scheduler parses the result data reported by each subtask and extracts R 2 field, determine the "best R" in the main task cache table in 2 2 " is less than this value, if so, replace it with this value; at the same time, "Number of completed subtasks" is increased by 1; if "Number of completed subtasks" is equal to "Number of subtasks", the main task is considered to have been completed, and the "Task Status" of the main task in Update 2 is "Completed", and "Best R 2 " is the best R in the main task cache table 2 ", set "Task completion time" to the current timestamp; The task scheduler will check whether the main task has timed out every 10 minutes. The timeout period is set to 1 hour. If the main task runs for more than 1 hour, the "task status" of the main task in Update 2 will be set to "timed out" and the "best R 2 " is the best R in the main task cache table 2 ", set "Task completion time" to the current timestamp; Finally, delete the task from the main task cache table.

6. The biochar adsorption heavy metal prediction system based on a multivariate machine learning framework as claimed in claim 1, characterized in that: The computing nodes specifically include: The computing node receives the task information sent by the task scheduler. First, it parses the subtask usage framework name, main task ID, and subtask ID from the received JSON task information, creates a directory to save the subtask result file, and the directory structure is: " / tmp / $subtask_id / ", where $subtask_id is the ID value of the subtask. Finally, the original task information is written to ". / task / $subtask_id"; The computing node starts the conda environment corresponding to the framework name in 1 and runs the corresponding python script; the python script first reads the task parameters from ". / task / $subtask_id" in 1, then sets the framework parameters according to the task parameters and runs; After the training is completed, the Python script writes the training results to the files in the " / tmp / $subtask_id / " directory in 1, including: trained model data, model performance indicator data, model performance comparison chart, etc. Finally, the computing node reads the data in each file and converts it into JSON data and reports it to the task scheduler through the TCP connection.

7. A method for predicting heavy metal adsorption by biochar based on the system according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1: The client receives the biochar characteristic data sample uploaded by the user. After the user uploads the file and selects the target Excel file, the file data is parsed on the client and the summary information of the file is displayed; S2: The client code converts the first row of data into floating-point numbers. If the conversion is successful, the column is considered to be a numeric type. If the conversion fails, the column is considered to be a label column. After the user modifies the unrecognized items and sets the data name, the data is submitted. S3: After the data sample is converted, the sample data is uploaded to the task scheduling module and saved in the database. The user enters the training parameter setting page to select the training field, select the test field, and set the training data ratio; S4: Set training parameters, including the framework used and the parameters corresponding to the framework, submit to the task scheduler, and create a training task; S5: The task scheduler splits the task into multiple subtasks according to different model frameworks, converts the subtask data into JSON format and transmits it to the computing node via TCP connection, then receives the training results of each computing node and saves them to the database; S6: The user views the training results through the task result page.

8. The biochar adsorption heavy metal prediction system based on a multivariate machine learning framework as claimed in claim 7, characterized in that: The biochar characteristics in step S1 include: biochar in water, specific surface area, cation exchange capacity, ash content (ash,%), biochar particle size, mass fraction of total carbon in biochar, oxygen-nitrogen-carbon molar ratio, oxygen-carbon molar ratio and hydrogen-carbon molar ratio; and also include adsorption conditions: solution, adsorption temperature, initial concentration ratio of heavy metals to biochar.

Images