Methods for building multi-task learning models

Abstract

The embodiment of the present application provides a method for building a multi-task learning model, the method comprising: collecting catalyst pilot evaluation data, the data comprising raw material data, operating parameters, and the content of at least two components in the raw material after the catalyst participates in the reaction; using the evaluation data, building a data-driven model for predicting the content of the at least two components; determining the weight coefficient of the total loss function in the data-driven model according to the distribution law of the content of the at least two components after the catalyst participates in the reaction; and using the weight coefficient, modifying the total loss function to build a new data-driven model. Through the above technical solution, according to the distribution characteristics of each task, the weight coefficient of each task variance is determined, so that the model reaches gradient balance during training, solving the problem of unbalanced relative error distribution when predicting multiple targets at the same time.

Description

Technical Field

[0001] The present application relates to the field of chemical engineering, and specifically to a method for constructing a multi-task learning model, a method for predicting component content in catalyst evaluation using the multi-task learning model, and a device for predicting component content in catalyst evaluation. Background Art

[0002] As a crucial component of the refining and chemical industry, catalyst development and utilization typically require bench-scale and pilot-scale evaluations before being deployed in industrial plants. Pilot-scale catalyst evaluation data serves as a crucial reference for industrial application. Building models to replace some of the time-consuming and laborious evaluation processes and explore catalyst performance is becoming a pressing need for process R&D personnel. Data models, due to their ease of construction and ease of use, have become an integral part of pilot-scale catalyst evaluation data models.

[0003] Currently, the models developed for pilot-scale evaluation data of wax oil hydrogenation catalysts are primarily mechanistic models. These models offer strong interpretability and clear processes, making them useful for developing catalyst evaluation models. However, when simultaneously predicting the sulfur and nitrogen contents of hydrogenated wax oil (i.e., wax oil after catalytic reaction), the interaction between the two must be considered. The resulting functional expressions are complex, making it difficult to solve for kinetic model parameters and inconvenient for practical application.

[0004] Using data from pilot-scale evaluations of wax oil hydrogenation catalysts, a model built using the Keras deep learning framework can simultaneously predict the sulfur and nitrogen contents of hydrogenated wax oil. Because the model's inherent total loss function is the sum of the variances between the predicted and true values for each task, when the data distributions of the sulfur and nitrogen contents differ significantly, the model can experience gradient imbalance during training, leading to large differences in the relative errors of the sulfur and nitrogen contents. Therefore, this paper proposes a new method for constructing a total loss function. Applying this method to the model can, to a certain extent, address this gradient imbalance. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method for constructing a multi-task learning model, a method for predicting component content in catalyst evaluation using the multi-task learning model, and an apparatus for predicting component content in catalyst evaluation.

[0006] In order to achieve the above-mentioned objectives, the first aspect of the present application provides a method for constructing a multi-task learning model, the method comprising: collecting catalyst pilot evaluation data, which data includes raw material data, operating parameters, and the content of at least two components in the raw material after the catalyst participates in the reaction; using the evaluation data to build a data-driven model for predicting the content of the at least two components; determining the weight coefficient of the total loss function in the data-driven model based on the distribution pattern of the content of the at least two components after the catalyst participates in the reaction; and using the weight coefficient to modify the total loss function to construct a new data-driven model.

[0007] In the embodiment of the present application, the modified total loss function is expressed as:

[0008]

[0009] Among them, L MTL (MTL, Multi task learning) is the multi-task total loss function when training the data-driven model, y ij is the measured value of the jth data in the i-th task; is the predicted value of the jth data in the i-th task; i is the number of components, and n is the number of training set data.

[0010] In the embodiment of the present application, it is assumed that the average values of the modeling data corresponding to each task in i tasks are and satisfy The weights are then determined as:

[0011]

[0012]

[0013] …

[0014]

[0015] In the examples of the present application, the catalyst pilot evaluation data is wax oil hydrogenation pilot catalyst evaluation data.

[0016] In an embodiment of the present application, the raw material data includes one or more of the following: raw material density, raw material distillation range, raw material refractive index, raw material S content, raw material N content; the operating parameters include one or more of the following: reaction pressure, reaction temperature, space velocity, hydrogen-to-oil ratio; and the contents of at least two components in the raw material after the catalyst participates in the reaction include the S content and N content in the hydrogenated wax oil.

[0017] In the embodiment of the present application, the modified total loss function is expressed as:

[0018]

[0019] in, and y 1j are the predicted and actual values of sulfur content in hydrogenated wax oil, and y 2j are the predicted and actual values of sulfur content in hydrogenated wax oil,

[0020] in, W1 represents the weight coefficient of the sulfur content variance in the total loss function, W2 represents the weight coefficient of the nitrogen content variance in the total loss function, the average sulfur content of the hydrogenated wax oil used when training the data-driven model is X, and the average nitrogen content of the hydrogenated wax oil used when training the data-driven model is Y.

[0021] A second aspect of the present application provides a method for predicting component content in catalyst evaluation, wherein the method includes using a multi-task learning model constructed by the above method to predict component content.

[0022] In a third aspect, the present application provides a device for predicting component content in catalyst evaluation, the device comprising: a memory; and a processor, the processor being configured to predict component content using the multi-task learning model constructed by the above method.

[0023] In a fourth aspect, the present application provides a machine-readable storage medium having instructions stored thereon, which, when executed by a processor, configure the processor to execute the above-mentioned method for constructing a multi-task learning model or the method for predicting component content in catalyst evaluation.

[0024] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned method for constructing a multi-task learning model or the method for predicting component content in catalyst evaluation.

[0025] Through the above technical solution, the weight coefficient of the variance of each task is determined according to the distribution characteristics of each task (for example, the content of each component to be predicted), so that the model can achieve gradient balance during the training process, solving the problem of unbalanced relative error distribution when predicting multiple targets (that is, multiple tasks) at the same time.

[0026] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present application but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0028] Figure 1 The following schematically illustrates a flow chart of a method for constructing a multi-task learning model according to an embodiment of the present application.

[0029] Figure 2 The schematic diagram shows the prediction effect of the S content and N content in hydrogenated wax oil on the test set using the constructed deep learning framework model without modifying the total loss function.

[0030] Figure 3 The schematic diagram shows the prediction effect of the S content and N content in hydrogenated wax oil on the test set using the constructed deep learning framework model after modifying the total loss function. DETAILED DESCRIPTION

[0031] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0033] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0034] It should be noted that the terms "task," "goal," and "component content" appear repeatedly in this case and are used interchangeably. The multi-task learning model in this case is applicable not only to "component content" prediction but also to other tasks, such as the prediction of various parameters such as temperature and humidity. Furthermore, the terms "model," "deep learning framework model," "multi-task learning model," and "data-driven model" appear repeatedly in this case and refer to the same model and are used interchangeably.

[0035] Figure 1 The following schematically shows a flow chart of constructing a multi-task learning model according to an embodiment of the present application. Figure 1 As shown, in one embodiment of the present application, a method for constructing a multi-task learning model is provided, and the method includes the following steps:

[0036] Step S110 , collecting catalyst pilot evaluation data, which includes raw material data, operating parameters, and the contents of at least two components in the raw material after the catalyst participates in the reaction.

[0037] Step S120: Using the evaluation data, build a data-driven model for predicting the contents of the at least two components.

[0038] Step S130: determining a weight coefficient of a total loss function in the data-driven model according to a distribution pattern of the contents of the at least two components after the catalyst participates in the reaction.

[0039] Step S140: Using the weight coefficient, modify the total loss function to construct a new data-driven model.

[0040] The present invention provides a new method for constructing a multi-task learning model for multi-task learning in neural networks. More precisely, it introduces corresponding weight coefficients for the total loss function of each task of the model. The purpose of this weight coefficient is to solve the problem that when the model is optimized for multiple tasks, the prediction error of a certain task will be prioritized at the expense of the prediction errors of other tasks, resulting in large differences in the prediction errors of each task.

[0041] Figure 1 FIG. 1 is a flow chart of a method for constructing a multi-task learning model in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0042] The following is a detailed explanation using the pilot-scale evaluation of a wax oil hydrogenation catalyst as an example, with the multi-task also illustrated using the sulfur and nitrogen content prediction in this pilot-scale evaluation. Of course, the present invention is not limited to this example, and other tasks that incorporate the core concept of the present invention (i.e., setting a weight coefficient for the total loss function to achieve gradient balance during model training and address the imbalanced relative error distribution during simultaneous multi-target prediction) are also feasible.

[0043] Specifically, the multi-task learning model can be built using Keras. Keras is a deep learning framework for building models and a high-level neural network application programming interface written in Python. It consists of a series of independent, fully configurable modules that can be assembled together according to actual needs. In particular, neural network layers, loss functions, optimizers, initialization methods, activation functions, and regularization methods are all modules that can be combined to build new models.

[0044] When building the model, Python was used to write the model, and the deep learning framework Keras was used to implement the model. Model training requires calling Excel input and output features and data separately. Input features are mainly divided into two parts: feedstock properties and operating parameters. Feedstock properties include: feedstock density, feedstock sulfur content, feedstock nitrogen content, feedstock refractive index, feedstock distillation range (IBP, 10%, 30%, ..., FBP), etc. Operating parameters include: reaction temperature, reaction pressure, space velocity, and hydrogen-to-oil ratio. Output features can be the sulfur content and nitrogen content of hydrogenated wax oil.

[0045] The features and data corresponding to the input and output are cleaned, and the features and data of the input and output are kept in one-to-one correspondence as much as possible. They are organized in Excel tables for easy reading by the algorithm program. All data are randomly divided into training set, validation set and test set in a ratio of 8:1:1. The training set data is used for model training, the validation set is used for model hyperparameter optimization, and the test set is used to test the model prediction effect. The evaluation indicators of the model are mean absolute error (MAE), mean relative error (MRE) and determination coefficient R 2 The relevant statistical parameters required to establish the model are as follows, among which,

[0046]

[0047]

[0048] In the above formula, n is the number of test set samples, y i,actual is the true value of the sample, y i,predicted is the sample prediction value, is the average true value of the sample, MAE reflects the degree to which all sample prediction values deviate from the true value, MRE reflects the credibility of all sample prediction results, R 2 It reflects the degree of fit between the model prediction value and the true value.

[0049] The weight coefficient expression for constructing a new loss function is as follows, where:

[0050] W1 represents the weight coefficient of the sulfur content variance in the total loss function, W2 represents the weight coefficient of the nitrogen content variance in the total loss function, the average sulfur content of the hydrogenated wax oil used when training the data-driven model is X, and the average nitrogen content of the hydrogenated wax oil used when training the data-driven model is Y.

[0051] The weight coefficients corresponding to the variances of the S and N contents in the model's total loss function are determined by the distribution patterns of the S and N contents. To eliminate the gradient differences between the S and N contents during model training, this method uses an alternating multiplication method: the weight coefficient for the S content variance is the average value of the N content in the training and validation sets divided by the sum of the average values of the S and N contents, respectively; and the weight coefficient for the N content variance is the average value of the S content in the training and validation sets divided by the sum of the average values of the S and N contents, respectively.

[0052] Through this solution, the prediction effects of the model before and after the modification of the total loss function were compared. It was found that the introduction of the new total loss function solved the problem of large relative error differences in the prediction of sulfur and nitrogen content during model training.

[0053] The present invention is further described in detail below by way of examples, but the present invention is not limited thereto. The examples may comprise the following steps:

[0054] (1) Collect the catalyst evaluation data of the pilot test of wax oil hydrogenation. The data on the raw material properties and operating conditions are used as the input data of the model and organized into Excel Table 1. The data corresponding to the nitrogen content and sulfur content in the hydrogenated wax oil are used as the output data and organized into Excel Table 2.

[0055] (2) All possible input features (feedstock properties, operating conditions) were mapped one-to-one with output features (S content and N content in hydrogenated wax oil). All data were randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The test set alone was used to examine the generalization ability of the model.

[0056] (3) Establish model M1. Use Python to write a program, build a deep learning framework, call Excel tables 1 and 2, and establish a prediction model for the S content and N content in hydrogenated wax oil. At this time, the total loss function expression used by the model is: The training set is used for model training, and the validation set is used to adjust the model's hyperparameters. The model can be considered well-trained when the evaluation metrics of the training and validation sets change with the number of iterations, and their trends and values remain roughly consistent.

[0057] (4) The input parameters and data of the test set are transferred into the model M1 to obtain the prediction results of the test set, such as Figure 2 The mean absolute error (MAE), mean relative error (MRE) and coefficient of determination R 2 It is a statistical indicator, and its prediction effect statistics are shown in Table 1.

[0058] (5) Establish model M2. Similar to step (3), the difference is that model M2 redefines the total loss function. The newly defined total loss function is:

[0059] (6) Determine the weight coefficients W1 and W2 of the S content and the N content in the new total loss function. Assume that W1 represents the weight coefficient of the sulfur content variance in the total loss function, W2 represents the weight coefficient of the nitrogen content variance in the total loss function, the average sulfur content of the hydrogenated wax oil used in training the model is X, and the average nitrogen content of the hydrogenated wax oil used in training the model is Y. The expression of the weight coefficient is:

[0060] (7) The input parameters and data of the test set are transferred into the model M2, and the prediction results of the test set are obtained as follows Figure 3 The mean absolute error (MAE), mean relative error (MRE) and coefficient of determination R 2 It is a statistical indicator, and its prediction effect statistics are shown in Table 2.

[0061] (8) Compare the prediction effects of models M1 and M2 on the test set data, Figure 2 Some points with medium S and N contents deviate far from the diagonal line. Figure 3 Some points of S content and N content are close to the diagonal line, indicating that after using the new total loss function, the deviation of the predicted values of S content and N content from the true values becomes smaller.

[0062] (9) Comparing the evaluation indicators in Table 1 and Table 2, the MAE of S content in model M2 is significantly smaller than that in model M1. In model M1, the MRE difference between S content and N content is large, while in model M2, the MRE difference between S content and N content is small.

[0063] (10) The results in Tables 1 and 2 fully demonstrate that constructing a new total loss function can effectively solve the problem of large differences in the MREs of S content and N content due to gradient imbalance during the simultaneous optimization of multiple tasks in deep learning framework models. In addition, constructing a new total loss function can further reduce the MAEs of S content and N content.

[0064] Table 1

[0065] Evaluation Metrics MAE MRE <![CDATA[R 2 ]]> Sulfur content 125.36ppm 32.6％ 0.98 Nitrogen content 45.95ppm 13.4％ 0.99

[0066] Table 2

[0067] Evaluation Metrics MAE MRE <![CDATA[R 2 ]]> Sulfur content 49.89ppm 9.98％ 0.99 Nitrogen content 38.62ppm 9.59％ 0.99

[0068] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0069] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0072] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0073] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0074] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0075] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0076] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for constructing a multi-task learning model, characterized in that: The method comprises: Collecting catalyst pilot evaluation data, the data including feedstock data, operating parameters, and the contents of at least two components in the feedstock after the catalyst participates in the reaction; Using the evaluation data, building a data-driven model for predicting the contents of the at least two components; Determining a weight coefficient of a total loss function within the data-driven model based on a distribution pattern of the contents of the at least two components after the catalyst participates in the reaction; and Using the weight coefficients, the total loss function is modified to construct a new data-driven model. Among them, the modified total loss function is expressed as: Among them, L MTL is the multi-task total loss function when training the data-driven model, MTL stands for multi-task learning, y ij is the measured value of the jth data in the i-th task; is the predicted value of the jth data of the i-th task; i is the number of components, n is the number of training set data, Among them, it is assumed that the average values of the modeling data corresponding to each task in the i tasks are and satisfy The weights are then determined as:

2. The method according to claim 1, wherein The catalyst pilot evaluation data is wax oil hydrogenation pilot catalyst evaluation data.

3. The method according to claim 2, wherein: The raw material data includes one or more of the following: raw material density, raw material distillation range, raw material refractive index, raw material S content, raw material N content; The operating parameters include one or more of the following: reaction pressure, reaction temperature, space velocity, hydrogen-to-oil ratio; and The contents of the at least two components in the raw material after the catalyst participates in the reaction include the S content and the N content in the hydrogenated wax oil.

4. The method according to claim 2 or 3, wherein: The modified total loss function is expressed as: in, and y 1j are the predicted and actual values of sulfur content in hydrogenated wax oil, and y 2j are the predicted and actual values of sulfur content in hydrogenated wax oil, W1 represents the weight coefficient of the sulfur content variance in the total loss function, W2 represents the weight coefficient of the nitrogen content variance in the total loss function, the average sulfur content of the hydrogenated wax oil used when training the data-driven model is X, and the average nitrogen content of the hydrogenated wax oil used when training the data-driven model is Y.

5. A method for predicting component content in catalyst evaluation, wherein: The method comprises using a multi-task learning model constructed according to the method of any one of claims 1 to 4 to predict component content.

6. A device for predicting component content in catalyst evaluation, characterized in that The device comprises: Memory; and A processor configured to predict component content using a multi-task learning model constructed according to the method of any one of claims 1 to 4.

7. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor is configured to execute the method for constructing a multi-task learning model according to any one of claims 1 to 4 or the method for predicting component content in catalyst evaluation according to claim 5.

8. A computer program product, comprising a computer program, which, when executed by a processor, causes the processor to be configured to perform the method for constructing a multi-task learning model according to any one of claims 1 to 4 or the method for predicting component content in catalyst evaluation according to claim 5.

Images