Training method of efficient instant learning framework and local model

Through efficient real-time learning framework and local model training method, multi-branch variable-scale convolutional units and nonlinear mapping are used to solve the problems of low model accuracy and prediction dependence caused by complex processes and dynamic characteristics in the petrochemical industry, achieving efficient and accurate sample selection and prediction.

CN118917373APending Publication Date: 2024-11-08EAST CHINA UNIV OF SCI & TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410957045.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the existing petrochemical industry, complex processes and dynamic characteristics make it difficult to establish high-precision soft measurement models. The LSTM model cannot effectively solve the problem of model degradation caused by changes in operating conditions and feed regulation. The multi-condition LSTM model prediction depends on past time. Node, the impact of operating conditions or feed quantity on the predicted indicators cannot be studied separately.

Method used

Adopting an efficient real-time learning framework, including similar sample selection module and learner module, select local model training samples through similarity calculation, sample length calculation and subset division, capture multi-scale features using multi-branch variable-scale convolutional units, and planarize The layer and the fully connected layer are nonlinearly mapped, and the learning parameters are adjusted for local model training.

Benefits of technology

It significantly improves the efficiency of instant learning and sample selection similarity, has strong nonlinear mapping capabilities, can more accurately reflect the dynamic changes of industrial processes, and solves the problems of model degradation and prediction dependence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917373A_ABST
    Figure CN118917373A_ABST
Patent Text Reader

Abstract

The invention provides an efficient instant learning framework, a local model training method and a storage medium. The efficient just-in-time learning framework comprises a similar sample selection module and a learner module. And the similar sample selection module comprises a similarity calculation unit, a sample length calculation unit and a subset division unit. The learner module comprises an input layer, a multi-branch variable scale convolution unit, a planarization layer, a full connection layer and a weighted summation layer. According to the efficient instant learning framework, the local model training method and the storage medium, the instant learning efficiency and the sample selection similarity can be remarkably improved, and the nonlinear mapping capability is relatively high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of petrochemical industry process modeling, and in particular to an efficient instant learning framework, a local model training method, and a computer-readable storage medium. Background Art

[0002] The oil industry plays a vital role in meeting global energy demand, but it also faces strict environmental protection and production standards. The large-scale, complex and information-based industrial processes have led to increasingly severe on-site environmental problems, making low-latency, low-cost and efficient soft sensing technology essential in engineering design and optimization. Data-driven machine learning methods have been widely used in petrochemical industrial processes to achieve online monitoring of industrial processes and soft sensor measurements.

[0003] However, due to the complex process and dynamic characteristics, it is difficult to establish a high-precision soft measurement model. Previous data models, such as RNN and LSTM, have the problem of gradient disappearance / explosion and cannot preserve long-term memory. Although the LSTM model can solve the problem of operating condition changes in dynamic industrial processes, it still cannot solve the problem of model degradation caused by changes in operating conditions and feed regulation. In addition, the prediction process of the multi-condition LSTM model relies on past time nodes, and it is impossible to study the impact of a certain operating condition or feed amount on related prediction indicators separately.

[0004] In order to overcome the above-mentioned defects of the prior art, this field is in urgent need of an efficient instant learning technology and a local model training technology to improve the efficiency of instant learning and the similarity of sample selection, and have strong nonlinear mapping capabilities. Summary of the invention

[0005] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or decisive elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides an efficient instant learning framework, a local model training method, and a computer-readable storage medium, which can significantly improve the efficiency of instant learning and the similarity of sample selection, and have strong nonlinear mapping capabilities.

[0007] Specifically, the efficient instant learning framework provided according to the first aspect of the present invention includes: a similar sample selection module, including a similarity calculation unit, a sample length calculation unit and a subset division unit. The similarity calculation unit calculates the similarity between the input sample and multiple historical samples in the historical sample set, and sorts each of the historical samples accordingly. The sample length calculation unit is used to adaptively select historical samples, and determine the optimal training sample length by calculating the fluctuation range and / or average change of the similarity. The subset division unit is used to cluster each of the historical samples according to the optimal length, and select the subset with the highest similarity between the cluster center and the input sample as the training sample set of the local model. The learner module includes an input layer, a multi-branch variable scale convolution unit, a flattening layer, a fully connected layer and a weighted summation layer. The multi-branch variable scale convolution unit uses convolution kernels of different sizes to capture and fuse the multi-scale features of each training sample in the training sample set in different branches of the neural network structure of the local model. The flattening layer is used to flatten the output features of each branch, and the fully connected layer is used to perform nonlinear mapping on the flattened features output by the flattening layer to obtain multiple intermediate results. The weighted summation layer is used to perform weighted summation on each intermediate result to obtain the prediction result corresponding to each training sample. The learner module adjusts the learning parameters of the local model according to the difference between the actual result and the prediction result of each training sample to train the local model.

[0008] Furthermore, in some embodiments of the present invention, the similarity is characterized by the Euclidean distance:

[0009]

[0010] Among them, D(x q ,x i ) represents the input sample x q With historical sample x i Distance similarity in multidimensional input space, W F,k is the weight of the kth feature, σ is the second weight that balances the relationship between noise and valid data, and L is the number of variables in the input sample.

[0011] Furthermore, in some embodiments of the present invention, the similarity is characterized by cosine similarity:

[0012] Similarity(x q ,x i )=λD(x q ,x i )+(1-λ)cos(θ qi )

[0013]

[0014] Among them, Similarity(x q ,x i ) represents the input sample x q With historical sample x i The composite similarity in the multidimensional input space, cos(θ qi ) represents the input sample x q With historical sample x i In the cosine similarity of the multi-dimensional input space, λ is a third weight for balancing the distance similarity and the cosine similarity.

[0015] Further, in some embodiments of the present invention, the sample length calculation unit is configured to: determine the sample length interval in which the prediction accuracy reaches a preset accuracy threshold by forward recursive analysis of the influence of the sample length on the prediction accuracy of the local model; and calculate the similarity index Γ between the input sample and the corresponding number of historical samples step by step from the lower limit of the sample length interval according to the arrangement order of each of the historical samples, so as to adaptively determine the optimal training sample length, wherein the similarity index Γ is expressed as:

[0016]

[0017] The similarity index Γ indicates the fluctuation range and / or average change of the similarity, U s is the upper limit of the sample length of the sample length interval, L s is the lower limit of the sample length of the sample length interval.

[0018] Further, in some embodiments of the present invention, the subset division unit is configured to: randomly extract a historical sample from the historical sample set X as the initial cluster center μ1 of the first cluster, and measure the similarity E between each of the historical samples in the historical sample set X and the initial cluster center μ1:

[0019]

[0020] Among them, C i is the clustering result divided from the historical sample set X, k is the preset number of subsets; select the historical sample with the greatest similarity to the initial cluster center μ1 as the initial cluster center of the next cluster, and so on, until the initial cluster centers Φ of all clusters are selected t ={μ1,μ2,......,μ k}; Calculate the mean of the coordinates of all historical samples belonging to the same cluster to iteratively update the cluster center of each cluster:

[0021]

[0022] Among them, μ j is the updated cluster center, c j is the number of sample points in the new cluster, x is the sample point in the new cluster, this formula indicates that the new cluster center is the mean of all sample points in the cluster; and in response to reaching the preset number of iterative updates, or the next round of cluster center set Φ t+1 and the cluster center set Φ of the current round t The difference between them is less than the preset threshold ε, and the clustering result C of the next round is output. t+1 ={c1,c2,......,c k}, as the training sample set of the local model.

[0023] Furthermore, in some embodiments of the present invention, the multi-branch variable-scale convolution unit is configured to: use a first convolution kernel of a larger size to capture a first feature of each training sample in the training sample set from the first branch of the neural network structure of the local model, wherein the first feature records more global information and less local information; use a second convolution kernel of a smaller size to capture a second feature of each training sample in the training sample set from the second branch of the neural network structure of the local model, wherein the second feature records less global information and more local information; and fuse the first feature and the second feature to output multi-scale features of each training sample in the training sample set.

[0024] Further, in some embodiments of the present invention, the number of branches of the neural network structure of the local model is defined as:

[0025]

[0026] Wherein, η is the number of branches, v is the dimension of the input variable of the input sample, and the size of the maximum convolution kernel of the multi-branch variable-scale convolution unit is κ max Determined as:

[0027]

[0028] Wherein, % represents a modulo operation, which is used to find the remainder after dividing two numbers, and the size attenuation factor τ between the convolution kernels of each branch is determined as:

[0029]

[0030] To ensure that the convolution kernel size of each branch decays exponentially.

[0031] Furthermore, in some embodiments of the present invention, the flattening layer is configured to: flatten the feature maps of multiple branches output by the multi-branch variable-scale convolution unit to convert them into corresponding one-dimensional feature vectors respectively; and input and output the flattened one-dimensional feature vectors into the corresponding fully connected layer.

[0032] Furthermore, in some embodiments of the present invention, the fully connected layer is configured to: l Perform nonlinear mapping to determine the intermediate result y of the corresponding branch l ′:

[0033] y′ l =ReLU(w FC × l +b FC )

[0034] Among them, w FC is the weight of the fully connected layer, b FC is the bias of the fully connected layer; and the intermediate result y l ’ Input the weighted summation layer.

[0035] Further, in some embodiments of the present invention, the weighted sum layer is configured to: l ′ is weighted summed to determine the prediction result y corresponding to the training sample:

[0036]

[0037] Wherein, ξ is the weight coefficient of each branch, which is updated as the local model is trained.

[0038] In addition, the training method of the local model provided in the second aspect of the present invention comprises the following steps: obtaining a query sample about the local model to be trained; inputting the query sample into the efficient instant learning framework as described in any one of the first aspect of the present invention, so as to sort the similarity, calculate the optimal training sample length, and cluster the multiple historical samples in the global historical sample set through its similar sample selection module, and select the subset with the highest similarity between the cluster center and the query sample as the training sample set of the local model;

[0039] Inputting each training sample in the training sample set into the AMVs-ICNN learner module of the efficient real-time learning framework in sequence to determine the corresponding prediction results respectively; and

[0040] According to the difference between the actual result and the predicted result of each training sample, the learning parameters of the local model are adjusted to train the local model.

[0041] In addition, the computer-readable storage medium provided according to the third aspect of the present invention stores computer instructions, which, when executed by a processor, implement the local model training method as described in the second aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or features may have the same or similar reference numerals.

[0043] Figure 1 A process flow chart of an ebullated bed residue oil hydrogenation unit according to some embodiments of the present invention is shown.

[0044] Figure 2 A process flow diagram of a debutanizer unit provided according to some embodiments of the present invention is shown.

[0045] Figure 3 A flowchart of the operation of an efficient JITL framework based on an adaptive multi-branch variable-scale integrated convolutional neural network provided according to some embodiments of the present invention is shown.

[0046] Figure 4 A structural diagram of an adaptive multi-branch variable-scale integrated convolutional neural network (AMVs-ICNN) provided according to some embodiments of the present invention is shown.

[0047] Figure 5A to Figure 5B A schematic diagram showing a comparison of different convolution kernel sizes provided according to some embodiments of the present invention is shown.

[0048] Figure 6 The basic framework of the JITL method provided according to some embodiments of the present invention is shown.

[0049] Figure 7A to Figure 7B A schematic diagram showing the influence of cosine similarity on sample similarity measurement according to some embodiments of the present invention is shown.

[0050] Figure 8 A flowchart of the operation of an efficient JITL framework based on AMVs-ICNN provided according to some embodiments of the present invention is shown.

[0051] Figure 9A to Figure 9B The sample distribution of some variables and the types of operation modes of EBRH provided according to some embodiments of the present invention are shown.

[0052] Fig.10 The sample distribution and operation mode types of some variables in DC provided according to some embodiments of the present invention are shown.

[0053] Figures 11A to 11F The absolute errors of different models provided according to some embodiments of the present invention are shown.

[0054] Figures 12A to 12F The prediction effects of different models provided according to some embodiments of the present invention are shown. DETAILED DESCRIPTION

[0055] The following specific embodiments illustrate the implementation of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will include many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description.

[0056] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] In addition, the terms "upper", "lower", "left", "right", "top", "bottom", "horizontal" and "vertical" used in the following description should be understood as the directions shown in the paragraph and the related drawings. Such relative terms are only used for the convenience of description and do not mean that the device described therein must be manufactured or operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0058] It is understood that although the terms "first", "second", "third", etc. may be used herein to describe various components, regions, layers and / or parts, these components, regions, layers and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers and / or parts. Therefore, the first component, region, layer and / or part discussed below may be referred to as a second component, region, layer and / or part without departing from some embodiments of the present invention.

[0059] As mentioned above, it is difficult to establish a high-precision soft measurement model due to complex processes and dynamic characteristics. Previous data models, such as RNN and LSTM, have the problem of gradient vanishing / explosion and cannot preserve long-term memory. Although the LSTM model can solve the problem of operating condition changes in dynamic industrial processes, it still cannot solve the problem of model degradation caused by changes in operating conditions and feed regulation. In addition, the prediction process of the multi-condition LSTM model relies on past time nodes, and it is impossible to study the impact of a certain operating condition or feed amount on related prediction indicators separately.

[0060] In order to overcome the above-mentioned defects of the prior art, the present invention provides an efficient instant learning framework, a local model training method, and a computer-readable storage medium, which can significantly improve the efficiency of instant learning and the similarity of sample selection, and have strong nonlinear mapping capabilities.

[0061] First, please refer to Figure 1-2 , Figure 1 A process flow chart of an ebullated bed residue oil hydrogenation unit according to some embodiments of the present invention is shown. Figure 2 A process flow diagram of a debutanizer unit provided according to some embodiments of the present invention is shown.

[0062] like Figure 1 As shown in the simplified representation of the ebullated bed residue oil hydrotreating unit, the EBRH unit is a key component of the refining and petrochemical industry process. It can process the residue oil from atmospheric and vacuum distillation units. The residual oil is heated by oil heat exchange in the reactor and mixed with hydrogen in the heating furnace. In the high temperature and catalyst environment of the reactor, impurities such as sulfur, nitrogen, and metals are removed. Then, various oil and gas products are produced through a series of separation systems and fractionation systems. Among them, the more critical finished oil products are naphtha, diesel, and paraffin oil.

[0063] like Figure 2 In the simplified representation of the debutanizer unit shown in FIG, the DC unit is a component of the desulfurization and naphtha cracking unit, and its purpose is to reduce the bottom C4 content as much as possible to improve process performance.

[0064] In some non-limiting embodiments, the training method of the local model provided in the second aspect of the present invention can be implemented based on the efficient instant learning framework provided in the first aspect of the present invention. Specifically, the efficient instant learning framework is configured with a memory and a processor. The memory includes but is not limited to the above-mentioned computer-readable storage medium provided in the third aspect of the present invention, on which computer instructions are stored. The processor is connected to the memory and is configured to execute the computer instructions stored on the memory to implement the training method of the local model provided in the second aspect of the present invention.

[0065] Specifically, the efficient instant learning framework includes a similar sample selection module and a learner module.

[0066] The similar sample selection module includes a similarity calculation unit, a sample length calculation unit and a subset division unit. The similarity calculation unit calculates the similarity between the input sample and multiple historical samples in the historical sample set, and sorts each historical sample accordingly. The sample length calculation unit is used to adaptively select historical samples, and determine the optimal training sample length by calculating the fluctuation range and / or average change of the similarity. The subset division unit is used to cluster each historical sample according to the optimal length, and select the subset with the highest similarity between the cluster center and the input sample as the training sample set of the local model.

[0067] The learner module includes an input layer, a multi-branch variable-scale convolution unit, a flattening layer, a fully connected layer and a weighted summation layer.

[0068] The multi-branch variable-scale convolution unit uses convolution kernels of different sizes to capture and fuse the multi-scale features of each training sample in the training sample set in different branches of the neural network structure of the local model. Specifically, the multi-branch variable-scale convolution unit uses a first convolution kernel of a larger size to capture the first feature of each training sample in the training sample set from the first branch of the neural network structure of the local model, wherein the first feature records more global information and less local information. A second convolution kernel of a smaller size is used to capture the second feature of each training sample in the training sample set from the second branch of the neural network structure of the local model, wherein the second feature records less global information and more local information. The first feature and the second feature are fused to output the multi-scale features of each training sample in the training sample set.

[0069] The flattening layer is used to flatten the output features of each branch. The fully connected layer is used to perform nonlinear mapping on the flattened features output by the flattening layer to obtain multiple intermediate results. The weighted summation layer is used to perform weighted summation on the intermediate results to obtain the prediction results corresponding to each training sample. The learner module adjusts the learning parameters of the local model according to the difference between the actual results and the predicted results of each training sample to train the local model.

[0070] The following will describe the working principle of the above-mentioned efficient instant learning framework in conjunction with some embodiments of the training method of the local model. Those skilled in the art will understand that the embodiments of these local model training methods are only some non-restrictive implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit all functions or all working methods of the efficient instant learning framework. Similarly, the training method of the local model is also only some restrictive implementation methods provided by the present invention, and does not constitute a limitation on the execution subject or execution order of each step in the training method of these local models.

[0071] Specifically, the training method of the local model can obtain query samples about the local model to be trained, and then input the query samples into an efficient instant learning framework such as any one of the first aspects of the present invention, so as to sort the similarity of multiple historical samples in the global historical sample set, calculate the optimal training sample length, and cluster them through its similar sample selection module, and select the subset with the highest similarity between the cluster center and the query sample as the training sample set of the local model. After that, each training sample in the training sample set is sequentially input into the AMVs-ICNN learner module of the efficient instant learning framework to determine its corresponding prediction results respectively. Finally, according to the difference between the actual results and the predicted results of each training sample, the learning parameters of the local model are adjusted to train the local model.

[0072] Please refer to Figure 3 , Figure 3 A flowchart of the operation of an efficient JITL framework based on an adaptive multi-branch variable-scale integrated convolutional neural network provided according to some embodiments of the present invention is shown.

[0073] like Figure 3 As shown, the learning framework can implement the training method of the local model by performing the following steps.

[0074] S1: Using the one-dimensional convolution (1D-Conv) method, a variable-scale convolution module is designed, and different variable-scale convolution modules are used in different branches to fuse local information and global information at different levels.

[0075] The calculation of 1D-Conv convolution is as follows:

[0076] z=f(w*x+b) (1)

[0077] Among them, z is the result of the convolution calculation; f is the activation function; w is the convolution layer weight; b is the convolution layer bias; * is the convolution operation.

[0078] S2: Define the number of neural network branches, calculate the maximum convolution kernel size of the variable convolution process, and calculate the convolution kernel size attenuation factor.

[0079] Please refer to Figure 4 , Figure 4 A structural diagram of an adaptive multi-branch variable-scale integrated convolutional neural network (AMVs-ICNN) provided according to some embodiments of the present invention is shown.

[0080] like Figure 4 As shown in the figure, in order to better balance and fuse local and global feature information, a variable scale convolution module (AMVs-ICNN) is designed. Different variable scale convolution modules are used in different branches to fuse local information and global information at different levels.

[0081] First, the number of neural network branches is defined by formula (2).

[0082]

[0083] Where η is the number of branches and v is the dimension of the input variable of the input sample.

[0084] The number of branches can flexibly adapt to data of different sizes, balance computing resources and model complexity, and improve the generalization ability of the model. By calculating the number of branches through v, the number of branches can be dynamically adjusted according to the size of the input data or the number of features. This method ensures the flexibility and adaptability of the model. By grouping, v can be divided into different intervals according to 5 units, and the number of branches can be appropriately allocated. The addition of 1 ensures that there is at least one branch even when v is small. This design avoids the situation where the model cannot fully learn features due to too few branches, and avoids the situation where the number of branches is too large, resulting in excessive calculation.

[0085] Then, the maximum convolution kernel size of the variable convolution process is calculated by formula (3):

[0086]

[0087] Among them, % represents the modulo operation, which is used to find the remainder after dividing two numbers.

[0088] The formula for calculating the convolution kernel size attenuation factor is:

[0089]

[0090] To ensure that the convolution kernel size of each branch decays exponentially.

[0091] Convolution kernels of different sizes can be obtained:

[0092] κ l =κ max -(η-l)×τ (5)

[0093] Feature extraction should limit the complexity of the network structure while balancing local information and global information. We set the kernel size type to be equal to the number of branches. Since l∈[1,η], the convolution process can be redescribed as:

[0094] z (l,m) =ReLU(w (l,m) *x in +b (l,m) ) (6)

[0095] ReLU is the activation function; (l,m) is the calculation result of the size of the mth convolution kernel of the lth branch; w (l,m) and b (l,m) -The weights and biases of the mth convolution kernel size of the lth branch. To prevent overfitting, the convolution layer of each branch in the network adopts a three-layer structure. In addition, the number of branches is equal to the type of kernel size used. The kernel size of the three convolution layers of each branch can be expressed as S l =[s l-ρ ,s l+1-ρ ,s l+2-ρ ]. When l-ρ=0, l+1-ρ=0, l+2-ρ=0, ρ=0, otherwise ρ=η (ρ of each convolutional layer is calculated independently). The weights and biases of the convolutional layer of each branch can be expressed as W l =[w l-ρ ,w l+1-ρ ,w l+2-ρ ] and B l =[b l-ρ ,b l+1-ρ ,b l+2-ρ ]. The calculation result of the lth branch variable scale convolution module is shown in formula (7).

[0096] z l =ReLU(w l+2-ρ *ReLU(w l+1-ρ *ReLU(w l-ρ *x+b l-ρ )+b l+1-ρ )+b l+2-ρ ) (7)

[0097] Please refer to Figure 5A to Figure 5B , Figure 5A to Figure 5B A schematic diagram showing a comparison of different convolution kernel sizes provided according to some embodiments of the present invention is shown.

[0098] like Figure 5A to Figure 5B As shown, Figure 5A The convolution kernel in is larger, so X3 has more global features, while Figure 5B The X3 in has more local features due to the smaller size of the convolution kernel.

[0099] S3: Use one FC layer to perform nonlinear mapping on the extracted features.

[0100] First, the framework can be flattened. The specific steps are: (1) Flatten the feature map output by the convolutional layer, that is, convert it into a one-dimensional vector. (2) Use the flattened one-dimensional vector as the input of the fully connected layer for subsequent processing and classification. Here, the purpose of flattening is to convert the feature representation extracted by the convolutional layer into a form that can be accepted by the fully connected layer for tasks such as classification and regression.

[0101] After flattening, a FC layer is used to perform nonlinear mapping on the extracted features. The purpose of nonlinear mapping in neural networks is to increase the expressive power of the network so that it can learn and represent more complex data patterns and relationships. The calculation is shown in formula (8):

[0102] y l ′=ReLU(w FC × l +b FC ) (8)

[0103] Among them, w FC is the weight of the fully connected layer; b FC is the bias of the fully connected layer.

[0104] S4: Based on S1, S2, and S3, a variable-scale convolution module AMVs-ICNN is constructed to extract different feature information in different branches. Each branch integrates global and local feature information at different levels. First, a variable-scale convolution module is designed to capture multi-scale features in different branches using convolution kernels of different sizes; then the number of branches is defined, the maximum convolution kernel size and the attenuation factor are calculated, and the convolution kernel size of each branch decays exponentially. Next, the features output by each branch are flattened and nonlinearly mapped through a fully connected layer. Finally, the outputs of each branch are combined and weighted summed to obtain the final result. The weight parameters can be learned through training. The results of each branch are weighted according to formula (9):

[0105]

[0106] Among them, ξ is the weighting coefficient. This parameter is part of the neural network and is updated during the training of the model, just like the neural network in other layers.

[0107] Here, the entire model, including the input layer, multi-branch variable-scale convolution module, flattening layer, fully connected layer and weighted summation layer, is trained and optimized through appropriate loss functions and optimization algorithms.

[0108] Please refer to Figure 6 , Figure 6 The basic framework of the JITL method provided according to some embodiments of the present invention is shown.

[0109] like Figure 6 As shown in the figure, the AMVs-ICNN model was proposed as a local model of JITL in the previous article. JITL is a method of selecting local samples to establish a local model, which can better grasp the complex dynamics of industrial production processes. Compared with soft measurements based on global models, soft measurements based on JITL need to select relevant samples and establish local models through similarity measurement. Since the local features of JITL can more accurately reflect the detailed dynamics of the process, it is necessary to calculate the similarity between the query sample and the historical sample to estimate the relevant variables.

[0110] S5: The characteristic factor of ED (Euclidean distance) is integrated to establish the characteristic scale relationship of similarity measurement.

[0111] Specifically, in the similarity measurement, ED can intuitively reflect the actual distance between two points in the multidimensional space. The calculation formula is as follows:

[0112]

[0113] Among them, x q is the query sample; x i is the historical sample; L is the number of variables.

[0114] On the characteristic scale, variables with high influence are easily disturbed by variables with low influence, making it difficult for ED to accurately describe the similarity between samples. For example, suppose there are two input samples, one of whose variables has a great influence on the model output, and the variable values ​​are significantly different. When the values ​​of other variables are very similar, the ED between them should be small (the similarity should be very high). This is because only one variable has a significantly different value. But in fact, this significantly different value has a great influence on the model output, and the values ​​of the target variables corresponding to the two input samples should also be significantly different. Therefore, the characteristic factor of ED is added in the present invention to establish the characteristic scale relationship of the similarity measurement. The characteristic factor W F,k ={w1,w2,......,w k}Feature importance is calculated by RFs. Input features that have a greater impact on the target variable have a greater weight in the similarity calculation. The calculation formula with characteristic factor ED (ED-FF) is as follows:

[0115]

[0116] Among them, W F,kis the weight of the kth feature. Considering the existence of certain noise in the data and the complex relationship between samples in multidimensional space, ED and ED-FF balance their roles in similarity measurement through weight σ. The distance similarity can be expressed as formula (12).

[0117]

[0118] Among them, D(x q ,x i ) represents the input sample x q With historical sample x i In the distance similarity of the multi-dimensional input space, σ is the second weight to balance the relationship between noise and valid data.

[0119] Please refer to Figure 7A to Figure 7B , Figure 7A to Figure 7B A schematic diagram showing the influence of cosine similarity on sample similarity measurement according to some embodiments of the present invention is shown.

[0120] like Figure 7A to Figure 7B As shown, assuming that in two-dimensional space, x is the query sample, x a and x b are two historical samples respectively. d1=|xx a | and d2=|xx b |. When d1=d2, the distance similarity of the samples is equal, but in terms of angle similarity, θ1<θ2. When the distance similarity is equal, the relationship of angle similarity in three-dimensional space is more complicated than in two-dimensional space. Similarly, in multidimensional space, there is a more complicated angle similarity relationship between samples. Cosine similarity is expressed by formula (13).

[0121]

[0122] cos(θ qi ) is larger, the higher the similarity is. Since different industrial data are affected differently by feature scales and noise, it is necessary to further combine multiple similarity metrics by weighting:

[0123] Similarity(x q ,x i )=λD(x q ,x i )+(1-λ)cos(θ qi ) (14)

[0124] Among them, Similarity(x q ,x i ) represents the input sample x q With historical sample x iThe composite similarity in the multidimensional input space, cos(θ qi ) represents the input sample x q The cosine similarity with the historical sample xi in the multi-dimensional input space, λ is the third weight that balances the distance similarity and cosine similarity.

[0125] S6. Adaptively select training samples according to the similarity of each query sample using the ASLS (Adaptive Sample Length Selection) method. The training sample length of each query sample is determined by determining the fluctuation range of the training sample length and the average change of the fluctuation range. Here, the training sample length refers to the number of samples or the length of the sample sequence selected when training the machine learning model.

[0126] The ASLS method mainly includes the following steps:

[0127] 1) Analyze the impact of sample length on model accuracy and training time through forward recursion. Determine the sample length constraint range U s (upper limit of sample length) and L s (Lower limit of sample length).

[0128] 2) Detect the similarity (x) between the query sample and the historical data sample through formula (14) q ,x i ), and sort the historical data in descending order of similarity.

[0129] 3) Calculate the similarity threshold Γ according to the constraint range of sample length, as shown in formula (15).

[0130]

[0131] The similarity index Γ indicates the fluctuation range and / or average change of the similarity, U s is the upper limit of the sample length of the sample length interval, L s is the lower limit of the sample length of the sample length interval.

[0132] In similar situations, from the lower limit of sample length to the upper limit of sample length, Γ can reflect the mutation of similarity within this range. Since the similarity state of each query sample is dynamic, Γ is also dynamic. Using a hard threshold Γ can better filter out samples with poor similarity.

[0133] S7. Use K-means algorithm to perform data binning and divide the entire data set into multiple sub-data sets.

[0134] There are different feature scales and time scales according to the needs, functions, and characteristics of industrial process data. When considering a 5-minute time scale, one year of process data can generate 105,120 samples. It takes a lot of time to traverse these samples to obtain the training sample with the highest similarity to the query sample. Therefore, data binning can greatly reduce the traversal process and improve screening efficiency. Commonly used data classification methods include equal depth classification, equal width classification, and chi-square test.

[0135] K-means algorithm is an effective clustering method, which uses similar ED similarity measurement technology to group similar samples together and divide the entire data set into multiple sub-datasets. Given input data X = {x1, x2, ..., x m}, C t ={c1,c2,......,c k} is the clustering result divided from the input data X. Where m is the number of samples and k is the number of clusters. The method of initializing the cluster center is as follows: a sample is randomly selected from the data set X as the initial cluster center μ1 of the first cluster. In order to form the corresponding cluster, the similarity between each sample and the cluster center is measured by formula (16). The principle of selecting the next cluster center is to probabilistically select a point with a larger distance until all cluster centers Φ t ={μ1,μ2,......,μ k} are all selected.

[0136]

[0137] Among them, C i is the clustering result divided from the historical sample set X, and k is the preset number of subsets.

[0138] Then, select the historical sample with the greatest similarity to the initial cluster center μ1 as the initial cluster center of the next cluster, and so on, until the initial cluster centers Φ of all clusters are selected. t ={μ1,μ2,......,μ k}, here, the cluster center is updated by formula (17).

[0139]

[0140] Among them, μ j is the updated cluster center, c j is the number of sample points in the new cluster, x is the sample point in the new cluster, and this formula indicates that the new cluster center is the mean of all sample points in the cluster.

[0141] The next set of cluster centers is Φ t+1 ={μ1,μ2,......,μk}. When the iteration is completed or with Φ t+1 and Φ t When the change between is less than ε, the output C t+1 ={c1,c2,......,c k}.

[0142] S8: Construct a similar sample selection strategy based on S5, S6, and S7.

[0143] Specifically, the similarity measure between the query sample and the potential training samples is first calculated using the Euclidean distance, and the samples are sorted according to the similarity. Then, the ASLS method is used to adaptively select training samples, and the optimal training sample length for each query sample is determined by calculating the fluctuation range and the average change. Next, the K-means algorithm is applied to divide the entire dataset into multiple sub-datasets, and the nearest cluster center of each query sample is selected to screen candidate training samples from the corresponding sub-datasets.

[0144] S9: An efficient JITL framework based on AMVs-ICNN is proposed.

[0145] JITL is a method for selecting local samples to build local models. First, the similar sample selection strategy proposed by S8 is used to select data buckets that are highly similar to the query sample, which can significantly reduce the time consumption. Then, the learner proposed by S4 is used to learn and predict the selected samples.

[0146] Here, the input data for prediction include feed variables and operating variables, and the output is product yield data of the petrochemical unit. The training samples are obtained by hybrid similarity measurement and adaptive sample length method, which involves adaptive extraction of training samples of corresponding length. Then the number of branches of the neural network and the size of the convolution kernel are determined, which involves multi-scale information extraction and nonlinear mapping.

[0147] Please refer to Figure 8 , Figure 8 A flowchart of the operation of an efficient JITL framework based on AMVs-ICNN provided according to some embodiments of the present invention is shown.

[0148] like Figure 8 As shown, in EJITL, the input data is normalized to [0, 1] using formula (18) to eliminate the different physical meanings and dimensions of the variables.

[0149]

[0150] In addition, based on the above-mentioned efficient real-time learning framework and the training method of the local model, technical personnel in this field verified the accuracy and feasibility of the model through two chemical processes.

[0151] a.Ebullated Bed Residue Hydrotreating Unit (EBRH)

[0152] Please continue to refer to Figure 1 , the EBRH unit is a key component of the refining and petrochemical industrial process. It can process residual oil from atmospheric and vacuum distillation units. The residual oil is heated by oil heat exchange in the reactor and mixed with hydrogen in the heating furnace. In the high temperature and catalyst environment of the reactor, impurities such as sulfur, nitrogen, and metals are removed. Then, various oil and natural gas products are produced through a series of separation systems and fractionation systems. Among them, the more critical finished oils are naphtha, diesel, and paraffin oil. The reaction raw materials involved in the production process include catalytic cycle oil, vacuum residue oil, recycled oil, hydrogen, and three kinds of normal initial top gas. The output of each oil in the output variable needs to be calculated by the feed amount. For raw gas, the feed amount needs to be converted into tons.

[0153] Furthermore, the output of each oil product is calculated by formula (19):

[0154]

[0155] Among them, C k is the output of the kth oil product; v is the total number of feed variables; F i is the feed amount of the ith feed variable.

[0156] Please refer to Figure 9A to Figure 9B And Table 1, Figure 9A to Figure 9B The sample distribution and operation mode types of some variables of EBRH provided according to some embodiments of the present invention are shown. Table 1 shows the input-output process variables used for modeling in EBRH.

[0157] Table 1 Input-output process variables used for modeling in EBRH

[0158]

[0159] As shown in Table 1, in terms of operating variables, the EBRH unit data was preliminarily screened based on expert experience. Since the EBRH unit is a complex, coupled, nonlinear unit, the Spearman correlation coefficient was used to analyze the correlation between the variables. To avoid high collinearity problems, variables with a Spearman correlation coefficient greater than 0.7 were further processed. Finally, a total of 18 variables were selected.

[0160] like Figure 9A to Figure 9BAs shown, this paper uses the 3σ method and box plot method to process the data, and uses the mean and variance of the normal distribution to remove some data that may be affected by noise and environment. Sample distribution of some variables and output. E1, E3, E5, and E7 are low-yield mode, high-yield mode, medium-yield mode I, and medium-yield mode II, respectively. E2, E4, E6, and E8 are transition state I, transition state II, transition state III, and transition state IV, respectively. Except for variable X12, the other three variables are basically consistent with the overall trend of production. Obviously, a change in the operating mode does not require a change in all operating conditions. A total of 5,000 sets of data were obtained. This study randomly selected 80% as training samples (4,000 samples) and 20% as query samples (1,000 samples).

[0161] b. Debutanizer (DC)

[0162] The DC unit is part of the desulfurization and naphtha cracking unit, and its purpose is to reduce the bottom C4 content as much as possible to improve process performance. Usually, the C4 content is analyzed in the laboratory or measured using a gas chromatograph. However, due to the delay in the concentration measurement of the analyzer, a soft sensor needs to be installed in the bottom flow to predict the C4 content of the process in real time.

[0163] Please refer to Fig.10 And Table 2, Fig.10 The sample distribution and operation mode types of some variables in the DC provided according to some embodiments of the present invention are shown. Table 2 describes the variable types of the DC (including input and output variables).

[0164] The DC dataset contains 2394 samples. 1197 samples are randomly selected from the dataset as training samples, and the remaining 1197 samples are used as query samples. D1 and D2 are the modes with high and low C4 content, respectively. D3 is the transition state of DC.

[0165] Table 2 Input and output process variables used in DC modeling

[0166]

[0167] c. Result analysis

[0168] In this study, the configuration settings of the proposed AMVs-ICNN are as follows: the number of iterations is set to 100, the optimizer is Adam, the loss function is MSE, and the batch size is 32. The number of branches of the AMVs-ICNN for the EBRH device is determined to be 4, and the convolution kernel sizes are 3, 5, 7, and 9, respectively, as shown in Equations (2) to (5). On the other hand, for the data of the DC device, the number of branches of the AMVs-ICNN is 2, and the convolution kernel sizes are 2 and 4.

[0169] Please refer to Figures 11A to 11F , Figures 12A to 12F And Table 3, Figures 11A to 11F The absolute errors of different models provided according to some embodiments of the present invention are shown. Figures 12A to 12F The prediction effects of different models provided according to some embodiments of the present invention are shown. Table 3 shows the prediction performance of different models for various indicators.

[0170] Table 3 Prediction performance of different models for each indicator.

[0171]

[0172] In the model comparison, this study compared several models, among which, there are two types of support vector regression (SVR), SVR1 using "linear" parameters (for linear models), SVR2 using "rbf" parameters (for nonlinear models) and MLP models. In addition, this study also considered three CNN-based control methods: CNN, CNN1 and CNN2. The CNN model does not adopt a multi-branch structure, and each convolutional layer uses the same optimal convolution kernel size. In contrast, CNN1 also adopts a multi-branch structure, but the convolution kernel sizes between different branches are different, while the convolution kernel size used in each branch is the same. At the same time, CNN2 does not adopt a multi-branch structure, but each convolutional layer uses a different size of convolution kernel.

[0173] From the above example, we can see that SVR1, as a linear model, does not have nonlinear mapping capabilities. Therefore, the performance of SVR1 is the worst. From the comprehensive effect, AMVs-ICNN performs well in three evaluation functions (RMSE, MAE, R 2 ) performed best.

[0174] like Figures 11A to 11F As shown, we can see that the box plots of AMVs-ICNN on each indicator are smaller, which indicates that the distribution of absolute errors in the samples is more concentrated and there are fewer outliers.

[0175] like Figures 12A to 12F As shown in Figure 2, the superiority of AMVs-ICNN is further demonstrated. In terms of different indicators, the points of AMVs-ICNN are closer to the black solid line, while the points of other methods are farther from the black line. These results together prove that AMVs-ICNN has strong nonlinear mapping capabilities.

[0176] In summary, the present invention proposes an efficient JITL framework based on EJITL-AMVs-ICNN. In the selection of training samples, the influence of features on the target variable is fully considered, and the feature factors are added to the hybrid similarity measurement method. Then MSM-FF is applied to the K-means algorithm for data binning, and the time consumption is significantly reduced by selecting data buckets that are highly similar to the query samples. According to the similarity of samples, the similarity within a limited sample length range is used to implement adaptive sample length selection (ASLS) to prevent the sample similarity from mutating at a certain node, resulting in the selection of training samples with poor similarity. Finally, an AMVs-ICNN is proposed as a local model of the JITL framework, which extracts different global and local features through different variable-scale convolution modules, and has strong nonlinear mapping capabilities.

[0177] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0178] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An efficient instant learning framework, characterized in that: include: a similar sample selection module, comprising a similarity calculation unit, a sample length calculation unit and a subset division unit, wherein the similarity calculation unit calculates the similarity between the input sample and a plurality of historical samples in the historical sample set, and sorts the historical samples accordingly; the sample length calculation unit is used to adaptively select historical samples, and determine the optimal training sample length by calculating the fluctuation range and / or average change of the similarity; the subset division unit is used to cluster the historical samples according to the optimal length, and select the subset whose cluster center has the highest similarity with the input sample as the training sample set of the local model; and A learner module comprises an input layer, a multi-branch variable-scale convolution unit, a flattening layer, a fully connected layer and a weighted summation layer, wherein the multi-branch variable-scale convolution unit uses convolution kernels of different sizes to respectively capture and fuse the multi-scale features of each training sample in the training sample set in different branches of the neural network structure of the local model, the flattening layer is used to flatten the output features of each branch, the fully connected layer is used to perform nonlinear mapping on the flattened features output by the flattening layer to obtain multiple intermediate results, the weighted summation layer is used to perform weighted summation on each of the intermediate results to obtain the prediction results corresponding to each of the training samples, and the learner module adjusts the learning parameters of the local model according to the difference between the actual results and the predicted results of each of the training samples to train the local model.

2. The efficient instant learning framework as claimed in claim 1, characterized in that: The similarity is characterized by the Euclidean distance: Among them, D(x q ,x i ) represents the input sample x q With historical sample x i Distance similarity in multidimensional input space, W F,k is the weight of the kth feature, σ is the second weight that balances the relationship between noise and valid data, and L is the number of variables in the input sample.

3. The efficient instant learning framework as claimed in claim 2, characterized in that: The similarity is characterized by cosine similarity: Similarity(x q ,x i )=λD(x q ,x i )+(1-λ)cos(θ qi ) Among them, Similarity(x q ,x i ) represents the input sample x q With historical sample x i The composite similarity in the multidimensional input space, cos(θ qi ) represents the input sample x q With historical sample x i In the cosine similarity of the multi-dimensional input space, λ is a third weight for balancing the distance similarity and the cosine similarity.

4. The efficient instant learning framework according to claim 1, characterized in that: The sample length calculation unit is configured as follows: By forward recursive analysis of the influence of sample length on the prediction accuracy of the local model, a sample length interval in which the prediction accuracy reaches a preset accuracy threshold is determined; and According to the arrangement order of each of the historical samples, the similarity index Γ between the input sample and the corresponding number of historical samples is calculated gradually and incrementally from the lower limit of the sample length interval to adaptively determine the optimal training sample length, wherein the similarity index Γ is expressed as: The similarity index Γ indicates the fluctuation range and / or average change of the similarity, U s is the upper limit of the sample length of the sample length interval, L s is the lower limit of the sample length of the sample length interval.

5. The efficient instant learning framework as claimed in claim 1, characterized in that: The subset division unit is configured to: A historical sample is randomly selected from the historical sample set X as the initial cluster center μ1 of the first cluster, and the similarity E between each historical sample in the historical sample set X and the initial cluster center μ1 is measured: Among them, C i is the clustering result divided from the historical sample set X, and k is the preset number of subsets; Select the historical sample with the greatest similarity to the initial cluster center μ1 as the initial cluster center of the next cluster, and so on, until the initial cluster centers Φ of all clusters are selected. t ={μ1,μ2,......,μ k }; The coordinates of all historical samples belonging to the same cluster are averaged to iteratively update the cluster centers of each cluster: Among them, μ j is the updated cluster center, c j is the number of sample points in the new cluster, x is the sample point in the new cluster, this formula indicates that the new cluster center is the mean of all sample points in the cluster; and in response to reaching the preset number of iterative updates, or the next round of cluster center set Φ t+1 and the cluster center set Φ of the current round t The difference between them is less than the preset threshold ε, and the clustering result C of the next round is output. t+1 ={c1,c2,......,c k }, as the training sample set of the local model.

6. The efficient instant learning framework according to claim 1, characterized in that: The multi-branch variable-scale convolution unit is configured as follows: Using a first convolution kernel of a larger size, capturing a first feature of each training sample in the training sample set from a first branch of the neural network structure of the local model, wherein the first feature records more global information and less local information; Using a second convolution kernel of a smaller size, capturing a second feature of each training sample in the training sample set from a second branch of the neural network structure of the local model, wherein the second feature records less global information and more local information; and The first feature and the second feature are fused to output a multi-scale feature of each training sample in the training sample set.

7. The efficient instant learning framework according to claim 1, characterized in that: The number of branches of the neural network structure of the local model is defined as: Where n is the number of branches, v is the dimension of the input variable of the input sample, The maximum convolution kernel size κ of the multi-branch variable-scale convolution unit max Identified as: Among them, % represents the modulo operation, which is used to find the remainder after dividing two numbers. The size attenuation factor τ between the convolution kernels of each branch is determined as: To ensure that the convolution kernel size of each branch decays exponentially.

8. The efficient instant learning framework as claimed in claim 1, characterized in that: The planarization layer is configured as follows: Flattening the feature maps of multiple branches output by the multi-branch variable-scale convolution unit to convert them into corresponding one-dimensional feature vectors respectively; and The flattened one-dimensional feature vector is input and output from the corresponding fully connected layer.

9. The efficient instant learning framework as claimed in claim 8, characterized in that: The fully connected layer is configured as: The one-dimensional feature vector z output by the flattening layer l Perform nonlinear mapping to determine the intermediate result y of the corresponding branch l ′: y′ l =ReLU(in FC ×z l +b FC ) Among them, w FC is the weight of the fully connected layer, b FC is the bias of the fully connected layer; and The intermediate result y l ’ Input the weighted summation layer.

10. The efficient instant learning framework according to claim 9, characterized in that: The weighted sum layer is configured as: The intermediate results y of each branch of the neural network structure of the local model l ′ is weighted summed to determine the prediction result y of the corresponding training sample: Wherein, ξ is the weight coefficient of each branch, which is updated as the local model is trained.

11. A local model training method, characterized in that: The following steps are involved: Obtain query samples about the local model to be trained; Inputting the query sample into the efficient instant learning framework as described in any one of claims 1 to 10, so as to sort the similarity, calculate the optimal training sample length, and cluster the multiple historical samples in the global historical sample set through its similar sample selection module, and select the subset with the highest similarity between the cluster center and the query sample as the training sample set of the local model; Inputting each training sample in the training sample set into the AMVs-ICNN learner module of the efficient real-time learning framework in sequence to determine the corresponding prediction results respectively; as well as According to the difference between the actual result and the predicted result of each training sample, the learning parameters of the local model are adjusted to train the local model.

12. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the local model training method as claimed in claim 11 is implemented.

Citation Information

Patent Citations

  • Spectral clustering analysis-based just-in-time learning modeling method

    CN107403196A