Fast Optimization Method for Industrial Big Data Model Based on Incremental Learning of Siamese Network

Abstract

The present invention discloses a method for rapidly optimizing an industrial big data model based on incremental learning of a siamese network, belonging to the fields of deep learning and industrial intelligent manufacturing. Aiming at the problem that the current industrial data model based on deep learning has a single usage scenario and cannot timely utilize the increment data generated in real time for self-optimization and iterative update, the present invention proposes a model optimization method based on a feedforward neural network FNN, a bidirectional long short-term memory network Bi-LSTM, a siamese network, and an attention model Attention. Compared with the existing industrial data models, this method improves the generalization ability of the industrial data model and can timely update and optimize the model by using the increment data generated in real time in industrial manufacturing.

Description

Technical Field

[0001] The present invention belongs to the field, and particularly relates to a method for rapidly optimizing an industrial big data model based on incremental learning of a twin network. Background Art

[0002] With the rapid development of artificial intelligence and Internet of Things technologies, information technologies such as industrial sensors, industrial Internet, automatic control systems, ERP, etc. have been widely applied in the industrial field. A large amount of data related to industrial production activities is collected in real time and stored in the enterprise's information system. Especially when the production line in a manufacturing enterprise is running at high speed, a large amount of data is generated on industrial equipment, and at the same time, a large amount of data is generated by people and computers in the enterprise. Analyzing and utilizing this data helps to improve production efficiency, improve production processes, reduce production costs, and lay a good foundation for intelligent manufacturing.

[0003] Traditional big data models and algorithms first analyze and explore historical data, and then design efficient and accurate algorithms for specific analyzed scenarios to detect anomalies in production manufacturing or to predict future probabilities and trends. Industrial big data not only has the 5V (volume, velocity, variety, value, veracity) characteristics of big data, but also has many characteristics closely related to industrial production characteristics, such as diverse data sources, low data quality, complex information contained in data, coupled uncertainty, high data real-time performance, etc. This makes the models for industrial big data analysis more complex and diverse, and has higher requirements for the real-time performance and applicability of algorithms.

[0004] Deep learning methods have good performance in fields such as prediction, identification and detection. Especially in the case of having a large amount of data support, by selecting a suitable deep learning model, using a large amount of historical data as input, and through multiple trainings, an end-to-end model under a specific scenario can be obtained. Inputting new data in the form of the data during training into the trained model can obtain the desired result. However, the existing industrial data models have the following problems:

[0005] The applicable scenarios are single. For factors such as different machine models, raw material formulas, and manufacturing processes in a factory, it requires a high computational cost to train different models to be used in actual production;

[0006] With the improvement of production process technologies and the change of newly added data, the model should also be optimized accordingly to adapt to new industrial manufacturing. However, traditional methods cannot update the model using the increment data generated in real time. Summary of the Invention

[0007] In view of the above technical problems existing in the prior art, the present invention proposes a rapid optimization method for an industrial big data model based on twin network incremental learning, with reasonable design, overcoming the deficiencies of the prior art and having good effects.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A rapid optimization method for an industrial big data model based on twin network incremental learning includes the following steps:

[0010] Step 1: Establish a traditional industrial big data model based on deep learning;

[0011] Step 2: Establish an industrial big data model adaptable to multiple scenarios;

[0012] On the basis of the traditional industrial big data model based on deep learning established in Step 1, a feedforward neural network FNN and a bidirectional long short-term memory network Bi-LSTM are added to establish an industrial data model adaptable to multiple scenarios;

[0013] Step 3: Incremental update mechanism based on twin network;

[0014] The twin network is divided into a historical data network and an incremental data network. Among them, the historical data network is trained with a large amount of historical data. The incremental data network is secondarily trained with the incrementally collected data on the basis of the trained historical data network, and the results of the two networks are weighted and summed through an attention model Attention to obtain the final result; the specific steps are as follows:

[0015] Step 3.1: Save the industrial data model adaptable to multiple scenarios in Step 2 after training as the historical data network;

[0016] Step 3.2: Copy the historical data network and save it as the incremental data network, including the trained internal parameters;

[0017] Step 3.3: Input the process indicators in the newly collected incremental data into the incremental data network for training;

[0018] Step 3.4: Input the results of the historical data network and the incremental data network into the attention model, and the attention model outputs the final predicted value. Calculate the loss value, that is, the loss value, through the final predicted value and the true value;

[0019] Step 3.5: Make the loss value converge after a small number of iterative trainings. Among them, the attention model only updates the internal parameters during the first training of the incremental data network, otherwise only the incremental data network is trained;

[0020] Step 3.6: Put it into production line use. When new incremental data is collected, repeat Steps 3.3 - 3.5 to quickly update and optimize the model.

[0021] Preferably, in Step 1, a large amount of historical data is pre - processed and then input into a built - in algorithm network (Deep Learning Network). According to the output result of the model and the prediction object, a loss function is defined. After multiple iterative trainings, the internal parameters of the model are adjusted and optimized to make the loss value reach the minimum, that is, the input result of the model is closest to the real data.

[0022] Preferably, in Step 2, it specifically includes the following steps:

[0023] Step 2.1: Perform one - hot encoding on k process features respectively to obtain a process feature vector C;

[0024] Step 2.2: Input the process feature vector C into a feed - forward neural network FNN;

[0025] Step 2.3: Input historical process indicators into the constructed deep - learning network;

[0026] Step 2.4: Horizontally splice (Filter concat) the output result of the feed - forward neural network FNN and the result of the deep - learning network and input them into a bidirectional long - short - term memory network Bi - LSTM to obtain the final prediction result, and calculate the loss value through the final prediction result and the real value;

[0027] Step 2.5: Perform multiple iterative trainings until the loss value converges.

[0028] The beneficial technical effects brought by the present invention:

[0029] Aiming at the problem that the current industrial data model based on deep learning has a single usage scenario and cannot timely utilize the increment data generated in real - time for self - optimization and iterative update, the present invention proposes a model optimization method based on a feed - forward neural network FNN, a bidirectional long - short - term memory network Bi - LSTM, a siamese network, and an attention model Attention. Compared with the existing industrial data models, this method improves the generalization ability of the industrial data model and can timely update and optimize the model by using the increment data generated in real - time during industrial manufacturing.

[0030] The present invention uses a feed - forward neural network FNN and a bidirectional long - short - term memory network Bi - LSTM to form a scenario adaptation mechanism, and inputs each scenario factor into the feed - forward neural network FNN after one - hot encoding to adjust the model's adaptation ability to different scenarios.

[0031] The present invention utilizes a siamese network and divides it into a historical data network and an incremental data network. The historical data network is trained using a large amount of historical data. The incremental data network is trained using real-time generated data on the basis of the completion of the training of the historical data network, and an attention model is used to fuse the results of the two networks to avoid the phenomenon of model overfitting caused by training with incremental data. The incremental data for training the incremental data network has the characteristics of new data and small volume, so the model can complete self-update relatively quickly and adapt to the new production process. Description of the Drawings

[0032] Figure 1 It is a traditional industrial big data model diagram based on deep learning;

[0033] Figure 2 It is an industrial big data model diagram adapted to multiple scenarios;

[0034] Figure 3 It is a flow chart of an industrial data model optimization method based on incremental learning of a siamese network. Detailed Implementation Manner

[0035] The present invention will be further described in detail below in conjunction with the drawings and the specific implementation manner:

[0036] The rapid optimization method for an industrial big data model based on incremental learning of a siamese network is as follows:

[0037] Step 1: Establish a traditional industrial big data model based on deep learning;

[0038] Taking the prediction of the temperature in the factory's mixing rubber compound process as an example, the system collects data on four state indicators, namely the rubber compound temperature, rotor speed, motor power, and chassis pressure, during the mixing process every second. Among them, the state indicators during each mixing process vary according to the rubber compound formula, machine model, and mixing process.

[0039] The deep learning model inputs a large amount of historical data after preprocessing into the established algorithm network, defines a loss function according to the output result of the model and the prediction object, and through multiple iterative trainings, adjusts and optimizes the internal parameters of the model to make the loss value reach the minimum, that is, the model input result is closest to the real data. In the present invention, the rubber compound formula, machine model, and process type are called process characteristics, and the four types of process indicators, namely the rubber compound temperature, rotor speed, motor power, and chassis pressure, are divided into historical data and incremental data respectively according to the collection time. Among them, the rubber compound temperature is also the prediction object Y.

[0040] The traditional industrial big data model based on deep learning is as Figure 1 shown. The indicators from time m to time m + n are input into the model, and finally the prediction result Y at time m + n + 1 is obtained ′Among them, X is the index feature, m is the moment, n is the time span of the index data input, and l is the time span between the prediction object and the input data.

[0041] Step 2: Establish an industrial big data model suitable for multiple scenarios;

[0042] Based on the traditional industrial big data model based on deep learning established in Step 1, this invention adds a feedforward neural network FNN and a bidirectional long short-term memory network Bi-LSTM to form an industrial data model suitable for multiple scenarios as shown in Figure 2 . Among them, C represents the process feature vector after one-hot encoding, k represents the number of process features, and the specific steps are as follows:

[0043] Step 2.1: First, perform one-hot encoding on k process features respectively to obtain the process feature vector C.

[0044] Step 2.2: Input the process feature vector C into the FNN.

[0045] Step 2.3: Input the historical process indicators into the constructed deep learning network.

[0046] Step 2.4: Horizontally splice (Filter concat) the output results of the feedforward neural network FNN and the results of the deep learning network, and input them into the bidirectional long short-term memory network Bi-LSTM to obtain the final prediction result. Calculate the loss value through the final prediction result and the true value;

[0047] Step 2.5: Iteratively train multiple times until the loss value converges.

[0048] Step 3: Incremental update mechanism based on twin networks;

[0049] Aiming at the problem that the traditional industrial data model cannot timely utilize the increment data generated in real time for self-optimization and iterative update, this invention proposes an incremental update mechanism based on twin networks, as shown in Figure 3 . The twin network is divided into a historical data network and an incremental data network. The historical data network is trained with a large amount of historical data. The incremental data network is based on the trained historical data network and is secondarily trained with the increment data collected in real time. The results of the two networks are weighted and summed through an attention model to obtain the final result, where {X m , …, X m+n} represents the data collected historically, and {X z , …, X z+n , …, X z+n+l} represents the recently newly collected increment data.

[0050] This model can effectively utilize incremental data to update the model and improve its accuracy. The specific steps are as follows:

[0051] Step 3.1: Save the industrial data model adapted to multiple scenarios in Step 2 after training as a historical data network;

[0052] Step 3.2: Copy the historical data network and save it as an incremental data network, including the trained internal parameters;

[0053] Step 3.3: Input the process indicators in the newly collected incremental data into the incremental data network for training;

[0054] Step 3.4: Input the results of the historical data network and the incremental data network into the attention model, and the attention model outputs the final predicted value, and calculates the loss value, i.e., the loss value, with the true value;

[0055] Step 3.5: After a small number of iterative trainings, the loss value can converge. Among them, the attention model only updates the internal parameters during the first training of the incremental data network, otherwise only the incremental data network is trained;

[0056] Step 3.6: Put it into production line for use. When new incremental data is collected, repeat Steps 3.3 - 3.5 to quickly update and optimize the model.

[0057] The pre - protected technical points are:

[0058] 1. The present invention believes that factors such as equipment models, raw material formulas, and manufacturing processes in the factory will generate different state information during the manufacturing process, so the changes in various indicators during the manufacturing process will be different. Based on this view, the present invention uses the FNN and Bi - LSTM models to form a scenario adaptation mechanism, and inputs the various scenario factors into the FNN after one - hot encoding to adjust the model's adaptation ability to different scenarios.

[0059] 2. The present invention uses a siamese network and divides it into a historical data network and an incremental data network. The historical data network is trained with a large amount of historical data, and the incremental data network is trained with real - time generated data on the basis of the completion of the historical data network training. The attention model is used to fuse the results of the two networks to avoid the phenomenon of model over - fitting caused by training with incremental data. The incremental data for training the incremental data network has the characteristics of new data and small volume, so the model can complete self - update faster and adapt to the new production process.

[0060] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the essence of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for rapidly optimizing an industrial big data model based on incremental learning of a twin network, characterized in that: It includes the following steps: Step 1: Establish a traditional industrial big data model based on deep learning; Step 2: Establish an industrial big data model adaptable to multiple scenarios; On the basis of the traditional industrial big data model based on deep learning established in Step 1, add a feedforward neural network FNN and a bidirectional long short-term memory network Bi-LSTM to establish an industrial data model adaptable to multiple scenarios; Step 3: Incremental update mechanism based on the twin network; The twin network is divided into a historical data network and an incremental data network. Among them, the historical data network is trained with a large amount of historical data. The incremental data network is based on the trained historical data network and is secondarily trained with the incrementally collected real-time data. The results of the two networks are weighted and summed through an attention model Attention to obtain the final result. The specific steps are as follows: Step 3.1: Save the industrial data model adaptable to multiple scenarios in Step 2 after training as the historical data network; Step 3.2: Copy the historical data network and save it as the incremental data network, including the trained internal parameters; Step 3.3: Input the process indicators in the newly collected incremental data into the incremental data network for training; Step 3.4: Input the results of the historical data network and the incremental data network into the attention model, and the attention model outputs the final predicted value. Calculate the loss value by comparing the final predicted value with the true value, namely value; Step 3.5: Through fewer iterations of training, make the value converge, where the attention model only updates its internal parameters during the first training of the incremental data network, otherwise only the incremental data network is trained; Step 3.6: Put it into production line use. When new incremental data is collected, repeat Steps 3.3 - 3.5 to quickly update and optimize the model; In Step 2, it specifically includes the following steps: Step 2.1: Respectively perform one-hot encoding on process features to obtain process feature vectors ; Step 2.2: Input the process feature vector into the feedforward neural network FNN; Step 2.3: Input the historical process indicators into the constructed deep learning network; Step 2.4: Horizontally splice the output result of the feedforward neural network FNN and the result of the deep learning network, and input them into the bidirectional long short-term memory network Bi-LSTM to obtain the final prediction result of the stock temperature, and calculate the value through the final prediction result and the true value value; Step 2.5: Iteratively train multiple times until the value converges; The process characteristics are rubber compound formula, machine model, and process type, and the process indicators are rubber compound temperature, rotor speed, motor power, and chassis pressure.

2. The method for quickly optimizing an industrial big data model based on incremental learning of a twin network according to claim 1, wherein: In step 1, the deep learning model preprocesses a large amount of historical data and inputs it into the established algorithm network. A loss function is defined based on the output result of the model and the prediction object. Through multiple iterative trainings, the internal parameters of the model are adjusted and optimized so that the value reaches the minimum, that is, the model input result is closest to the real data.

Images