A digital modeling method based on neural architecture search

Abstract

A digital modeling method based on neural architecture search provided by the present invention, the modeling method comprising: acquiring real-time operation data and historical data of aerospace equipment; predicting by training a recurrent neural network based on deep transfer learning according to the real-time operation data and the historical data to generate prediction data; performing data augmentation according to the prediction data to obtain augmented data; constructing a search space according to the augmented data by using a modular network structure and assembling; and modeling the search space into a Markov decision process by using a reinforcement learning method to generate a convolutional neural network architecture model. The real-time fault diagnosis capability of complex aerospace equipment systems is realized, and the risk of faults occurring in complex aerospace equipment systems is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular, to a digital modeling method based on neural architecture search. Background Art

[0002] The disadvantages of the prior art are mainly manifested in the following aspects as the model scale expands and the number of hyperparameters increases: the structure encoding method cannot represent the complex network structure search space, the encoding space is too large resulting in the inability of these search algorithms to converge, and the training time of the deep learning model is too long leading to a reduction in the computational efficiency of the black-box optimization method, etc.

[0003] To solve the problem, the architecture search network was first proposed at ICML in 2017, and a reinforcement learning method was used to solve this new problem. The architecture search network is widely used in various fields, and is gradually stronger than manual operation in constructing network models and adjusting network parameters, and has better real-time performance, trainability, and consistency in some landing algorithms.

[0004] To solve the problem that the existing digital modeling technology based on the simulation platform cannot meet the modeling requirements, and to meet the requirements of realizing the real-time fault diagnosis ability of aerospace complex equipment systems and reducing the risk of faults occurring in aerospace complex equipment systems. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a digital modeling method based on neural architecture search that overcomes the above problems or at least partially solves the above problems.

[0006] According to one aspect of the present invention, there is provided a digital modeling method based on neural architecture search, the modeling method comprising:

[0007] Obtain the real-time operation data and historical data of the aerospace equipment;

[0008] Based on the real-time operation data and the historical data, train a recurrent neural network prediction based on deep transfer learning to generate prediction data;

[0009] Perform data augmentation according to the prediction data to obtain augmented data;

[0010] According to the augmented data, use a modular network structure and assemble to construct a search space;

[0011] Adopt a reinforcement learning method to model the search space into a Markov decision process to generate a convolutional neural network architecture model.

[0012] Optionally, the training of the recurrent neural network prediction based on deep transfer learning specifically includes:

[0013] Input the training sample data, where the training sample data is the overall operation status data of aerospace equipment, and the data includes historical data over the years and real-time data;

[0014] Preprocess the historical operation status data samples of aerospace equipment over the years to obtain preprocessed historical aerospace equipment data;

[0015] Construct a deep neural network and select parameters, and input the training sample data;

[0016] Use the iterative backpropagation algorithm to optimize the residuals to obtain the deeply optimized data results;

[0017] Obtain a source domain recurrent neural network prediction model that meets the threshold performance according to the optimized data results;

[0018] Preprocess the operation status data samples of aerospace equipment to obtain preprocessed training sample data;

[0019] Input the preprocessed training sample data into the source domain recurrent neural network prediction model;

[0020] Use the iterative backpropagation algorithm to optimize the residuals to obtain the recurrently optimized data results;

[0021] Obtain a target domain recurrent neural network prediction model that meets the threshold performance according to the recurrently optimized data results.

[0022] Optionally, the data augmentation based on the prediction data to obtain augmented data specifically includes:

[0023] The conditions satisfied by the data augmentation include: the augmented data should be a true reflection of the overall operation status of aerospace equipment, and the distribution of the augmented data is basically similar to the data distribution over the years;

[0024] The amount of the augmented data ensures that it can meet the data requirements of the optimization algorithm, enabling the neural architecture search to search for an effective and locally optimal deep convolutional neural network;

[0025] The amount of the augmented data ensures that it can meet the data requirements of model evaluation, verifying whether the deep convolutional neural network can effectively simulate the real-time operation status of real physical aerospace equipment;

[0026] The data augmentation is a dynamic process that needs to be continuously iterated to continuously provide data support for the digital modeling based on neural architecture search.

[0027] Optionally, the modeling method further includes:

[0028] Use the method of surrogate model to establish a model to evaluate the convolutional neural network.

[0029] A digital modeling method based on neural architecture search provided by the present invention, the modeling method comprising: obtaining real-time operation data and historical data of a space equipment; training a recurrent neural network prediction based on deep transfer learning according to the real-time operation data and the historical data to generate prediction data; performing data augmentation according to the prediction data to obtain augmented data; using a modular network structure according to the augmented data and assembling to construct a search space; adopting a reinforcement learning method to model the search space into a Markov decision process to generate a convolutional neural network architecture model. The real-time fault diagnosis ability of the complex space equipment system is realized, and the risk of faults occurring in the complex space equipment system is reduced.

[0030] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other objects, features and advantages of the present invention more obvious and understandable, the following specifically gives the specific embodiments of the present invention. Brief Description of the Drawings

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0032] Figure 1 It is a flowchart of a digital modeling method based on neural architecture search provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0033] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0034] The terms "including" and "having" and any variations thereof in the description of the embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusion. For example, a series of steps or units are included.

[0035] The technical solutions of the present invention will be further described in detail below with reference to the drawings and embodiments.

[0036] As Figure 1 shown, a digital modeling method based on neural architecture search, the modeling method comprising:

[0037] Obtain the real-time operation data and historical data of aerospace equipment.

[0038] Based on the real-time operation data and historical data, train a recurrent neural network prediction based on deep transfer learning to generate prediction data.

[0039] Perform data augmentation according to the prediction data to obtain augmented data.

[0040] According to the augmented data, use a modular network structure and assemble it to construct a search space.

[0041] Using a modular network structure and assembling it to construct a search space, including common residual modules, attention mechanisms, activation functions, etc. Use a cell-based method to reduce the search space, and consider the structures of cells and combined cells as a hierarchy based on a hierarchical idea. With the idea of model transfer, reduce the number of training times at the beginning of each iteration. After each training starts, only use a small number of samples to train the network, so it is also called one-shot learning, which is somewhat different from transfer learning. Transfer learning generally requires a source domain and a target domain.

[0042] Adopt a reinforcement learning method to model the search space as a Markov decision process to generate a convolutional neural network architecture model.

[0043] Training the recurrent neural network prediction based on deep transfer learning specifically includes:

[0044] Input training sample data, where the training sample data is the overall operation state data of aerospace equipment, and the data includes historical data and real-time data over the years.

[0045] Preprocess the historical operation state data samples of aerospace equipment over the years to obtain preprocessed historical aerospace equipment data.

[0046] Construct a deep neural network and select parameters, and input the training sample data.

[0047] Adopt an iterative backpropagation algorithm to optimize the residuals to obtain a deeply optimized data result.

[0048] According to the optimized data result, obtain a source domain recurrent neural network prediction model that meets the threshold performance.

[0049] Preprocess the operation state data samples of aerospace equipment to obtain preprocessed training sample data. Input the preprocessed training sample data into the source domain recurrent neural network prediction model.

[0050] Adopt an iterative backpropagation algorithm to optimize the residuals to obtain a recurrently optimized data result.

[0051] According to the loop-optimized data result, a target domain recurrent neural network prediction model that meets the threshold performance is obtained.

[0052] Performing data augmentation on the predicted data to obtain augmented data specifically includes:

[0053] The conditions satisfied by the data augmentation include: the augmented data should be a true reflection of the overall operating state of the aerospace equipment, and the distribution of the augmented data is basically similar to the data distribution over the years;

[0054] The amount of the augmented data is guaranteed to meet the data requirements of the optimization algorithm, enabling the neural architecture search to search for an effective and locally optimal deep convolutional neural network;

[0055] The amount of the augmented data is guaranteed to meet the data requirements of the model evaluation, verifying whether the deep convolutional neural network can effectively simulate the real-time operating state of the real physical aerospace equipment;

[0056] The augmentation of the data is a dynamic process that needs to be continuously iteratively augmented to continuously provide data support for the digital modeling based on the neural architecture search.

[0057] Using the reinforcement learning method to model the network architecture search as a Markov decision process, and using the RL method to generate the convolutional neural network architecture. For each layer of the convolutional neural network, the learning agent will select the type of the layer and the corresponding parameters. The evaluation accuracy obtained after training the generated network structure is used as the reward and is used to participate in the Q-learning training. The research uses a recurrent neural network as the controller to sample and generate a string describing the network structure, which will be used for training and obtain the evaluation accuracy rate, and then uses the REINFORCE algorithm to learn the parameters of the controller so that it can generate a network structure with a higher accuracy rate.

[0058] The modeling method further includes:

[0059] Adopting the method of the surrogate model to establish a model to evaluate the convolutional neural network. The model evaluation is to ensure that the model of the neural architecture search can effectively simulate the operation process of the real physical aerospace equipment and output its operating state in real time. The model evaluation is an important guarantee for digital modeling. The model evaluation occupies most of the time consumption of the neural architecture search. Therefore, this system adopts some methods to optimize this part of the process of the model evaluation. Since the evaluation of the original model is time-consuming, the method of the surrogate model is adopted to establish a model to estimate the individual, thereby reducing the evaluation consumption. For the trade-off problem, that is, the more accurate the model is, the more time-consuming it will be. On the contrary, if the model is inaccurate, it cannot accurately estimate the quality of the offspring. A reasonable surrogate model is used to effectively track the trend of the original model, so as to accurately rank the individuals and select good individuals.

[0060] The significance of neural architecture search lies in solving the problem of tuning the parameters of deep learning models. The depth and width of the layers in the network, as well as the hyperparameters for training the network, are cross-research combining optimization and machine learning. Before deep learning, traditional machine learning models also encountered the problem of tuning model parameters. Since the structure of shallow models is relatively simple, most research unified the model structure as hyperparameters for search. For example, the number of hidden neurons in a three-layer neural network. The main methods for optimizing these hyperparameters are black-box optimization methods, such as evolutionary optimization, Bayesian optimization, and reinforcement learning respectively.

[0061] Beneficial effects:

[0062] (1) Automation:

[0063] With the proposal of neural architecture search, the design of neural network structures is transforming from manual design to machine automatic design. During the process of simulating the operation of real physical aerospace equipment, data is fed to the machine to achieve the automation of hyperparameter selection, reducing the impact brought by human operation.

[0064] (2) Simplicity:

[0065] Using data as the driving force, a digital simulation model of a digital twin system is built by using gradient boosting and architecture search network to construct a neural network, solving the limitation of traditional digital modeling technology on professional qualities and the problem of excessive difficulty in digital modeling of complex aerospace equipment, thereby realizing the real-time fault diagnosis ability of aerospace complex equipment systems and reducing the risk of faults occurring in aerospace complex equipment systems.

[0066] The above specific implementation manners have further elaborated the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only the specific implementation manners of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A digital modeling method based on neural architecture search, characterized in that, The described modeling method includes: Obtaining the real-time operation data and historical data of the space equipment; Based on the real-time operation data and the historical data, training a source domain recurrent neural network prediction model and a target domain recurrent neural network prediction model through deep transfer learning to generate prediction data, including: Inputting training sample data, which is the real-time operation data and historical data of the space equipment; Preprocessing the real-time operation data and historical data samples of the space equipment to obtain the preprocessed training sample data; Constructing a deep neural network and selecting parameters, and inputting the preprocessed training sample data; Using the iterative backpropagation algorithm to optimize the residuals to obtain the deep optimization data result; Obtaining a source domain recurrent neural network prediction model that meets the threshold performance according to the optimization data result; Preprocessing the real-time operation data and historical data samples of the space equipment to obtain the preprocessed training sample data; Inputting the preprocessed training sample data into the source domain recurrent neural network prediction model; Using the iterative backpropagation algorithm to optimize the residuals to obtain the recurrent optimization data result; Obtaining a target domain recurrent neural network prediction model that meets the threshold performance according to the recurrent optimization data result; Performing data augmentation according to the prediction data to obtain augmented data; Based on the augmented data, assembling a modular network structure to construct a search space; Using the reinforcement learning method to model the search space into a Markov decision process to generate a convolutional neural network architecture model.

2. The digital modeling method based on neural architecture search according to claim 1, wherein The specific process of performing data augmentation according to the prediction data to obtain augmented data includes: The conditions satisfied by the data augmentation include: the augmented data is the real-time operation data and historical data of the space equipment; The amount of the augmented data ensures that it can meet the data requirements for evaluating the convolutional neural network architecture model, and verifies whether the deep convolutional neural network can simulate the real-time operation state of the real physical space equipment; The data augmentation is a dynamic process and needs to be continuously iteratively augmented.

3. A digital modeling method based on neural architecture search according to claim 1, characterized in that The described modeling method further includes: Using the method of a surrogate model to establish a model to evaluate the convolutional neural network architecture model.

Images