Neural network deployment system and method for airborne electromechanical system

Abstract

The invention provides an airborne electromechanical system-oriented neural network deployment system and method. According to the deployment system and method of the neural network oriented to the airborne electromechanical system, the client and the electronic equipment are combined, compression, performance optimization, parameter quantization and programming language conversion are sequentially carried out on the neural network oriented to the airborne electromechanical system through the client, and the computing power of the electronic equipment is improved, so that the deployment efficiency of the neural network oriented to the airborne electromechanical system is improved. A large neural network can be converted into a smaller and more efficient lightweight network, so that the deployability of the network on an airborne electromechanical system with limited hardware resources is improved, it is ensured that the network can still be effectively operated under limited computing power, and the challenge of neural network deployment in the airborne electromechanical system is solved; the real-time performance and the multi-task processing capacity are remarkably improved, effective support is provided for engineering landing, the neural network application oriented to the airborne electromechanical system can be more efficient, flexible and reliable due to the advantages, and a foundation is laid for future intelligent development.

Description

Technical Field

[0001] The present application relates to the technical field of airborne electromechanical systems, and in particular to a system and method for deploying a neural network for airborne electromechanical systems. Background Art

[0002] At present, neural network algorithms are generally used to perform various tasks in the field of airborne mechatronic systems. When actually deploying neural networks for airborne mechatronic systems, most of the neural networks used are designed based on typical neural networks. This neural network has a large amount of calculation and high requirements for hardware resources, which seriously limits its application in the field of airborne mechatronic systems. Summary of the invention

[0003] In view of this, the present application provides a system and method for deploying a neural network for airborne mechatronic systems, so as to quickly optimize and deploy the neural network for airborne mechatronic systems, and effectively support the engineering implementation of the neural network for airborne mechatronics.

[0004] Specifically, the present application is implemented through the following technical solutions:

[0005] In a first aspect, the present application provides a deployment system of a neural network for an airborne electromechanical system, wherein the neural network is used to perform health monitoring and fault diagnosis tasks, control tasks, product design tasks or product manufacturing tasks in the airborne electromechanical system; the deployment system includes a client for optimizing the neural network, and an electronic device for storing and executing the optimized neural network optimized by the client; wherein:

[0006] The client includes a cascaded compression module, a performance optimization module, a parameter quantization module and a programming language conversion module; the client is specifically used to compress the neural network model using the compression module to obtain a lightweight neural network corresponding to the neural network model, and to optimize the performance of the lightweight neural network using the performance optimization module to obtain an optimized model after performance optimization corresponding to the lightweight neural network model, and then to quantize the parameters of the optimized model using the parameter quantization module to obtain a quantized model after parameter quantization, and to convert the quantized model from a first programming language to a second programming language using the programming language conversion module to obtain an optimized neural network corresponding to the second programming language of the quantized model; wherein the first programming language and the second programming language are different;

[0007] The electronic device includes a processor, a memory, a neural network processor NPU, an FPGA, and the optimized neural network stored in the memory and executable on the NPU; wherein the processor is used to perform task scheduling and allocation of computing resources; the FPGA is used to preprocess data from sensors and transmit the preprocessed data to the NPU; the NPU is used to execute and accelerate computing tasks related to the optimized neural network based on the preprocessed data; wherein the electronic device is used to accelerate computing of the optimized neural network. The second aspect of the present application provides a method for deploying a neural network for an airborne electromechanical system, the deployment method being implemented based on the deployment system described in any one of the first aspects of the present application; the deployment method includes:

[0008] Utilizing the client to optimize the neural network to be deployed, to obtain an optimized neural network;

[0009] The optimized neural network is deployed on the electronic device to accelerate the calculation of the optimized neural network through the electronic device.

[0010] The deployment system and method of a neural network for an airborne mechatronic system provided in the present application, by setting up a client and an electronic device, and sequentially compressing, optimizing performance, quantizing parameters and converting programming languages ​​of the neural network for the airborne mechatronic system through the client, can not only convert a larger neural network into a smaller, more efficient lightweight network, so as to improve its deployability on an airborne mechatronic system with limited hardware resources, and ensure that it can still operate effectively under limited computing power; in addition, by converting the neural network from a first programming language to a second programming language, it is not only helpful to adapt to different hardware platforms, but also beneficial to improve the operating efficiency of the system, so as to ensure that the neural network can quickly respond to data from sensors and improve the real-time performance of the system; further, in terms of deploying optimized electronic equipment after optimization, by using a processor, a neural network processor and an FPGA architecture, using the processor to schedule tasks and allocate computing resources, and using NPU and FPGA to perform optimization calculations, the reasoning speed of the neural network can be significantly improved, ensuring that the system has a fast response capability when performing control tasks, so as to further improve the real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a first embodiment of a neural network deployment system for an airborne electromechanical system provided by the present application;

[0012] Figure 2 A schematic diagram of a client shown as an exemplary embodiment of the present application;

[0013] Figure 3A schematic diagram of an electronic device shown as an exemplary embodiment of the present application;

[0014] Figure 4 A schematic diagram of a user interface of a client according to an exemplary embodiment of the present application;

[0015] Figure 5 A schematic diagram of a graphical user interface of a client shown as an exemplary embodiment of the present application;

[0016] Figure 6 A flowchart of a compression module implementing model compression as shown in an exemplary embodiment of the present application;

[0017] Figure 7 This is a schematic diagram of the implementation principle of model compression shown in an exemplary embodiment of the present application;

[0018] Figure 8 A schematic diagram of a graphical user interface of a client according to another exemplary embodiment of the present application;

[0019] Fig. 9 A flowchart of a performance optimization module implementing performance optimization as shown in an exemplary embodiment of the present application;

[0020] Fig.10 This is a schematic diagram showing an implementation principle of performance optimization according to an exemplary embodiment of the present application;

[0021] Fig.11 A graphical user interface of a client shown as another exemplary embodiment of the present application;

[0022] Fig.12 A flowchart of a programming language conversion module implementing programming language conversion as shown in an exemplary embodiment of the present application;

[0023] Fig.13 A flowchart of a first embodiment of a method for deploying a neural network for an airborne electromechanical system provided in the present application. DETAILED DESCRIPTION

[0024] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0025] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0028] Figure 1 A schematic diagram of a first embodiment of a neural network deployment system for an airborne electromechanical system provided by the present application, Figure 2 This is a schematic diagram of a client according to an exemplary embodiment of the present application. Figure 3 This is a schematic diagram of an electronic device according to an exemplary embodiment of the present application. Figure 1 , Figure 2 and Figure 3 The deployment system of a neural network for an airborne electromechanical system provided by the present application is used to optimize the deployment of a neural network for an airborne electromechanical system, wherein the neural network is used to perform health monitoring and fault diagnosis tasks, control tasks, product design tasks or product manufacturing tasks in the airborne electromechanical system; the deployment system includes a client for optimizing the neural network, and an electronic device for storing and executing the optimized neural network after the client optimizes; wherein:

[0029] The client includes a cascaded compression module, a performance optimization module, a parameter quantization module and a programming language conversion module; the client is specifically used to compress the neural network model using the compression module to obtain a lightweight neural network corresponding to the neural network model, and to optimize the performance of the lightweight neural network using the performance optimization module to obtain an optimized model after performance optimization corresponding to the lightweight neural network model, and then to quantize the parameters of the optimized model using the parameter quantization module to obtain a quantized model after parameter quantization, and to convert the quantized model from a first programming language to a second programming language using the programming language conversion module to obtain an optimized neural network corresponding to the second programming language of the quantized model; wherein the first programming language and the second programming language are different;

[0030] The electronic device includes a processor, a memory, a neural network processor NPU, an FPGA, and the optimized neural network stored in the memory and executable on the NPU; wherein the processor is used to perform task scheduling and allocation of computing resources; the FPGA is used to preprocess data from sensors and transmit the preprocessed data to the NPU; and the NPU is used to execute and accelerate computing tasks related to the optimized neural network based on the preprocessed data.

[0031] It should be noted that in the field of airborne mechatronic systems, the application research of artificial intelligence technology is mainly concentrated in the fields of health monitoring and fault diagnosis, intelligent control of mechatronic systems, intelligent product design, and intelligent manufacturing. In the airborne mechatronic system mission scenario, when the relevant neural network is actually deployed, the neural network is mostly designed with reference to the typical neural network structure parameters, and the structure is not optimized according to the specific airborne mechatronic tasks. There may be many redundant neuron nodes, which occupy a large amount of hardware computing resources and seriously affect the algorithm operation speed.

[0032] Furthermore, considering that neural networks have high requirements for hardware computing resources, the current application of deep learning models in the industry is mostly neural networks deployed on servers, workstations, and even data centers with large storage capacity, sufficient computing power, and stable power supply. At this time, it is necessary to upload the monitoring data of the edge device to the cloud data center, perform inference calculations, and finally send the calculation results to the edge device. However, this cloud computing method has problems with network latency, data privacy security, and high hardware costs, which seriously limits the application of neural networks in the field of airborne electromechanical.

[0033] The deployment system provided by this application can realize the structural compression, performance optimization, parameter quantization, and program language conversion (rapid deployment across hardware platforms) of the neural network model through the client, which can reduce redundant neuron nodes and speed up the calculation speed. Furthermore, after optimizing the neural network through the neural network model, it can be deployed to the electronic device to realize related tasks on the electronic device, and promote the engineering implementation of artificial intelligence technology on airborne electromechanical systems.

[0034] Specifically, the deployment system includes a client and an electronic device. Among them, the client can be a small program, an application APP, a web client, a browser client, etc., which will not be described in detail in this embodiment. Furthermore, when the client is a small program or an application APP, it can run on a computer, a server, and other devices. In addition, the electronic device can be a computer, a server, etc., which will not be limited in this embodiment. Optionally, in a possible implementation, the electronic device is an edge device.

[0035] It should be noted that edge devices specifically refer to airborne sensors (such as vibration sensors, temperature sensors), avionics equipment (such as flight data recorders, avionics control units), airborne communication equipment (such as aviation radio stations, satellite communication terminals), etc.

[0036] Referring to the previous description, it can be understood that the client mainly completes the structural compression, performance optimization, parameter quantization, and cross-programming language conversion functions of the neural network, and the electronic device is mainly used to run the optimized neural network after the client is optimized, and perform hardware calculations on the optimized neural network.

[0037] The following is a detailed introduction to the specific implementation principles of the client and electronic devices:

[0038] Optionally, in a possible implementation, the client provides a graphical user interface for interacting with the user; the client responds to instructions triggered by the user through the graphical user interface and executes actions corresponding to the instructions; wherein the instructions include at least compression instructions, performance optimization instructions, parameter quantization instructions, and programming language conversion instructions.

[0039] It should be noted that Figure 4 This is a schematic diagram of a user interface of a client according to an exemplary embodiment of the present application. Figure 4 , the client provides a graphical user interface for interaction with the user, and the graphical user interface provides virtual buttons for triggering relevant instructions. Figure 4 In the example shown, the graphical user interface provides a compression command trigger button ( Figure 4 ), the performance optimization command trigger button ( Figure 4), parameter quantization command trigger button ( Figure 4 ) and the programming language conversion command trigger button ( Figure 4 Furthermore, for example, under the compression instruction trigger button, a model import function trigger button, a data set recording function trigger button, etc. may be further provided.

[0040] When the user wants to perform relevant processing on the neural network, he can select the functional module he needs to use in the client.

[0041] In this implementation, for the neural network to be deployed (the neural network for the airborne electromechanical system), it is compressed, performance optimized, parameter quantized and programmed into a language in sequence through a compression module, a performance optimization module, a parameter quantization module and a programming language conversion module to obtain an optimized neural network.

[0042] Optionally, in a possible implementation, the first programming language is Python; the second programming language is C or C++.

[0043] It should be noted that for airborne mechatronic system controllers, the real-time requirements for algorithms are relatively high. System control programs are mostly implemented based on C or C++, while neural networks are mostly developed based on Python. The computing speed of the neural network is relatively slow under the Python language platform, and different algorithm development frameworks make it difficult to directly transplant algorithms to embedded controllers, which seriously affects the engineering application of neural networks. In this application, by converting neural networks from Python to C or C++, effective support is provided for the engineering implementation of neural networks for airborne mechatronic systems, so that they can be quickly deployed.

[0044] Furthermore, in order to improve the real-time performance of the neural network and improve the computing efficiency, in this embodiment, the internal structure of the electronic device that deploys the neural network is improved. Figure 3 In the present application, the electronic device includes a processor, a memory, a neural network processor NPU, an FPGA, and an optimized neural network stored in the memory and executable on the NPU.

[0045] Among them, the processor can be CPU, GPU, MCU, etc., which is used to schedule execution tasks and allocate computing resources; the memory is used to store and optimize neural networks; FPGA is used to preprocess data collected from sensors and transmit the preprocessed data to NPU; the neural network processor NPU is used to execute and accelerate computing tasks related to the optimization of neural networks based on the preprocessed data.

[0046] It should be noted that in this embodiment, the processor is good at processing logical tasks, so that the processor can perform comprehensive task management; the NPU is good at parallel computing, so that the NPU can perform algorithm acceleration; the FPGA is flexible and has the characteristics of parallel computing, so that the FPGA can realize front-end signal acquisition and preprocessing tasks.

[0047] Referring to the previous description, it can be understood that the hardware system of this electronic device generally adopts the "CPU+NPU+FPGA" architecture. For airborne electromechanical multi-task scenarios, a high-performance CPU is used for multi-task scheduling and computing resource management; based on high-speed bus technology, multi-chip interconnection and data interaction within the board are realized; based on the embedded NPU, the corresponding neural network acceleration computing driver is integrated to realize the accelerated computing of the neural network; for the real-time collection requirements of massive data, FPGA is used for real-time collection and preprocessing of multi-channel data to reduce the resource loss of the CPU. Specifically, FPGA can collect, solve and filter the data of the sensor in real time, and the CPU transmits data. Facing the scenario requirements and technical independent control requirements of high-performance and complex control tasks of airborne electromechanical systems, based on domestic operating systems, a real-time operating system that matches the hardware system is adapted; general software development tools and hardware underlying drivers are developed to facilitate the deployment of different master control programs and intelligent algorithms, and to improve the versatility of neural networks.

[0048] The following is a brief introduction to the specific working principles of each functional module in the client:

[0049] (1) Compression module

[0050] Figure 5 This is a schematic diagram of a graphical user interface of a client according to an exemplary embodiment of the present application. Figure 4 When the user wants to compress the neural network model, he can click the model compression button. In response to the user's click operation, the client will display the model compression interface to the user. The model compression interface is as follows Figure 5 Please refer to Figure 5 Through the model compression module, the user imports the neural network to be compressed, as well as the training samples and test samples corresponding to the neural network. Furthermore, the user can click the "Hyperparameter Setting" button to set the corresponding hyperparameters. After setting the hyperparameters, click the "Start Compression" button. At this time, the client responds to the compression instruction triggered by the user clicking Start Compression, and compresses the neural network imported by the user. After the compression is completed, the compression module outputs the corresponding lightweight neural network model.

[0051] In addition, after the compression module outputs the lightweight neural network model, the user can click the "Save Model" button to save the compressed lightweight neural network model.

[0052] It should be noted that after the compression is completed, the number of neurons / convolution kernels before and after the compression can also be displayed to the user, so that the user can intuitively determine whether he is satisfied with the current compression effect through the display.

[0053] The following is an introduction to the specific principles of how the compression module implements model compression:

[0054] Specifically, the compression module is mainly implemented based on reinforcement learning and network pruning technology, unifying the pruning action of the neural network, as well as the performance test and evaluation process into a Markov process, and building a network pruning action and interactive environment. The importance evaluation criteria of the basic units of the neural network are established, the network pruning process is uniformly modeled, and the neural network is pruned and structurally optimized from the convolution kernel and neuron dimensions.

[0055] Furthermore, a pruning agent model is designed, and the parameters of the pruning agent model are optimized based on the deep deterministic gradient strategy optimization method. The real-time pruning strategy (pruning rate) of the neural network is output, and finally a lightweight neural network with a balance between network scale and performance is obtained.

[0056] Specifically, Figure 6 This is a flowchart of a compression module implementing model compression as shown in an exemplary embodiment of the present application. Please refer to Figure 6 The compression module is provided with a pruning agent model; the specific process of the compression module performing model compression may include:

[0057] S601. Take the neural network as the current network, input the state parameters of each layer of the current network into the pruning agent model, and output the current pruning strategy from the action network of the pruning agent model; wherein the current pruning strategy includes the current pruning rate of each layer.

[0058] Specifically, Figure 7 This is a schematic diagram of the implementation principle of model compression shown in an exemplary embodiment of this application, please refer to Figure 7 ,The pruning agent model consists of two parts: the action network and the evaluation network. The action network is used to generate the pruning strategy (i.e., the pruning rate), while the evaluation network is used to evaluate the effect of the current strategy. Furthermore, the input of the action network is the state parameters of each layer, and the output is the pruning rate of each layer. The input state parameters are processed by the action network, and the pruning rate of each layer is output.

[0059] It should be noted that the state parameters of each layer may include the network layer number, network layer type, the number of neurons in the network layer, convolution kernel, calculation amount, parameter amount, etc.

[0060] S602: Based on a pre-constructed importance evaluation criterion, importance evaluation is performed on the basic units of the current network to obtain an importance evaluation result.

[0061] Specifically, the importance evaluation criteria are pre-established. In addition, the specific content of the importance evaluation criteria is set according to actual needs and is not limited in this embodiment. For example, in one possible implementation, the importance evaluation criteria can be constructed based on the parameter size of the current network, the degree of influence on the output result, etc. It should be noted that the basic unit of the network can be a neuron, a convolution kernel, etc.

[0062] For example, in one embodiment, when implementing, assuming that the basic unit of the current network is a neuron, the variance of the neuron output can be evaluated according to the pre-established importance evaluation criteria, and the neuron with a larger output variance is more important. Based on this, the importance of each neuron in the current network can be evaluated.

[0063] S603: Perform a pruning operation on the current network according to the importance evaluation result and the current pruning rate of each layer to generate an intermediate network.

[0064] In specific implementation, pruning operations are performed in combination with the importance evaluation results and the current pruning rate of each layer. For example, in one embodiment, the current pruning rate of a certain network layer is 30%. According to the importance evaluation results of the network layer, the first 30% of unimportant basic units in the network layer are removed in order from low to high importance to obtain the intermediate network corresponding to the network layer.

[0065] S604: Train and test the intermediate network using the training sample set and the test sample set to obtain performance indicators of the intermediate network.

[0066] Specifically, the intermediate network is trained using the training sample set, and the trained intermediate network is tested using the test sample set to obtain the performance index of the intermediate network. It should be noted that the performance index may be an index such as accuracy and loss.

[0067] S605. Feedback the performance indicator and the state parameters of the intermediate network to the pruning agent model, and the evaluation network of the pruning agent model calculates the reward value of the current pruning strategy according to the performance indicator and the state parameters of the intermediate network.

[0068] Specifically, referring to the previous description, the evaluation network calculates the reward value of the current pruning strategy according to the performance indicators of the intermediate network and the feedback information of the state parameters as the basis for optimization.

[0069] It is understandable that if the performance indicators of the intermediate network do not decrease significantly, or even improve, the reward value of the current pruning strategy is high; otherwise, the reward value of the current pruning strategy is low.

[0070] S606, using the reward value as a return, updating the action network through the policy gradient method, and using the intermediate network as the current network, and inputting the state parameters of each layer of the current network into the pruned agent model again, until the termination condition is met, thereby obtaining the pruned lightweight neural network.

[0071] Specifically, the policy gradient method is a reinforcement learning algorithm used to update the action network in the pruned agent model, that is, to update the pruning strategy of each layer. In specific implementation, the action network is updated using the policy gradient method according to the calculated reward value, so that the action network can obtain a pruning strategy with a high reward value in subsequent outputs. After updating the action network, steps S601 to S606 can be repeated until the termination condition is met to obtain the pruned lightweight neural network. In this way, after multiple iterations and optimizations, the obtained lightweight neural network has fewer parameters while maintaining performance, achieving a lightweight effect.

[0072] The deployment system provided in this embodiment not only evaluates the importance of each parameter or channel through importance evaluation criteria (such as weight size, gradient information, activation value, etc.) to quickly identify those parts that contribute less to the network as potential pruning targets, but also in each iteration after pruning, the pruning agent model dynamically adjusts the pruning strategy based on actual performance (the actual performance of the model after pruning) feedback through a reinforcement learning algorithm. In this way, the pruning process not only relies on static importance evaluation, but also optimizes the pruning strategy through dynamic learning. Through continuous iterative optimization, the pruning agent can find the best balance between performance and efficiency, avoid performance degradation caused by excessive pruning, and better balance performance and efficiency.

[0073] It should be noted that compared with the original neural network, the lightweight neural network has a slight decrease in performance such as robustness. Therefore, in this embodiment, the performance of the lightweight network continues to be optimized to improve the accuracy and robustness of the lightweight neural network.

[0074] The following is an introduction to the performance optimization model:

[0075] (2) Performance optimization module

[0076] Figure 8 This is a schematic diagram of a graphical user interface of a client according to another exemplary embodiment of the present application. Figure 4When the user wants to optimize the performance of the neural network, he can click the model optimization button. In response to the user's click operation, the client will display the model optimization interface to the user. The model optimization interface is as follows Figure 8 Please refer to Figure 8 Through the model optimization interface, the user can import the compressed lightweight neural network model into the student model input box, and import the pre-trained target deep neural network into the teacher model input box, and import the corresponding training samples and test samples into the corresponding data set input box. After successfully importing the data, the user can further click the "Execute button". At this time, in response to the performance optimization instruction triggered by the user through the execute button, the client uses the target deep neural network input by the user to optimize the performance of the lightweight neural network model, and further outputs the optimized model after performance optimization.

[0077] In addition, please continue to refer to Figure 8 ,After the performance optimization is completed, the user also clicks the “Save Model” button to save the ,optimized model after performance optimization.

[0078] It should be noted that after the performance optimization is completed, the network parameters and model performance can also be displayed to the user, so that the user can directly determine whether the optimized model meets the requirements through the display.

[0079] The specific principles of how the performance optimization module achieves performance optimization are introduced below.

[0080] Specifically, the performance optimization module uses a target deep neural network with a larger structure and better performance as the teacher network, and a compressed lightweight neural network as the student network. It introduces the feature loss of the intermediate layer and the probability loss of the output layer as soft labels in the loss function of the student network, and reconstructs the loss function of the student network in combination with the hard labels of the original data set. By amplifying and reducing the loss value, knowledge transfer is achieved during the training process to improve the training effect of the lightweight network.

[0081] Specifically, Fig. 9 This is a flowchart of a performance optimization module implementing performance optimization as shown in an exemplary embodiment of the present application. Please refer to Fig. 9 The specific process of implementing performance optimization by the performance optimization module may include:

[0082] S901. Use a pre-trained target deep neural network as a teacher network and the lightweight neural network model as a student network; wherein the input and output of the target deep neural network and the lightweight neural network model are the same, the number of parameters of the target deep neural network is greater than the number of parameters of the lightweight neural network model, the depth of the target deep neural network is deeper than the depth of the lightweight neural network, and the performance of the target deep neural network is better than the performance of the lightweight neural network model.

[0083] Specifically, compared with the original deep neural network, the robustness and other performance of the lightweight neural network obtained after compression is slightly reduced. Therefore, it is necessary to optimize the performance of the lightweight neural network to improve the accuracy and robustness of the lightweight neural network.

[0084] Furthermore, the pre-trained target deep neural network is used as the teacher network, and the lightweight neural network model is used as the student network, and the teacher network is used to optimize the performance of the student network.

[0085] It should be noted that the number of parameters of the teacher network is greater than that of the student model, and the depth of the teacher network is deeper than that of the student network. In this way, the teacher network has stronger expression and learning capabilities, and can perform multi-level abstraction and feature extraction of data to help the student network learn and optimize performance. In addition, the input and output of the teacher network and the student network need to be the same. It should be noted that the pre-trained target deep neural network can be the original neural model before compression, or other models with the same input and output as the lightweight neural network, which will not be described here.

[0086] S902. Through the knowledge distillation technology, the teacher network is used to guide the student network to train, so as to obtain a student network with optimized performance; wherein the loss function of the student network is a loss function reconstructed by combining the first loss function, the second loss function and the third loss function; the first loss function is the loss of the true label, which is used to measure the difference between the predicted output of the student network and the true label; the second loss function is the feature loss of the middle layer, which is used to compare the difference between the features extracted by the student network and the teacher network in the middle layer; the third loss function is the probability loss of the output layer, which is used to compare the difference between the probability distribution output by the student network and the teacher network in the output layer; the first loss function, the second loss function and the third loss function have different weights, and the weight of the first loss function, the weight of the second loss function and the weight of the third loss function are dynamically adjusted during the training process.

[0087] Specifically, Fig.10 This is a schematic diagram of an exemplary embodiment of the present application showing the implementation principle of performance optimization, please refer to Fig.10 ,The core principle of knowledge distillation technology is to let the student network learn the ,behavior and knowledge of the teacher network, and use the teacher network to guide the ,training of the student network, and realize knowledge transfer during the ,training process.

[0088] In this implementation, a pre-trained target deep neural network is used as the teacher network, and a lightweight neural network is used as the student network. It can be understood that compared with the student network, the teacher network is a homogeneous deep neural network with a larger number of parameters and a deeper depth.

[0089] Furthermore, by introducing the predicted output of the student network and the true label of the original data set and combining the difference between the two, the first loss function of the student network is constructed. By comparing the difference in feature representation between the teacher network and the student network, the second loss function of the student network is constructed. By comparing the difference between the output layers of the teacher network and the student network, the third loss function of the student network is constructed.

[0090] Referring to the previous description, it can be understood that the first loss function is used to measure the difference between the predicted output of the student network and the true label. If the predicted output of the student network is significantly different from the true label, the value of the first loss function will be larger, which can prompt the student network to adjust its parameters to better fit the true label. Furthermore, the second loss function is used to compare the difference between the features extracted by the student network and the teacher network in the middle layer. By minimizing the difference between the features of the two, the student network can learn the effective feature representation learned by the teacher network in the middle layer. The student network can learn the advantages of the teacher network in extracting these features through the second loss function. In addition, the third loss function is used to compare the difference between the probability distributions output by the student network and the teacher network in the output layer, prompting the output probability distribution of the student network to move closer to the teacher network.

[0091] Optionally, in a possible implementation, the loss function of the student network is a weighted sum of a first loss function, a second loss function, and a third loss function, and the weights of the first loss function, the second loss function, and the third loss function are different and are dynamically adjusted during training.

[0092] In combination with the above introduction, in a specific implementation, in one embodiment, the loss function of the student network is:

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] Wherein, A is the first loss function;

[0098] B is the second loss function;

[0099] The C is the third loss function;

[0100] Said Output for student networks;

[0101] Said is the output of the true label;

[0102] Said is the middle layer feature of the teacher network;

[0103] Said is the middle layer feature of the student network;

[0104] Said is the output of the teacher network;

[0105] Said is the soft label loss weight coefficient;

[0106] Said is the feature loss weight coefficient;

[0107] Said is the distillation temperature.

[0108] The deployment system provided in this embodiment adopts knowledge distillation technology to guide the optimization of lightweight neural network (student network) through pre-trained deep neural network (teacher network), aiming to improve the performance of lightweight model, which is specifically manifested as follows: First, by introducing three different losses (loss of true label, feature loss of intermediate layer, probability loss of output layer) into the loss function of student network, the training process of student network can be made more refined, so that not only the accuracy of student network prediction output can be guaranteed, but also the student network can learn intermediate features from teacher network, so as to improve its representation ability while keeping the model structure simple; secondly, by dynamically adjusting the weights of the three losses during the training process, the student network can focus on different aspects at different training stages, so as to achieve more refined optimization effect; finally, through the above method, the optimized network can maintain the lightweight advantage and the performance is close to that of teacher network, so as to improve its application on devices with limited resources (such as mobile terminals, embedded systems, etc.).

[0109] (3) Parameter quantization module

[0110] It should be noted that parameter quantization is a technology used in the field of deep learning to reduce the storage space and computational complexity of model parameters. Its basic implementation principle is to convert network parameters (such as weights and biases) stored and calculated with high precision in deep learning models into low-precision data types. The quantization mapping function is used to map parameter values ​​from high-precision representation space to low-precision representation space based on the analysis and statistics of the original parameter distribution. The purpose is to reduce the storage requirements of the model, speed up the calculation process, and reduce hardware costs.

[0111] The client provided in this embodiment can implement the parameter quantization function. For example, in this embodiment, the optimization model can be quantized with low precision to further compress the space occupied by the neural network on the embedded computing platform and improve the real-time performance of the algorithm. It should be noted that the specific implementation process of parameter quantization can refer to the description of the relevant technology and will not be repeated here.

[0112] (4) Programming language conversion module

[0113] Fig.11 This is a graphical user interface of a client shown in another exemplary embodiment of the present application. Figure 4 When the user wants to convert the programming language of the neural network, he can click the model conversion button. Further, in response to the user's click operation, the client will display the model conversion interface to the user. The model conversion interface is as follows Fig.11 Please refer to Fig.11 Through the model conversion interface, the user can import the quantized model with quantized parameters into the network model input box, and import the corresponding test set samples and test set labels into the data set input box. Further, after successfully importing the data, click the "Execute button". At this time, in response to the programming language conversion instruction triggered by the user through the execute button, the client performs programming language conversion on the quantized model, and after the programming language conversion is completed, the optimized neural network is output.

[0114] In addition, the user can also save the optimized neural network by clicking the "Save Model" button after outputting the optimized neural network on the client.

[0115] Specifically, Fig.12 This is a flowchart of a programming language conversion module implementing programming language conversion as shown in an exemplary embodiment of the present application. Please refer to Fig.12 , converting the programming language of the quantization model from a first programming language to a second programming language to obtain an optimized neural network of the quantization model corresponding to the second programming language, comprises:

[0116] S1201. Analyze the quantization model according to a pre-built analysis tool to obtain multiple target operators included in the quantization model.

[0117] Specifically, the analysis tool can be used to traverse the model structure of the quantization model, and then the operator identification can be performed on the nodes in the quantization model to identify all the operators included in the quantization model. For the convenience of distinction, all the operators included in the quantization model are the target operators.

[0118] It should be noted that the pre-built parsing tool can be a tool that comes with the deep learning framework, a third-party model parsing tool, a custom script parsing tool, etc., and is not limited to this in this embodiment.

[0119] S1202. Obtain a library function corresponding to each of the target operators from a pre-built database; wherein the database stores library functions corresponding to each basic operator; the library function corresponding to each basic operator is a library function built based on the second programming language, and is used to express the reasoning process for implementing the basic operator.

[0120] Specifically, the pre-built database includes library functions corresponding to all basic operators. It should be noted that all basic operators include all operators that may be involved in neural networks, for example, at least including the following operators: convolution calculation, full connection calculation, batch normalization calculation, pooling calculation and activation calculation. In addition, the library function corresponding to each basic operator is a library function built based on the second programming language, which is used to express the reasoning process of implementing the basic operator.

[0121] Optionally, in a possible implementation, the process of constructing the pre-constructed database includes:

[0122] (1) Constructing multiple forward reasoning neural network models; wherein each of the forward reasoning neural network models includes at least the following basic operators: convolution calculation, full connection calculation, batch normalization calculation, pooling calculation and activation calculation.

[0123] (2) For each basic operator, according to the input variables and output variables of the basic operator, based on the second programming language, construct a library function corresponding to the basic operator, so as to express and implement the reasoning process of the operator through the library function.

[0124] (3) Construct the database according to the library functions corresponding to each basic operator.

[0125] Specifically, the forward reasoning neural network model is used to generate predicted outputs based on input data. It can be understood that the forward reasoning neural network model is usually composed of multiple basic operators (such as convolution, full connection, batch normalization, pooling and activation functions). These basic operators are the basic computing units in the forward reasoning neural network, which can perform various operations on the input data, extract features and finally obtain outputs.

[0126] When implementing it, you can choose a suitable deep learning framework (such as TensorFlow, PyTorch, etc.) to build a forward reasoning neural network model. Specifically, when building a forward reasoning neural network model, you can design multiple neural network models with different structures, and each model uses a different combination of operators to ensure that multiple operation modes are covered. For example, you can build a convolutional neural network (CNN) that includes a convolutional layer, a pooling layer, and a fully connected layer.

[0127] Specifically, for each basic operator, the corresponding library function must be constructed using the second programming language according to its input variables and output variables. Among them, the input variable refers to the data required by the operator when performing the operation, and the output variable is the data obtained after the operator is operated. It can be understood that each basic operator has its unique input and output characteristics in the reasoning process. By writing library functions, these operations can be encapsulated, which is convenient for subsequent programming language conversion based on this.

[0128] Referring to the above description, it can be understood that the pre-built database stores the library functions corresponding to the various target operators identified in step S1201. In this step, the library function corresponding to each target operator can be directly searched from the database.

[0129] It should be noted that by building a database, the library functions of all basic operators can be centrally managed to facilitate subsequent calls and maintenance. In addition, the database can be used to store the metadata of the basic operators (such as name, type, input and output dimensions, etc.) and the corresponding implementation code to facilitate subsequent rapid conversion.

[0130] S1203, based on the network structure of the quantization model, the library functions corresponding to each target operator are connected in series to obtain the optimized neural network.

[0131] Specifically, after obtaining the library functions corresponding to each target operator, these library functions are connected in sequence and logically according to the network structure of the quantization model, and finally an optimized neural network based on the second programming language is obtained.

[0132] The deployment system provided in this embodiment can determine that the optimized neural network can run on different development environments and platforms by converting the quantization model from the first programming language to the second programming language, which can not only improve the applicability of the model and facilitate its use in application scenarios, but also shorten the development cycle and reduce development costs. In addition, by encapsulating the implementation of each basic operator as a library function and pre-building a database based on this, it can not only reduce repetitive coding work and improve the reusability of the code, but also realize efficient calling of the basic operator and improve the conversion speed. In addition, the library functions of the basic operators are stored in the database, which is very convenient for unified management, maintenance, and expansion.

[0133] It is understandable that the hardware computing resources of airborne mechatronic systems are weak, the deployability of neural networks is poor, and the real-time performance is poor. The deployment system provided in this implementation can optimize the neural networks for airborne mechatronic systems based on model compression, performance optimization, and cross-programming language conversion. In addition, by improving the electronic equipment that deploys the optimized optimization model, multi-core accelerated computing of multiple types of tasks such as data acquisition, neural network accelerated computing, and task management can be achieved to ensure the real-time performance of system tasks and intelligent algorithms. In this way, rapid optimization and deployment of neural networks for airborne mechatronic systems can be achieved, effectively supporting the engineering implementation of neural networks for airborne mechatronic systems.

[0134] The deployment system provided in this embodiment sets up a client and an electronic device, and sequentially compresses, optimizes performance, quantizes parameters, and converts programming languages ​​for the neural network for the airborne electromechanical system through the client. In this way, not only can a larger neural network be converted into a smaller, more efficient lightweight network, so as to improve its deployability on the airborne electromechanical system with limited hardware resources, and ensure that it can still operate effectively under limited computing power; in addition, by converting the neural network from a first programming language to a second programming language, it is not only helpful to adapt to different hardware platforms, but also beneficial to improve the operating efficiency of the system, so as to ensure that the neural network can quickly respond to data from sensors and improve the real-time performance of the system; further, in terms of the optimized electronic device after deployment, by using a processor, a neural network processor, and an FPGA architecture, using the processor to schedule tasks and allocate computing resources, and using NPU and FPGA for optimization calculations, the reasoning speed of the neural network can be significantly improved, ensuring that the system has a fast response capability when performing control tasks, so as to further improve the real-time performance.

[0135] In summary, it can be understood that the deployment system provided in this embodiment, combined with the client and electronic devices, solves the challenges of neural network deployment in airborne mechatronic systems through model compression, performance optimization, parameter quantization and cross-programming language conversion, as well as the improvement of the computing power of electronic devices, significantly improves real-time and multi-tasking capabilities, and provides effective support for engineering implementation. These advantages make neural network applications for airborne mechatronic systems more efficient, flexible and reliable, laying the foundation for future intelligent development.

[0136] Corresponding to the aforementioned embodiment of a system for deploying a neural network for an airborne mechatronic system, the present application also provides an embodiment of a method for deploying a neural network for an airborne mechatronic system.

[0137] Fig.13 This is a flowchart of the first embodiment of the method for deploying a neural network for an airborne electromechanical system provided by this application. Fig.13 The deployment method provided in this embodiment is implemented based on the deployment system provided in the first aspect of the present application; the deployment method includes:

[0138] S1301. Utilize the client to optimize the neural network to be deployed, and obtain an optimized neural network.

[0139] S1302: deploy the optimized neural network on the electronic device.

[0140] The method of this embodiment is based on Figure 1 The implementation principle and process of the deployment system embodiment shown are similar and will not be repeated here.

[0141] The method provided in this embodiment combines the client and the electronic device to deploy the neural network optimized by the client to the electronic device, thereby improving the computing power of the electronic device, solving the challenges of neural network deployment in airborne mechatronic systems, significantly improving real-time performance and multi-tasking capabilities, and providing effective support for engineering implementation. These advantages make neural network applications for airborne mechatronic systems more efficient, flexible, and reliable, laying the foundation for future intelligent development.

[0142] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A neural network deployment system for airborne electromechanical systems, characterized in that: The neural network is used to perform health monitoring and fault diagnosis tasks, control tasks, product design tasks or product manufacturing tasks in the airborne electromechanical system; the deployment system includes a client for optimizing the neural network and an electronic device for storing and executing the optimized neural network optimized by the client; wherein: The client includes a cascaded compression module, a performance optimization module, a parameter quantization module and a programming language conversion module; the client is specifically used to compress the neural network model using the compression module to obtain a lightweight neural network corresponding to the neural network model, and to optimize the performance of the lightweight neural network using the performance optimization module to obtain an optimized model after performance optimization corresponding to the lightweight neural network model, and then to quantize the parameters of the optimized model using the parameter quantization module to obtain a quantized model after parameter quantization, and to convert the quantized model from a first programming language to a second programming language using the programming language conversion module to obtain an optimized neural network corresponding to the second programming language of the quantized model; wherein the first programming language and the second programming language are different; The electronic device includes a processor, a memory, a neural network processor NPU, an FPGA, and the optimized neural network stored in the memory and executable on the NPU; wherein the processor is used to perform task scheduling and allocation of computing resources; the FPGA is used to preprocess data from sensors and transmit the preprocessed data to the NPU; the NPU is used to execute and accelerate computing tasks related to the optimized neural network based on the preprocessed data; wherein the electronic device is used to accelerate computing of the optimized neural network.

2. The deployment system according to claim 1, characterized in that: The converting the programming language of the quantization model from a first programming language to a second programming language to obtain an optimized neural network of the quantization model corresponding to the second programming language includes: Parsing the quantization model according to a pre-built parsing tool to obtain multiple target operators included in the quantization model; Obtaining a library function corresponding to each of the target operators from a pre-built database; wherein the database stores library functions corresponding to each of the basic operators; the library function corresponding to each basic operator is a library function built based on the second programming language, and is used to express the reasoning process for implementing the basic operator; The library functions corresponding to each target operator are connected in series based on the network structure of the quantization model to obtain the optimized neural network.

3. The deployment system according to claim 2, characterized in that: The construction process of the pre-built database includes: Constructing multiple forward reasoning neural network models; wherein each of the forward reasoning neural network models includes at least the following basic operators: convolution calculation, full connection calculation, batch normalization calculation, pooling calculation and activation calculation; For each basic operator, according to the input variables and output variables of the basic operator, based on the second programming language, a library function corresponding to the basic operator is constructed, so as to express and implement the reasoning process of the operator through the library function; The database is constructed according to the library functions corresponding to each basic operator.

4. The deployment system according to claim 1, characterized in that: The performance optimization module is specifically used for: A pre-trained target deep neural network is used as a teacher network, and the lightweight neural network model is used as a student network; wherein the input and output of the target deep neural network and the lightweight neural network model are the same, and the parameter amount of the target deep neural network is greater than the parameter amount of the lightweight neural network model, the depth of the target deep neural network is deeper than the depth of the lightweight neural network, and the performance of the target deep neural network is better than the performance of the lightweight neural network model; Through the knowledge distillation technology, the teacher network is used to guide the student network to train, and a student network with optimized performance is obtained; wherein the loss function of the student network is a loss function reconstructed by combining the first loss function, the second loss function and the third loss function; the first loss function is the loss of the true label, which is used to measure the difference between the predicted output of the student network and the true label; the second loss function is the feature loss of the middle layer, which is used to compare the difference between the features extracted by the student network and the teacher network in the middle layer; the third loss function is the probability loss of the output layer, which is used to compare the difference between the probability distribution output by the student network and the teacher network in the output layer; the first loss function, the second loss function and the third loss function have different weights, and the weight of the first loss function, the weight of the second loss function, and the weight of the third loss function are dynamically adjusted during the training process.

5. The deployment system according to claim 1, characterized in that: The compression module is provided with a pruning agent model; the compression module is specifically used for: The neural network is used as the current network, and the state parameters of each layer of the current network are input into the pruning agent model, and the action network of the pruning agent model outputs the current pruning strategy; wherein the current pruning strategy includes the current pruning rate of each layer; Based on the pre-constructed importance evaluation criteria, importance evaluation is performed on the basic units of the current network to obtain an importance evaluation result; Performing a pruning operation on the current network according to the importance evaluation result and the current pruning rate of each layer to generate an intermediate network; Using the training sample set and the test sample set to train and test the intermediate network, and obtain the performance index of the intermediate network; Feeding back the performance index and the state parameters of the intermediate network to the pruning agent model, wherein the evaluation network of the pruning agent model calculates the reward value of the current pruning strategy according to the performance index and the state parameters of the intermediate network; The reward value is used as a return, the action network is updated through the policy gradient method, and the intermediate network is used as the current network. The state parameters of each layer of the current network are input into the pruned agent model again until the termination condition is met, thereby obtaining the pruned lightweight neural network.

6. The deployment system according to claim 4, characterized in that: The loss function of the student network is: ； ； ； ； Wherein, A is the first loss function; B is the second loss function; The C is the third loss function; Said Output for student networks; Said is the output of the true label; Said is the middle layer feature of the teacher network; Said is the middle layer feature of the student network; Said is the output of the teacher network; Said is the soft label loss weight coefficient; Said is the feature loss weight coefficient; Said is the distillation temperature.

7. The deployment system according to claim 1, characterized in that: The client provides a graphical user interface for interacting with the user; the client responds to the instructions triggered by the user through the graphical user interface and executes the actions corresponding to the instructions; wherein the instructions include at least compression instructions, performance optimization instructions, parameter quantization instructions, and programming language conversion instructions.

8. The deployment system according to claim 1, characterized in that: The first programming language is Python; the second programming language is C or C++.

9. The deployment system according to claim 1, characterized in that: The electronic device is an edge device.

10. A method for deploying a neural network for an airborne electromechanical system, characterized in that: The deployment method is implemented based on the deployment system according to any one of claims 1 to 9; the deployment method comprises: Utilizing the client to optimize the neural network to be deployed, to obtain an optimized neural network; The optimized neural network is deployed on the electronic device to accelerate the calculation of the optimized neural network through the electronic device.

Images