On-line reasoning system and reasoning method for airborne intelligent auxiliary decision-making algorithm

Abstract

The invention belongs to the field of artificial intelligence and flight tests, and particularly relates to an online reasoning system and method for an airborne intelligent aid decision algorithm. The method comprises the following steps of: 1, preprocessing input data to obtain input data of an identification algorithm and input data of a modified algorithm, and storing the input data and the input data in a storage area of an NPU (Network Processing Unit); 2, performing data format conversion on the storage area data of the NPU to obtain format-converted data; 3, reasoning by using the data subjected to format conversion and an intelligent auxiliary flight decision algorithm to obtain an algorithm reasoning result of the NPU; 4, the algorithm reasoning result of the NPU is sent to a CPU for format conversion and post-processing, and a final auxiliary decision-making result of the principle prototype is obtained; and 5, sending a final auxiliary decision result of the principle prototype to an airborne test system and an airborne display control system to obtain a visual intelligent auxiliary flight decision knot.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence and flight tests, and particularly relates to an online inference system and an inference method for an airborne intelligent auxiliary decision-making algorithm. Background Art

[0002] The intelligent flight auxiliary decision-making algorithm trained in the ground flight simulation test environment needs to be loaded and inferred online in the airborne system to carry out intelligent auxiliary enhanced flight tests. Currently, airborne processing systems such as airborne flight control computers, flight management computers, and integrated display and control computers are used to process conventional C code programs. Their closed operating systems and development platforms cannot provide the loading and running of intelligent algorithms, and lack corresponding algorithm libraries. Therefore, it is necessary to study an airborne intelligent auxiliary flight prototype to build a computing platform for the intelligent flight auxiliary decision-making algorithm and realize the online inference of the intelligent auxiliary algorithm. Summary of the Invention

[0003] Object of the Invention: To provide an online inference system and an inference method for an airborne intelligent auxiliary decision-making algorithm, which are used for the operation of the intelligent auxiliary flight decision-making algorithm, support the loading and online inference of the intelligent algorithm, and perform real-time data interaction with the airborne flight control system and the test system, so as to support the intelligent auxiliary enhanced flight test.

[0004] Technical Solution:

[0005] An online inference system for an airborne intelligent auxiliary decision-making algorithm includes: an airborne test system, an airborne intelligent auxiliary enhanced prototype, and an airborne display and control system. Among them, the airborne intelligent auxiliary enhanced prototype includes an NPU and a CPU, and the NPU and the CPU communicate with each other through a backplane bus; the airborne intelligent auxiliary enhanced prototype communicates with the airborne test system through Ethernet, and the airborne intelligent auxiliary enhanced prototype communicates with the airborne display and control system through an ARINC429 bus.

[0006] An online inference method for an airborne intelligent auxiliary decision-making algorithm is executed by means of the above-mentioned online inference system for an airborne intelligent auxiliary decision-making algorithm, and the method includes:

[0007] Step 1: Preprocess the input data to obtain the input data of the recognition algorithm and the input data of the recovery algorithm, and store them in the storage area of the NPU;

[0008] Step 2: Convert the data format of the data in the storage area of the NPU to obtain the data after format conversion;

[0009] Step 3: Use the data after format conversion and the intelligent auxiliary flight decision-making algorithm for inference to obtain the algorithm inference result of the NPU;

[0010] Step 4: Send the algorithm inference result of the NPU to the CPU for format conversion and post-processing to obtain the final auxiliary decision result of the prototype;

[0011] Step 5: Send the final auxiliary decision result of the prototype to the airborne test system and the airborne display and control system to obtain the visual intelligent auxiliary flight decision result.

[0012] Furthermore, in Step 1, the acquisition of the input data of the recognition algorithm includes:

[0013] First, the CPU receives data from the airborne test system in real time through Ethernet, receiving 36 floating-point data each time and storing them as a one-dimensional array with a length of 144 bytes; packetize the data according to the requirements of the recognition algorithm in the intelligent auxiliary flight decision, that is, continuously receive 50 Ethernet data, and combine 50 one-dimensional arrays in sequence to form a two-dimensional array with 144 columns and 50 rows. The first row stores the data received first, and the 50th row stores the data received last; then normalize all the data in the two-dimensional array, that is, map the floating-point data with an absolute value > 1 to data ≤ 1 according to the data range; finally, the CPU transfers the normalized data to the data storage area of the NPU through the backplane bus;

[0014] After continuously receiving 5 Ethernet data, the data storage area of the NPU is updated once: discard the data in the first to fifth rows of the previous two-dimensional array according to the FIFO principle, shift the data in the sixth to 50th rows to the first to 45th rows, and at the same time supplement the received 5 frames of data to the 46th to 50th rows of the two-dimensional array to form a new data block for driving the real-time solution of the intelligent recognition algorithm.

[0015] Furthermore, in Step 1, the acquisition of the input data of the go-around algorithm is specifically as follows:

[0016] First, the CPU receives data from the airborne test system in real time through Ethernet, receiving 36 floating-point data each time and storing them as a one-dimensional array with a length of 144 bytes; then normalize all the data in the one-dimensional array, and finally the CPU transfers the normalized data to the data storage area of the NPU through the backplane bus;

[0017] Secondly, receive the output result UINT8 data of the recognition algorithm and jointly form the input data of the go-around algorithm with the data storage area of the NPU.

[0018] Furthermore, Step 2 is specifically as follows:

[0019] The data in the storage area of the NPU are all 32-bit floating-point data. According to the intelligent auxiliary flight decision-making algorithm, the data in the storage area of the NPU are divided into higher-precision data and lower-precision data. The higher-precision data include three-axis angular rate, three-axis attitude angle, and three-axis overload. The lower-precision data include speed, altitude, and vertical speed. For the higher-precision data, they are converted into data in the FP16 format, while for the lower-precision data, they are converted into data in the INT8 format.

[0020] Further, step three is specifically as follows:

[0021] The recognition algorithm LSTM adopts a dual-core parallel acceleration single-algorithm mode, and the improved algorithm PPO adopts a dual-core dual-algorithm acceleration mode.

[0022] The two-dimensional data after format conversion are simultaneously input into CORE0 and CORE1, and processed using the recognition algorithm LSTM. The output results of the recognition algorithm are respectively input into CORE2 and CORE3. The one-dimensional array after format conversion is simultaneously input into CORE2 and CORE3. The NPU selects to use the improved algorithm 1 or the improved algorithm 2 for calculation according to the recognition algorithm results, and obtains the output results of the improved algorithm, which are used as the rudder surface deflection instructions to be executed by the aircraft.

[0023] Further, step four is specifically as follows:

[0024] Specifically: The algorithm inference results of the NPU are sent to the data storage area of the CPU through the backplane bus. The inference results are divided into two types: recognition results and improved instructions. For the inference results, it is necessary to convert the percentage statistical results of each scenario into UINT8 type data, that is, set the scenario flag corresponding to the maximum percentage to 1, and set the other scenario flags to 0 as the final recognition result; while the improved instructions should be sorted according to the control rudder surface type and throttle control, which are respectively the elevator deflection instruction, aileron deflection instruction, rudder deflection instruction, and throttle lever instruction.

[0025] Further, step five is specifically as follows:

[0026] Specifically: The intelligent auxiliary flight decision-making results are sent to the airborne test system in real time through Ethernet for data recording and monitoring; at the same time, the intelligent auxiliary flight decision-making results are sent to the airborne display and control system in real time through the ARINC429 bus to assist the pilot in making recognition and improvement operations in dangerous scenarios.

[0027] Beneficial effects:

[0028] 1. Through the design technology of the airborne intelligent auxiliary flight principle prototype, the present invention realizes the online loading and inference of the intelligent auxiliary flight decision-making algorithm, thus strongly supporting the smooth development of the intelligent auxiliary flight decision-making algorithm in multiple scenarios of flight tests.

[0029] 2. Compared with the previously used airborne flight control computer, the offline compilation and online inference technology of the present invention can compile the neural network algorithm for ground simulation training into an algorithm file recognizable by the principle prototype, and at the same time associate the algorithm file with the NPU, so as to complete the loading and inference of the intelligent algorithm.

[0030] 3. Compared with the previously used airborne flight control computer, the online inference management technology of the present invention binds the intelligent algorithm with the inference engine, binds it with the neural network computing chip kernel, and provides application modes such as multi-core multi-algorithm simultaneous calculation and multi-core single-algorithm parallel calculation, supports applications in multiple scenarios, and ensures real-time inference and solution during flight. Brief Description of the Drawings

[0031] Figure 1 It is the architecture diagram of the online inference system for the airborne intelligent auxiliary decision-making algorithm;

[0032] Figure 2 It is the flowchart of the online inference method for the airborne intelligent auxiliary decision-making algorithm. Detailed Embodiment

[0033] The present invention belongs to the field of flight tests, and specifically relates to a design method for an airborne intelligent auxiliary flight principle prototype, which is used for the operation of an intelligent auxiliary flight decision-making algorithm, supports the loading and online inference of intelligent algorithms, and performs real-time data interaction with the airborne flight control system and the test system. Through the design technology of the airborne intelligent auxiliary flight principle prototype, the loading and online inference of the neural network model and the intelligent auxiliary flight decision-making algorithm are realized, so as to support the smooth development of intelligent auxiliary enhanced flight tests.

[0034] Such as Figure 1 , an online inference system for an airborne intelligent auxiliary decision-making algorithm according to an embodiment of the present invention includes: an airborne test system, an airborne intelligent auxiliary enhanced principle prototype, and an airborne display and control system. The airborne intelligent auxiliary enhanced principle prototype includes an NPU and a CPU, and the NPU and the CPU perform data communication through a backplane bus; the airborne intelligent auxiliary enhanced principle prototype communicates with the airborne test system through Ethernet, and the airborne intelligent auxiliary enhanced principle prototype communicates with the airborne display and control system through an ARINC429 bus.

[0035] System Architecture Design:

[0036] The network port of the principle prototype receives the aircraft and system status data sent by the airborne test system, and after being processed by the CPU board, it is transmitted to the NPU board. The NPU board calls the intelligent auxiliary flight decision-making algorithm for inference calculation, and the calculation result is transmitted to the CPU board in real time. The CPU board calls other bus ports to send it to the airborne display and control system and the airborne test system, providing real-time intelligent auxiliary flight decision-making information for the pilot.

[0037] Principle prototype architecture design:

[0038] The principle prototype architecture design includes two parts: offline compilation and online reasoning.

[0039] Offline compilation mainly converts the algorithm configuration file trained on the PC into a format recognizable by the NPU through the compiler; online inference is to associate the input data and intelligent algorithm with the NPU to complete the hardware acceleration of the neural network algorithm, and the calculation result is then post-processed by the CPU to obtain the final calculation result of the original machine prototype.

[0040] The input data types supported by NPU include FP32, INT32, INT16, INT8, and UINT8; the supported neural network framework types include pytorch, caffe, darknet, Keras, etc.

[0041] Data format conversion technology of airborne intelligent auxiliary enhancement principle prototype:

[0042] The data format conversion technology converts the FP32 floating point format of the neural network training framework into the FP16 floating point or INT8 integer format used by the prototype NPU for inference calculation. Among several data format conversions, since the data conversion from FP32 to INT8 often brings a large loss of precision, it is necessary to classify the input data of the algorithm to ensure that the calculation result has a small deviation.

[0043] Reasoning operation management technology of airborne intelligent auxiliary enhanced principle prototype:

[0044] The inference operation management technology includes multi-core parallel acceleration of a single algorithm, multi-core multi-algorithm, etc. For the single recognition algorithm part of intelligent assisted flight decision, multi-core parallel acceleration of a single algorithm is used for reasoning to ensure the reliability and real-time performance of recognition; for multiple correction algorithm parts of intelligent assisted flight decision, multi-core multi-algorithm is used for reasoning to realize the function of multiple correction algorithms to independently perform real-time reasoning, and finally meet the real-time online reasoning of intelligent assisted flight decision algorithms in multiple scenarios. This embodiment takes the airborne intelligent assisted flight principle prototype in the civil aircraft intelligent flight assistance enhancement technology project as the research object, and the core processing module is the NPU module. The workflow of the airborne intelligent assisted flight principle prototype includes: pre-processing of input data, format conversion of input data, NPU online reasoning, format conversion of output data, post-processing and data transmission between airborne systems.

[0045] like Figure 2 The online reasoning method of an airborne intelligent auxiliary decision-making algorithm of the present invention is performed by means of an online reasoning system of the airborne intelligent auxiliary decision-making algorithm, and the method comprises:

[0046] Step 1: Preprocess the input data to obtain the data for the recognition algorithm and the data for the pull-out algorithm;

[0047] Specifically: To obtain the data for the recognition algorithm, first, the CPU receives data in real-time from the airborne test system via Ethernet. As Figure 1 shown, it receives 36 floating-point data each time and stores them as a one-dimensional array with a length of 144 bytes; packetize the data according to the requirements of the recognition algorithm in intelligent assisted flight decision-making, that is, continuously receive Ethernet data 50 times, and combine 50 one-dimensional arrays in sequence to form a two-dimensional array with 144 columns and 50 rows. The first row stores the data received first, and the 50th row stores the data received last; then normalize all the data in the two-dimensional array, that is, map the floating-point data with an absolute value > 1 to data ≤ 1 according to the data range to ensure that the intelligent assisted flight decision-making algorithm can converge; finally, the CPU transfers the normalized data to the data storage area of the NPU via the backplane bus.

[0048] After continuously receiving Ethernet data 5 times, the data storage area of the NPU is updated once. According to the FIFO principle, discard the data in the first to fifth rows of the previous two-dimensional array, shift the data in the sixth to 50th rows to the first to 45th rows, and at the same time supplement the 5 frames of received data to the 46th to 50th rows of the two-dimensional array to form a new data block to drive the real-time solution of the intelligent recognition algorithm.

[0049] To obtain the data for the pull-out algorithm, first, the CPU receives data in real-time from the airborne test system via Ethernet, receives 36 floating-point data each time and stores them as a one-dimensional array with a length of 144 bytes; then normalize all the data in the one-dimensional array, and finally the CPU transfers the normalized data to the data storage area of the NPU via the backplane bus; secondly, receive the output result UINT8 data of the recognition algorithm and jointly form the input data of the pull-out algorithm with the data storage area of the NPU.

[0050] The data storage area of the NPU supports multiple data types, including FP32, INT32, INT16, INT8, UINT8.

[0051] Step 2: Perform data format conversion on the data in the storage area of the NPU to obtain the data after format conversion;

[0052] Specifically: The data in the storage area of the NPU are all 32-bit floating-point data. Since the airborne-level NPU chip used for inference calculation uses two formats, 16-bit floating-point data or INT8 integers, it is necessary to perform data precision reduction processing. According to the intelligent auxiliary flight decision-making algorithm, the data in the NPU storage area are divided into higher-precision data and lower-precision data. The higher-precision data include three-axis angular rate, three-axis attitude angle, and three-axis overload, while the lower-precision data include speed, altitude, and vertical speed. For the higher-precision data, they are converted into FP16 format data, and for the lower-precision data, they are converted into INT8 format data. Using the classified data format conversion can not only ensure that this loss basically has no impact on the calculation results, but also improve the calculation efficiency of the intelligent auxiliary flight decision-making algorithm and ensure real-time performance.

[0053] Step 3: Use the data after format conversion and the intelligent auxiliary flight decision-making algorithm for inference to obtain the algorithm inference result of the NPU;

[0054] Specifically: As Figure 2 shown, the inputs required for inference include the data after format conversion and the intelligent auxiliary flight decision-making algorithm. Among them, the data after format conversion are divided into a two-dimensional array required by the recognition algorithm and a one-dimensional array required by the breakout algorithm; the intelligent auxiliary flight decision-making algorithm is a neural network algorithm pre-trained on a PC, including a recognition algorithm (LSTM), a breakout algorithm 1 (PPO), and a breakout algorithm 2 (PPO).

[0055] The inference process uses two algorithm modes supported by the NPU. Among them, for the recognition algorithm (LSTM), a multi-core parallel acceleration single algorithm mode is adopted, and for the breakout algorithm (PPO), a multi-core multi-algorithm acceleration mode is adopted. The recognition algorithm and the two-dimensional array after format conversion are simultaneously sent into NPU CORE0 and CORE1. The two cores of the NPU work in parallel and simultaneously solve to obtain the output result of the recognition algorithm, that is, the recognition scene flag; the breakout algorithm 1 (PPO), the one-dimensional array after format conversion, and the output result of the recognition algorithm are sent to NPU CORE2, and the breakout algorithm 2 (PPO), the one-dimensional array after format conversion, and the output result of the recognition algorithm are sent to NPU CORE3. The NPU selects to use either the breakout algorithm 1 or the breakout algorithm 2 for solution according to the recognition algorithm result, and finally obtains the breakout algorithm result, that is, the rudder surface deflection instruction that the aircraft will execute.

[0056] Step 4: Send the algorithm inference result of the NPU to the CPU for format conversion and post-processing to obtain the final auxiliary decision result of the principle prototype;

[0057] Specifically: As Figure 2As shown, the algorithm inference results of the NPU are sent to the data storage area of the CPU through the backplane bus. The inference results are divided into two types: recognition results and ejection instructions. For the inference results, it is necessary to convert the percentage statistical results of each scenario into UINT8 type data, that is, set the scenario flag corresponding to the maximum percentage to 1, and set the flags of other scenarios to 0 as the final recognition result; while the ejection instructions should be sorted according to the control surface type and throttle control, which are the elevator deflection instruction, aileron deflection instruction, rudder deflection instruction, and throttle lever instruction respectively.

[0058] Step Five: Send the final auxiliary decision-making results of the principle prototype to the airborne test system and the airborne display and control system to obtain the visual intelligent auxiliary flight decision-making results.

[0059] Specifically, as Figure 1 shown, the intelligent auxiliary flight decision-making results are sent to the airborne test system in real time through Ethernet for data recording and monitoring; at the same time, the intelligent auxiliary flight decision-making results are sent to the airborne display and control system in real time through the ARINC429 bus to assist the pilot in making recognition and ejection operations in dangerous scenarios.

[0060] Through the design technology of the airborne intelligent auxiliary flight principle prototype, the present invention realizes the online loading and inference of the intelligent auxiliary flight decision-making algorithm, binds the intelligent algorithm with the inference engine, binds it with the neural network computing chip kernel, and provides multiple application modes, meeting the normal calculation and processing of the intelligent auxiliary flight decision-making algorithm in multiple scenarios of flight tests.

Claims

1. An online reasoning system for an airborne intelligent decision-making aid algorithm, characterized in that: include: Airborne test system, airborne intelligent assisted enhancement principle prototype, and airborne display and control system. The airborne intelligent assisted enhancement principle prototype includes NPU and CPU, and NPU and CPU communicate data through the backplane bus; the airborne intelligent assisted enhancement principle prototype communicates with the airborne test system through Ethernet, and the airborne intelligent assisted enhancement principle prototype communicates with the airborne display and control system through ARINC429 bus.

2. An online reasoning method for an airborne intelligent auxiliary decision-making algorithm, characterized in that: The method is performed by means of an online reasoning system of the onboard intelligent auxiliary decision-making algorithm according to claim 1, and the method comprises: Step 1: Pre-process the input data to obtain the input data of the recognition algorithm and the input data of the modification algorithm, and store them in the storage area of ​​the NPU; Step 2: Convert the data format of the storage area data of the NPU to obtain the data after format conversion; Step 3: Use the format-converted data and the intelligent assisted flight decision algorithm to perform reasoning and obtain the algorithm reasoning result of the NPU; Step 4: Send the algorithm inference results of the NPU to the CPU for format conversion and post-processing to obtain the final auxiliary decision-making results of the prototype; Step 5: The final auxiliary decision results of the prototype are sent to the airborne test system and the airborne display and control system to obtain visualized intelligent auxiliary flight decision results.

3. The online reasoning method of the airborne intelligent auxiliary decision-making algorithm according to claim 2 is characterized in that: In step 1, the input data of the recognition algorithm is obtained, including: First, the CPU receives data from the airborne test system via Ethernet in real time, receiving 36 floating-point data each time and storing them into a one-dimensional array with a length of 144 bytes; the data is packaged according to the recognition algorithm requirements in the intelligent assisted flight decision, that is, 50 Ethernet data are received continuously, and the 50 one-dimensional arrays are combined in order into a two-dimensional array with 144 columns and 50 rows, with the first row storing the first received data and the 50th row storing the last received data; all the data in the two-dimensional array are normalized, that is, the floating-point data with an absolute value greater than 1 is mapped to data ≤1 according to the data range; finally, the CPU transmits the normalized data to the data storage area of ​​the NPU through the backplane bus; After receiving Ethernet data five times in succession, the data storage area of ​​the NPU is updated once: according to the FIFO principle, the data in the previous 1st to 5th rows of the two-dimensional array are discarded, and the data in rows 6 to 50 are moved to rows 1 to 45. At the same time, the received 5 frames of data are added to rows 46 to 50 of the two-dimensional array to form a new data block, which is used to drive the intelligent recognition algorithm to solve in real time.

4. The online reasoning method of the airborne intelligent auxiliary decision-making algorithm according to claim 2 is characterized in that: In step 1, the input data of the algorithm is obtained as follows: First, the CPU receives data from the airborne test system in real time via Ethernet, receiving 36 floating-point data each time and storing them into a one-dimensional array with a length of 144 bytes; then normalizes all the data in the one-dimensional array, and finally the CPU transmits the normalized data to the data storage area of ​​the NPU via the backplane bus; Secondly, the output result UINT8 data of the recognition algorithm is received, and together with the data storage area of ​​the NPU, it constitutes the input data of the algorithm.

5. The online reasoning method of the airborne intelligent auxiliary decision-making algorithm according to claim 2 is characterized in that: Step 2: The data in the storage area of ​​NPU are all 32-bit floating-point data. According to the intelligent assisted flight decision algorithm, the data in the storage area of ​​NPU are divided into higher-precision data and lower-precision data. The higher-precision data include three-axis angular rate, three-axis attitude angle, and three-axis overload. The lower-precision data include speed, altitude, and ascent and descent speed. The higher-precision data are converted into FP16 format data, and the lower-precision data are converted into INT8 format data.

6. The online reasoning method of the airborne intelligent auxiliary decision algorithm according to claim 2 is characterized in that: Step three, specifically: The recognition algorithm LSTM adopts dual-core parallel acceleration of a single algorithm mode, and the modified algorithm PPO adopts dual-core dual-algorithm acceleration mode. The two-bit data after format conversion is input into CORE0 and CORE1 at the same time, and processed by the recognition algorithm LSTM. The output results of the recognition algorithm are input into CORE2 and CORE3 respectively. The one-dimensional array after format conversion is input into CORE2 and CORE3 at the same time. NPU chooses to use the recovery algorithm 1 or the recovery algorithm 2 for solving according to the recognition algorithm result, and obtains the recovery algorithm output result as the control surface deflection instruction to be executed by the aircraft.

7. The online reasoning method of the airborne intelligent auxiliary decision-making algorithm according to claim 2 is characterized in that: Step 4: Specifically: the algorithm inference results of the NPU are sent to the data storage area of ​​the CPU through the backplane bus, where the inference results are divided into recognition results and recovery instructions. For the inference results, the percentage statistical results of each scene need to be converted into UINT8 type data, that is, the scene mark position corresponding to the maximum percentage is 1, and the other scene mark positions are 0, as the final recognition result; and the recovery instructions should be sorted according to the control surface type and throttle control, namely elevator deflection instruction, aileron deflection instruction, rudder deflection instruction and throttle stick instruction.

8. The online reasoning method of the airborne intelligent auxiliary decision-making algorithm according to claim 2 is characterized in that: Step 5: Specifically: the results of intelligent assisted flight decisions are sent in real time to the airborne test system via Ethernet for data recording and monitoring; at the same time, the results of intelligent assisted flight decisions are sent in real time to the airborne display and control system via the ARINC429 bus to assist pilots in identifying and recovering from dangerous scenarios.

Images