Method and system for storing airborne equipment data based on machine learning

Abstract

The present invention provides an airborne equipment data storage method based on machine learning, comprising the following steps: step S10, data preprocessing, processing the data to be stored and hardware resources to generate a label data group; step S20, storage data processing, including an input layer, multiple hidden layers and an output layer; multiple hidden layers, extracting statistical features in the data through supervised data layer-by-layer training and nonlinear transformation, and establishing a mapping relationship from bottom-level signals to high-level data storage methods; output layer, using the output result of the last hidden layer as the input of the model of the output layer, and finally outputting data Y; step S30, data storage, performing airborne equipment data storage according to the method in Y. Compared with the related art, the airborne equipment data storage method based on machine learning provided by the present invention can realize efficient storage of airborne equipment data. The present invention also provides an airborne equipment data storage system based on machine learning.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and data storage technology, and in particular to an airborne equipment data storage method and system based on machine learning. Background Art

[0002] The data rate of airborne equipment is high, which can reach hundreds of Mbps or even Gbps in many applications. Such a high data rate is far higher than the current wireless data transmission rate between air and ground, so airborne equipment requires an onboard data storage system in many applications.

[0003] NAND Flash has many excellent features such as small size, light weight, low power consumption, high storage density, fast data access speed, good seismic resistance, etc., and is widely used in data storage fields with high performance requirements such as aerospace. However, solid-state storage data systems based on this type of media generally have a difficult problem that affects high-speed storage: file management time is too long.

[0004] like Figure 1 As shown, Figure 1 The structure of the existing data storage file system shown mainly includes the FAT table, FDT table and data area, which are used to establish the cluster link relationship in the storage medium, describe the file directory information, and actually store data. The FAT table uses a 4B space to describe the link relationship of a cluster, and the FDT table uses a 32B space to describe the information of a file. The cluster in the data area is the basic unit of file management, which is generally dozens of sectors in size, and each sector is fixed to 512B. Each storage of a cluster of data includes data write operations in three areas, namely the FAT table, the FDT table and the data area. The data storage time is the data update time of these three areas.

[0005] At the same time, the update time of the three areas in the data storage is shown in Table 1:

[0006] Table 1 Data update time of each area in traditional file system

[0007]

[0008]

[0009] Among them, C is the number of sectors contained in a cluster, which is usually dozens, T w is the average writing time of 1B data, T w-res is the address jump response time. T w-res Generally, it takes a long time, which is related to the working mechanism of NAND Flash, because when writing data in the flash memory, the original data in the area must be erased first. As can be seen from the table above, 4T w-resThe address jump response time caused by file management is even longer than the actual writing time of each cluster of data, 512·C·T w , which will significantly reduce the data storage rate. The time expenditure of these file management operations cannot be ignored relative to the data writing time, and usually greatly reduces the data writing rate.

[0010] Therefore, it is necessary to provide an airborne equipment data storage method and system based on machine learning that can significantly reduce file management time and use minimal hardware resources, thereby achieving efficient storage of airborne equipment data to solve the above technical problems. Summary of the invention

[0011] In view of the shortcomings of the prior art, the object of the present invention is to provide an airborne equipment data storage method and system based on machine learning, which can not only significantly reduce file management time but also use minimal hardware resources, thereby realizing efficient storage of airborne equipment data.

[0012] The technical solution adopted by the present invention to solve the technical problem is:

[0013] A method for storing airborne equipment data based on machine learning, comprising the following steps:

[0014] Step S10, data preprocessing, processing the data to be stored and the hardware resources to generate a label data group, wherein the label data group includes a plurality of components, each component being a vector signal representing a type of data;

[0015] Step S20, storage data processing, including an input layer, multiple hidden layers and an output layer;

[0016] The input layer, the signal of the input layer is the label data group generated after the preprocessing in step S10, denoted as X;

[0017] The plurality of hidden layers are trained layer by layer with supervised data and nonlinear transformation to extract statistical features in the data, thereby realizing the continuous evolution of the internal pattern, establishing a mapping relationship from the bottom layer signal to the high-level data storage method, and completing the network initialization;

[0018] The output layer uses the output result of the last hidden layer as the input of the model of the output layer, and finally outputs data Y, where Y includes a FAT table query method, a FAT table update method, an FDT table query method, an FDT table update method, and a data area write method. The output layer integrates the network parameters of all the hidden layers and the output layer model into the parameters of a network learning model, and initializes the entire network parameters;

[0019] Step S30, data storage: after using the file management method set Y obtained in step S20, perform airborne equipment data storage according to the method in Y.

[0020] Preferably, in step S20, f l ,l∈N l represents the activation function vector of each hidden layer, N l represents the number of hidden layers, W l and b l Represent the weight vector and bias vector of the lth layer respectively,

[0021] The transformation steps of the label data group in the multiple hidden layers are specifically as follows:

[0022] Step S21: the input layer inputs data X, trains the network parameters of the first hidden layer, and activates the parameters as the output X1 of the first hidden layer, denoted as: X 1 =f 1 (W 1 X+b 1 );

[0023] Step S22: Use the output of step S21 as the input of the second hidden layer to train the network parameters of the second hidden layer, and activate the parameters as the output X2 of the second hidden layer, which is recorded as:

[0024] X 2 =f 2 (W 2 X+b 2 )

[0025] =f 2 (W 2 (f 1 (W 1 X+b 1 ))+b 2 );

[0026] Step S23, repeat the training until the outputs of all the hidden layers are trained, and the output of the last hidden layer is recorded as:

[0027] X l-1 =f l-1 (W l-1 (f l-2 (W 1-3 Lf 1 (W 1 X+b 1 ))+b l-2 )+b l-1 );

[0028] Step S24: Use the output of the last hidden layer as the input of the model of the output layer. The final data output is recorded as:

[0029] Y=f l (W l (f l-1 (W 1-2 Lf 1 (W 1 X+b 1 ))+b l-1 )+b l ).

[0030] Preferably, the method further includes step S25, overall optimization of the network.

[0031] The cost function E(X,Y) of the entire network is solved iteratively through optimization theory. The value of the cost function is the ratio μ of file management time to actual data writing time. By fine-tuning all network parameters, the optimized parameters of the entire network are finally obtained, and the optimized parameters are used for subsequent classification or prediction work.

[0032] Preferably, the tag data group includes nine types of data, namely storage file size, continuous write data packet size, data rate, data type, interface type, data transmission rate, computing resources, cache resources and storage resources.

[0033] A machine learning-based airborne equipment data storage system is used to implement any of the machine learning-based airborne equipment data storage methods described above, including a microprocessor for controlling data storage, a data cache as a cache device within the response time of a storage medium, and a storage medium. The microprocessor receives external data and puts the data into a cache in the data cache device together with internally generated data, performs data preprocessing and machine learning-based storage data processing in sequence, classifies and determines a storage method suitable for each application scenario and data type, and finally reads the data in the cache and writes it to the storage medium according to the storage method determined by the classification.

[0034] Preferably, the microprocessor comprises:

[0035] A data receiving module receives external data to be stored from airborne equipment;

[0036] The data generation module generates processing results and status data waiting to be stored in the processor;

[0037] A medium data storage module for storing data to be stored;

[0038] A cache control module, which temporarily stores the data sent by the data receiving module and the data generating module in the data cache, and reads the data in the data cache and sends it to the medium data storage module when the storage medium allows writing;

[0039] A data preprocessing module generates the labeled data group with labels;

[0040] A storage data processing module processes the tag data group and classifies and determines a data storage method;

[0041] The data storage module writes the data to be stored into the storage medium according to the five data storage methods of FDT table query, FDT table update, FAT table query, FAT table update and data area writing determined by the storage data processing module.

[0042] Preferably, the microprocessor is FPGA or DSP.

[0043] Preferably, the data buffer is DDR3, FIFO or RAM, and the storage medium is NAND Flash, CF card, SD card or solid state drive.

[0044] In summary, compared with the prior art, the machine learning-based airborne equipment data storage method and system provided by the present invention aims to solve the problem in the prior art that the file management operation time expenditure is relatively long compared to the data writing time, which greatly reduces the data writing rate. Through machine learning of the data to be stored in the airborne equipment, the storage algorithm and mode under each application scenario are determined in combination with the classification of hardware resources, and the file management algorithm parameters are adjusted in real time. Compared with the traditional data storage method, it can not only significantly reduce the file management time, but also use the least hardware resources to achieve efficient storage of airborne equipment data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the structure of a data storage file system in the prior art;

[0046] Figure 2 A flow chart of the method for storing airborne equipment data based on machine learning provided by the present invention;

[0047] Figure 3 A schematic diagram of the structure of the airborne equipment data storage system based on machine learning provided by the present invention. DETAILED DESCRIPTION

[0048] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments. The following experimental examples and embodiments are used to further illustrate but not limit the present invention.

[0049] See also Figure 2The present invention provides an airborne equipment data storage method based on machine learning, comprising the following steps:

[0050] Step S10, data preprocessing, processing the data to be stored and the hardware resources to generate a label data group, wherein the label data group includes a plurality of components, each component being a vector signal representing a type of data;

[0051] Step S20, storage data processing, including an input layer, multiple hidden layers and an output layer;

[0052] The input layer, the signal of the input layer is the label data group generated after the preprocessing in step S10, denoted as X;

[0053] Preferably, in this embodiment, the tag data group includes nine types of data, namely storage file size, continuous write data packet size, data rate, data type, interface type, data transmission rate, computing resources, cache resources and storage resources. Of course, this is not a limitation of the present invention. In other embodiments of the present invention, the tag data group can also change the data type of the tag data group according to actual conditions, all within the protection scope of the present invention.

[0054] The plurality of hidden layers are trained layer by layer with supervised data and nonlinear transformation to extract statistical features in the data, thereby realizing the continuous evolution of the internal pattern, establishing a mapping relationship from the bottom layer signal to the high-level data storage method, and completing the network initialization;

[0055] The output layer uses the output result of the last hidden layer as the input of the model of the output layer, and finally outputs data Y, where Y includes a FAT table query method, a FAT table update method, an FDT table query method, an FDT table update method, and a data area write method. The output layer integrates the network parameters of all the hidden layers and the output layer model into the parameters of a network learning model, and initializes the entire network parameters;

[0056] Specifically, in this step, using f l ,l∈N l represents the activation function vector of each hidden layer, N l represents the number of hidden layers, W l and b l Represent the weight vector and bias vector of the lth layer respectively;

[0057] Then the transformation steps of the label data group in the multiple hidden layers are specifically as follows:

[0058] Step S21: the input layer inputs data X, trains the network parameters of the first hidden layer, and activates the parameters as the output X of the first hidden layer.1 , recorded as: X 1 =f 1 (W 1 X+b 1 );

[0059] Step S22: Use the output of step S21 as the input of the second hidden layer to train the network parameters of the second hidden layer, and activate the parameters as the output X of the second hidden layer. 2 , recorded as:

[0060] X 2 =f 2 (W 2 X+b 2 )

[0061] =f 2 (W 2 (f 1 (W 1 X+b 1 ))+b 2 );

[0062] Step S23, repeat the training until the outputs of all the hidden layers are trained, and the output of the last hidden layer is recorded as:

[0063] X l-1 =f l-1 (W l-1 (f l-2 (W 1-3 Lf 1 (W 1 X+b 1 ))+b l-2 )+b l-1 );

[0064] Step S24: Use the output of the last hidden layer as the input of the model of the output layer. The final data output is recorded as:

[0065] Y=f l (W l (f l-1 (W 1-2 Lf 1 (W 1 X+b 1 ))+b l-1 )+b l );

[0066] Step S25, overall tuning of the network, through the optimization theory iteration to solve the cost function E (X, Y) of the entire network, the value of the cost function is the ratio of file management time to actual data writing time μ, by fine-tuning all network parameters, finally get the optimization parameters of the entire network, use the optimization parameters for subsequent classification or prediction work.

[0067] Step S30, data storage: after using the file management method set Y obtained in step S20, perform airborne equipment data storage according to the method in Y.

[0068] Also, see Figure 3 The present invention also provides an airborne equipment data storage system based on machine learning, which is used to implement the aforementioned airborne equipment data storage method based on machine learning, including a microprocessor for controlling data storage, a data buffer as a cache device within the storage medium response time, and a storage medium. The microprocessor receives external data, puts it into the cache of the data cache device together with the internally generated data, performs data preprocessing and storage data processing based on machine learning in sequence, classifies and determines the storage method suitable for each application scenario and data type, and finally reads the data in the cache and writes it to the storage medium according to the storage method determined by classification.

[0069] Specifically, the microprocessor includes a data receiving module for receiving external data to be stored of airborne equipment; a data generating module for generating processing results and status data to be stored inside the processor; a medium data storage module; a cache control module for temporarily storing the data sent by the data receiving module and the data generating module in the data buffer, and reading the data in the data buffer and sending it to the medium data storage module when the storage medium allows writing; a data preprocessing module for generating the labeled data group; a storage data processing module for processing the labeled data group and classifying and determining the data storage method; a data storage module for writing the data to be stored into the storage medium according to the five data storage methods classified and determined by the storage data processing module, namely, FDT table query, FDT table update, FAT table query, FAT table update, and data area write.

[0070] Preferably, the microprocessor is FPGA or DSP, the data buffer is DDR3, FIFO or RAM, and the storage medium is NAND Flash, CF card, SD card or solid state drive.

[0071] Compared with the prior art, the machine learning-based airborne equipment data storage method and system provided by the present invention aims to solve the problem that the file management operation time expenditure is relatively long compared to the data writing time in the prior art, which greatly reduces the data writing rate. Through machine learning of the data to be stored in the airborne equipment, the storage algorithm and mode under each application scenario are determined in combination with the hardware resource classification, and the file management algorithm parameters are adjusted in real time. Compared with the traditional data storage method, it can significantly reduce the file management time and realize the efficient storage of airborne equipment data with the least hardware resources.

[0072] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for storing airborne equipment data based on machine learning, characterized in that: The steps include: Step S10, data preprocessing, processing the data to be stored and the hardware resources to generate a label data group, wherein the label data group includes a plurality of components, each component being a vector signal representing a type of data; Step S20, storage data processing, including an input layer, multiple hidden layers and an output layer; The input layer, the signal of the input layer is the label data group generated after the preprocessing in step S10, denoted as X; The plurality of hidden layers are trained layer by layer with supervised data and nonlinear transformation to extract statistical features in the data, thereby realizing the continuous evolution of the internal pattern, establishing a mapping relationship from the bottom layer signal to the high-level data storage method, and completing the network initialization; The output layer uses the output result of the last hidden layer as the input of the model of the output layer, and finally outputs data Y, where Y includes a FAT table query method, a FAT table update method, an FDT table query method, an FDT table update method, and a data area write method. The output layer integrates the network parameters of all the hidden layers and the output layer model into the parameters of a network learning model, and initializes the entire network parameters; Step S30, data storage, using the file management method set Y obtained in step S20, execute airborne equipment data storage according to the method in Y, in step S20, using f l , l∈N l represents the activation function vector of each hidden layer, N l represents the number of hidden layers, W l and b l Represent the weight vector and bias vector of the lth layer respectively; Then the transformation steps of the label data group in the multiple hidden layers are specifically as follows: Step S21: the input layer inputs data X, trains the network parameters of the first hidden layer, and activates the parameters as the output X1 of the first hidden layer, denoted as: X1 = f 1 (W 1 X+b 1 ); Step S22: Use the output of step S21 as the input of the second hidden layer to train the network parameters of the second hidden layer, and activate the parameters as the output X2 of the second hidden layer, which is recorded as: X2=f 2 (W 2 X+b 2 ) =f 2 (W 2 (f 1 (W 1 X+b 1 ))+b 2 ); Step S23, repeat the training until the outputs of all the hidden layers are trained, and the output of the last hidden layer is recorded as: X l-1 =f l-1 (W l-1 (f l-2 (W 1-3 L f 1 (W 1 X+b 1 ))+b l-2 )+b l-1 ); Step S24: Use the output of the last hidden layer as the input of the model of the output layer. The final data output is recorded as: Y=f l (W l (f l-1 (W 1-2 L f 1 (W 1 X+b 1 ))+b l-1 )+b l )); Among them, X1, X2, ..., X l-1 , respectively represent the output of the first layer, the second layer, and the last layer; f 1 , f 2 , ..., f l-2 , f l-1 , respectively represent the activation function vectors of the first layer, the second layer, and the last layer; W 1 , ..., W 1-3 , W 1-2 , W l-1 Represent the weight vectors of the first layer, the second layer, and the last layer respectively; b l , b 2 , ..., b l-2 , b l-1 Represent the bias vectors of the first layer, the second layer, and the last layer respectively.

2. The method for storing airborne equipment data based on machine learning according to claim 1, characterized in that: The step S25 is also included, namely, overall tuning of the network: The cost function E(X,Y) of the entire network is solved iteratively through optimization theory. The value of the cost function is the ratio μ of file management time to actual data writing time. By fine-tuning all network parameters, the optimized parameters of the entire network are finally obtained, and the optimized parameters are used for subsequent classification or prediction work.

3. The method for storing airborne equipment data based on machine learning according to claim 1, characterized in that: The tag data group includes nine types of data, namely storage file size, continuous write data packet size, data rate, data type, interface type, data transmission rate, computing resources, cache resources and storage resources.

4. A machine learning-based airborne equipment data storage system, used to implement the machine learning-based airborne equipment data storage method according to any one of claims 1 to 3, characterized in that: It includes a microprocessor for controlling data storage, a data cache as a cache device within the storage medium response time, and a storage medium. The microprocessor receives external data and puts them into the cache of the data cache device together with internally generated data, performs data preprocessing and storage data processing based on machine learning in sequence, classifies and determines the storage method suitable for each application scenario and data type, and finally reads the data in the cache and writes it to the storage medium according to the storage method determined by the classification.

5. The machine learning-based airborne equipment data storage system according to claim 4, characterized in that: The microprocessor comprises: A data receiving module receives external data to be stored from airborne equipment; The data generation module generates processing results and status data waiting to be stored in the processor; A medium data storage module, which stores various data to be stored; A cache control module, which temporarily stores the data sent by the data receiving module and the data generating module in the data cache, and reads the data in the data cache and sends it to the medium data storage module when the storage medium allows writing; A data preprocessing module generates the labeled data group with labels; A storage data processing module processes the tag data group and classifies and determines a data storage method; The data storage module writes the data to be stored into the storage medium according to the five data storage methods of FDT table query, FDT table update, FAT table query, FAT table update and data area writing determined by the storage data processing module.

6. The machine learning-based airborne equipment data storage system according to claim 5, characterized in that: The microprocessor is FPGA or DSP.

7. The machine learning-based airborne equipment data storage system according to claim 6, characterized in that: The data buffer is DDR3, FIFO or RAM, and the storage medium is NAND Flash, CF card, SD card or solid state hard disk.

Images