Data management apparatus and method based on in-memory chain descriptive file system

Abstract

The application discloses a data management device and method based on a memory chain descriptive file system, wherein the whole file system computing, searching and other operations are directly completed in the memory, the process of CPU informing key information of FPGA is parallel with FPGA data moving, the whole time cost is small, the limit of the data amount that can be processed by FPGA is greatly improved, and the performance of FPGA can be fully exerted. The application solves the problem that the existing data storage method based on FPGA data collection and storage to storage medium cannot fully exert the high performance of FPGA.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a data management device and method based on a memory-linked descriptive file system. Background Technology

[0002] With the continuous development of computer storage technology, specific fields such as communications, military, and national defense have increasingly higher requirements for the recording and storage of specific source data, leading to the emergence of various storage solutions. In these specific application areas, the introduction of various high-speed transmission buses has resulted in a rapid increase in the amount of data transmitted on these buses, placing even higher demands on storage efficiency.

[0003] To improve data acquisition efficiency, FPGAs (Field-Programmable Gate Arrays) are typically used to efficiently acquire data from high-speed transmission buses. When the FPGA stores data onto a storage medium, it inevitably needs to store data for a specific period into a file. This file requires file system support, which is usually managed and accessed by the CPU. The storage device driver writes valid data to a specific location on the storage medium so that correct data can be retrieved during subsequent data playback. See the attached diagram for details. Figure 1 As shown, the process from FPGA data acquisition to transmission to the host computer (CPU control end), and then from the host computer to the storage medium, involves at least three critical data transfers, more than two driver processes on the CPU control end, and layer-by-layer calculations by the file system on the operating system end. Each step, including calculations, delays, and data transfers, affects the final data transmission efficiency, often preventing the FPGA from reaching its full potential. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, a data management device and method based on a memory-linked descriptive file system are provided to solve the problem that existing data storage methods based on FPGA data acquisition and storage on storage media cannot fully utilize the high performance of FPGAs.

[0005] To achieve the above objectives, a data management device based on a memory-linked descriptive file system is provided, comprising:

[0006] The storage medium, wherein the storage space of the storage medium is divided into multiple storage blocks;

[0007] The FPGA used for acquiring input data and playing back output data is connected to the storage medium;

[0008] The memory connected to the storage medium has a memory controller including an initialization module, a control module, a configuration module, and a release module. The initialization module generates a storage medium mapping information free list and a file list. The file list includes multiple file information items, each including file key information and a data storage mapping list. The data storage mapping list includes at least one storage medium mapping item, which describes a uniquely corresponding storage block. The storage medium mapping information free list includes storage medium mapping information corresponding to the storage block where the input data is to be written. The configuration module obtains the file key information of the input data, retrieves the storage medium mapping information from the storage medium mapping information free list, generates file information along with the file key information, and adds it to the file list. The release module deletes the file information corresponding to the file key information to be deleted, so that the storage medium mapping information in the file information corresponding to the file key information to be deleted returns to the storage medium mapping information free list.

[0009] The CPU is used to receive instructions and control the FPGA to read or store data in the storage blocks on the storage medium corresponding to the storage medium mapping information fed back by the control module, based on the storage medium mapping information fed back by the control module. The CPU is connected to the memory and the FPGA.

[0010] Furthermore, the storage medium is a hard disk.

[0011] Furthermore, the file information also includes a file number and a file name.

[0012] This invention provides a data management method for a data management device based on a memory-linked descriptive file system, comprising the following steps:

[0013] When the storage medium is formatted or accessed for the first time, the initialization module generates a free list and a file list for storage medium mapping information.

[0014] Upon receiving a data storage instruction, the CPU generates a storage signal based on the key file information carried in the data storage instruction and sends it out.

[0015] The control module receives the storage signal, the control configuration module obtains the file key information of the storage signal, obtains the storage medium mapping information from the storage medium mapping information free list, generates file information together with the file key information in the storage signal, and adds it to the file list;

[0016] The control module obtains the storage medium mapping information from the newly added file information in the file chain and sends it outwards;

[0017] The CPU obtains the storage medium mapping information from the newly added file information in the file list, controls the FPGA to collect input data and directly writes it into the storage block corresponding to the storage medium mapping information in the newly added file information in the file list;

[0018] Upon receiving a data playback instruction, the CPU obtains key information about the file to be played back and sends it out.

[0019] The control module obtains key information of the file to be replayed, traverses the file linked list to obtain the data storage mapping linked list in the file information to be replayed, and sends it out.

[0020] The CPU obtains the data storage mapping list based on the file information to be played back, controls the FPGA to read the output data in the storage block corresponding to the storage medium mapping information in the data storage mapping list of the file information to be played back, and outputs it.

[0021] When a data deletion instruction is received, the CPU obtains key information about the file to be deleted;

[0022] The control module obtains key information of the file to be deleted and traverses the file list to obtain the file information corresponding to the key information of the file to be deleted;

[0023] The release module deletes the file information corresponding to the key information of the file to be deleted, so that the storage medium mapping information in the file information corresponding to the key information of the file to be deleted returns to the storage medium mapping information free list.

[0024] The beneficial effects of the present invention are that the data management device based on the memory chain descriptive file system of the present invention completes the calculation, search and other operations of the entire file system directly in memory. In addition, the process of the CPU notifying the FPGA of key information and the parallel processing of FPGA data transfer are combined, the overall time overhead is small and can even be ignored, which greatly improves the limit of the amount of data that the FPGA can process and can also give full play to the performance of the FPGA. Attached Figure Description

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0026] Figure 1 This is a schematic diagram of the current product data flow and functions.

[0027] Figure 2 This is a schematic diagram of the data management device based on a memory-linked descriptive file system according to the present invention.

[0028] Figure 3This is a schematic diagram of the data flow and functions of the data management device based on a memory-linked descriptive file system according to the present invention.

[0029] Figure 4 This is a schematic diagram of the memory-linked descriptive file system of the present invention.

[0030] Figure 5 This is a schematic diagram of the free linked list mapping of the mapping information of the present invention. Detailed Implementation

[0031] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0032] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] Reference Figure 2 As shown, the present invention provides a data management device based on a memory-chain descriptive file system, including: a storage medium 1, an FPGA 2, memory, and a CPU 4.

[0034] Storage medium 1 is used to store data. In some embodiments, storage medium 1 is a hard disk.

[0035] In this embodiment, combined with Figure 4 and Figure 5 The storage space of storage medium 1 is divided into multiple storage blocks (block 0, block m+2).

[0036] FPGA2 is connected to storage medium 1. FPGA2 is used to acquire input data and play back output data. FPGA (Field Programmable Gate Array) is a field-programmable logic gate array.

[0037] The memory is connected to storage medium 1. The memory controller includes an initialization module 32, a control module 31, a configuration module 33, and a release module 34.

[0038] Specifically, the initialization module 32 is used to generate a free list and a file list for storage medium mapping information.

[0039] The file linked list contains information about multiple files. This file information includes key file information and a data storage mapping linked list. The data storage mapping linked list includes at least one storage medium mapping. This storage medium mapping information describes a uniquely corresponding storage block. The storage medium mapping information free linked list contains the storage medium mapping information corresponding to the storage block where input data is to be written.

[0040] The configuration module 33 is used to obtain key file information of the input data, obtain storage medium mapping information from the storage medium mapping information free list, generate file information together with key file information, and add it to the file list.

[0041] The release module 34 is used to delete the file information corresponding to the key information of the file to be deleted so that the storage medium mapping information in the file information corresponding to the key information of the file to be deleted is returned to the storage medium mapping information free list.

[0042] CPU (Central Processing Unit) is the central processing unit. CPU4 is connected to the memory and FPGA2. CPU4 is used to receive user instructions and control FPGA2 to read or store data on the storage block of storage medium 1 corresponding to the storage medium mapping information fed back by control module 31 based on the storage medium mapping information fed back by control module 31.

[0043] This invention provides a data management method for a data management device based on a memory-linked descriptive file system, comprising the following steps:

[0044] S1: When storage medium 1 is formatted or accessed for the first time, initialization module 32 generates a storage medium mapping information free list and file list.

[0045] S2: Upon receiving a user's data storage instruction, CPU4 generates a storage signal based on the key file information carried in the data storage instruction and sends it out.

[0046] S3: The control module 31 receives the storage signal, the control configuration module 33 obtains the file key information of the storage signal, obtains the storage medium mapping information from the storage medium mapping information free list, generates file information together with the file key information in the storage signal, and adds it to the file list.

[0047] S4: Control module 31 obtains the storage medium mapping information from the information of newly added files in the file chain and sends it out.

[0048] S5: CPU4 obtains the storage medium mapping information from the newly added file information in the file chain, controls FPGA2 to collect input data and directly writes it into the storage block corresponding to the storage medium mapping information of the newly added file information in the file chain.

[0049] When a file needs to add new data, a storage medium mapping information is obtained from the set of free storage medium mapping information and added to the data storage mapping chain. The FPGA is then notified to write the corresponding data directly into the storage medium logic block described by the storage medium mapping information.

[0050] S6: When a data playback instruction is received, CPU4 obtains the key information of the file to be played back and sends it out.

[0051] S7: Control module 31 obtains key information of the file to be replayed, traverses the file linked list to obtain the data storage mapping linked list in the file information to be replayed, and sends it out.

[0052] S8: CPU4 obtains the data storage mapping list based on the file information to be replayed, controls FPGA2 to read the output data in the storage block corresponding to the storage medium mapping information in the data storage mapping list of the file information to be replayed, and outputs it.

[0053] When a file needs to be replayed for post-analysis, the data playback function can be completed by traversing the file's data storage mapping linked list and notifying the FPGA to perform data reading operations in sequence.

[0054] S9: When a data deletion instruction is received, CPU4 obtains key information about the file to be deleted.

[0055] S10: Control module 31 obtains key information of the file to be deleted and traverses the file linked list to obtain the file information corresponding to the key information of the file to be deleted.

[0056] S11: Release module 34 deletes the file information corresponding to the key information of the file to be deleted so that the storage medium mapping information in the file information corresponding to the key information of the file to be deleted returns to the storage medium mapping information free list.

[0057] When a file needs to be deleted, the data storage mapping list for that file is traversed, and the corresponding storage medium mapping information is placed back into the set of free storage medium mapping information. Then, other relevant information about the file is removed from the file list and the associated resources are released.

[0058] See Figure 2To improve transmission efficiency, the data management device and method based on a memory-linked descriptive file system of the present invention bypasses numerous intermediate steps such as CPU drivers, multiple data transfers, and file system overhead, directly performing data acquisition or playback along the shortest path. Specifically, the data management device and method based on a memory-linked descriptive file system of the present invention achieves simplicity and efficiency by employing a memory-based descriptive file system. The CPU instructs the FPGA on the storage location and organizes and maintains file information. By mapping the storage medium based on a small amount of control information in memory, the device and method achieve the goal of controlling the FPGA to perform efficient data storage or playback.

[0059] See Figure 4 As shown, the data management device and method based on a memory-linked descriptive file system of the present invention uses storage medium mapping information of each data structure to describe a specific block area on a uniquely corresponding storage medium. The set of all storage medium mapping information represents the entire storage space of the storage medium. For each file, in addition to file-related information, the set of storage medium mapping information on the data storage mapping linked list stores all block areas on the storage medium used by that file. The data structures of all file information form a linked list of file sets to represent the top-level structure of the file system.

[0060] See Figure 5 As shown, when the storage medium is formatted or accessed for the first time, the data management device and method of the present invention based on the memory chain descriptive file system initializes a set of free storage medium mapping information. This set of free storage medium mapping information is mapped one-to-one with all logical blocks (storage blocks) of the storage medium.

[0061] The data management device and method based on the memory-chain descriptive file system of the present invention completes the calculation, search and other operations of the entire file system directly in memory. In addition to the process of notifying the FPGA of key information and the parallel processing of FPGA data transfer, the overall time overhead is small and can even be ignored. This can greatly improve the limit of the amount of data that the FPGA can process and give full play to the high-efficiency data transmission performance of the FPGA.

[0062] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A data management device based on a memory-linked descriptive file system, characterized in that, include: The storage medium is a hard disk, and the storage space of the storage medium is divided into multiple storage blocks; The FPGA used for acquiring input data and playing back output data is connected to the storage medium to form a data transmission channel independent of the CPU. The memory connected to the storage medium has a memory controller including an initialization module, a control module, a configuration module, and a release module. The initialization module generates a storage medium mapping information free list and a file list. The file list includes multiple file information items, each including file key information and a data storage mapping list. The data storage mapping list includes at least one storage medium mapping item, which describes a uniquely corresponding storage block. The storage medium mapping information free list includes storage medium mapping information corresponding to the storage block where the input data is to be written. The configuration module obtains the file key information of the input data, retrieves the storage medium mapping information from the storage medium mapping information free list, generates file information along with the file key information, and adds it to the file list. The release module deletes the file information corresponding to the file key information to be deleted, so that the storage medium mapping information in the file information corresponding to the file key information to be deleted returns to the storage medium mapping information free list. Each data structure is used to store medium mapping information to describe a specific block area on the unique corresponding storage medium; The collection of all storage medium mapping information, that is, the storage space represented by the entire storage medium; The CPU is used to receive instructions and control the FPGA to read or store data in the storage blocks on the storage medium corresponding to the storage medium mapping information fed back by the control module, based on the storage medium mapping information fed back by the control module. The CPU is connected to the memory and the FPGA.

2. The data management device based on a memory-linked descriptive file system according to claim 1, characterized in that, The file information also includes the file number and file name.

3. A data management method for a data management device based on a memory-linked descriptive file system as described in claim 1 or 2, characterized in that, Includes the following steps: When the storage medium is formatted or accessed for the first time, the initialization module generates a free list and a file list for storage medium mapping information. Upon receiving a data storage instruction, the CPU generates a storage signal based on the key file information carried in the data storage instruction and sends it out. The control module receives the storage signal, the control configuration module obtains the file key information of the storage signal, obtains the storage medium mapping information from the storage medium mapping information free list, generates file information together with the file key information in the storage signal, and adds it to the file list; The control module obtains the storage medium mapping information from the newly added file information in the file chain and sends it outwards; The CPU obtains the storage medium mapping information from the newly added file information in the file list, controls the FPGA to collect input data and directly writes it into the storage block corresponding to the storage medium mapping information in the newly added file information in the file list; Upon receiving a data playback instruction, the CPU obtains key information about the file to be played back and sends it out. The control module obtains key information of the file to be replayed, traverses the file linked list to obtain the data storage mapping linked list in the file information to be replayed, and sends it out. The CPU obtains the data storage mapping list based on the file information to be played back, controls the FPGA to read the output data in the storage block corresponding to the storage medium mapping information in the data storage mapping list of the file information to be played back, and outputs it. When a data deletion instruction is received, the CPU obtains key information about the file to be deleted; The control module obtains key information of the file to be deleted and traverses the file list to obtain the file information corresponding to the key information of the file to be deleted; The release module deletes the file information corresponding to the key information of the file to be deleted, so that the storage medium mapping information in the file information corresponding to the key information of the file to be deleted returns to the storage medium mapping information free list.

Images