High-speed data storage method and device

Abstract

The present invention provides a high-speed data storage method and device, and relates to the field of data processing technology. The method formats an SSD according to a general format, and receives a multi-channel data acquisition instruction through a CPU; then, according to the multi-channel data acquisition instruction, a cache area is divided for each channel of data in a DDR memory, and at the same time, multiple storage cluster areas with continuous addresses are allocated for each channel of data in the formatted SSD according to the cache time granularity; then, the multi-channel data is cached in parallel to the cache area through an FPGA module; finally, based on a parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written in parallel to the storage cluster area through the FPGA module. The high-speed data storage method provided by the present invention realizes high-speed storage of data, and can directly realize data viewing, modification, and migration and exchange through the SSD on the device side, so as to manage the stored data more flexibly and conveniently.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a high-speed data storage method and device. Background Art

[0002] In the field of wireless communications, the data storage and processing rate requirements are as high as several GB per second, and the demand for high-bandwidth, multi-channel high-speed data acquisition and storage processing applications is very widespread.

[0003] The existing technology mainly uses FPGA for data acquisition and uses raw data (i.e. raw data) for storage. However, this method has problems such as weak data management function, low efficiency of data migration and export, insufficient multi-channel acquisition support capability, and inability to efficiently store data, resulting in a poor user experience. Summary of the invention

[0004] The present invention provides a high-speed data storage method and device, which are used to solve the technical problems existing in the prior art, such as low high-speed data acquisition efficiency, difficult data management and poor user experience.

[0005] The present invention provides a high-speed data storage method, comprising the following steps:

[0006] Format the SSD according to the exFAT format, and receive data acquisition instructions for multiple channels of data through the CPU;

[0007] Based on the data acquisition instruction, generate the auxiliary information of the multiple channels of data;

[0008] Based on the auxiliary information of the multiple data, a cache time granularity is set, and a cache area is divided for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, a plurality of storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit;

[0009] Cache the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module;

[0010] Based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written into the storage cluster area in parallel according to the cache time granularity through the FPGA module; the NVME instructions are generated based on the principle that data is written at continuous addresses on the SSD.

[0011] According to a high-speed data storage method provided by the present invention, the cache time granularity is set based on the auxiliary information of the multiple data paths, and a cache area is divided for each data path in the DDR memory, including:

[0012] Based on the auxiliary information of the multiple data channels, a cache time granularity is set, and a cache space required for each data channel is calculated;

[0013] Based on the cache space required for each data path, a cache area is divided for each data path in the DDR memory.

[0014] According to a high-speed data storage method provided by the present invention, the ancillary information includes center frequency, bandwidth, sampling rate, sampling start time, sampling end time, sampling position information, acquisition description information and digital intermediate frequency point.

[0015] According to a high-speed data storage method provided by the present invention, the FPGA module caches the multiple data in parallel to the cache area according to the cache time granularity, including:

[0016] Obtaining a buffer address corresponding to each channel of data in the multiple channels of data;

[0017] Based on the cache area address, each path of data is cached in parallel in the cache area corresponding to each path of data according to the cache time granularity.

[0018] According to a high-speed data storage method provided by the present invention, based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written to the storage cluster area in parallel according to the cache time granularity through the FPGA module, including:

[0019] Based on the parallel synchronous ping-pong cache mechanism, each cache area is divided into two cache sub-areas;

[0020] Determining the state of each cache sub-region in each cache region;

[0021] When the state of the cache sub-area is a readable state, based on the NVME instruction, the data in the cache sub-area is written into the storage cluster area in parallel through the FPGA module according to the cache time granularity.

[0022] According to a high-speed data storage method provided by the present invention, the method further comprises:

[0023] When all the data in the buffer area are written into the storage cluster area, the file entry information is updated, or when a data acquisition end instruction is received through the CPU, the file entry information is updated.

[0024] The present invention also provides a high-speed data storage device, comprising the following modules:

[0025] A receiving module is used to format the SSD according to the exFAT format and receive data acquisition instructions for multiple channels of data through the CPU;

[0026] A generating module, used for generating the auxiliary information of the multiple channels of data based on the data acquisition instruction;

[0027] A partition module, configured to set a cache time granularity based on the auxiliary information of the multiple data paths, and to divide a cache area for each data path in the DDR memory, wherein the addresses of the multiple cache areas corresponding to the multiple data paths are continuous; and, based on the auxiliary information of the multiple data paths, to allocate multiple storage cluster areas with continuous addresses according to the cache time granularity for each data path in the formatted SSD; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit;

[0028] A cache module, used for caching the multiple data in parallel to the cache area according to the cache time granularity through an FPGA module;

[0029] The storage module is used to write the data in the cache area into the storage cluster area in parallel according to the cache time granularity through the FPGA module based on the parallel synchronous ping-pong cache mechanism and NVME instructions; the NVME instructions are generated based on the principle that data is written at consecutive addresses on the SSD.

[0030] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the high-speed data storage methods described above is implemented.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the high-speed data storage methods described above.

[0032] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the high-speed data storage method as described above is implemented.

[0033] The invention provides a high-speed data storage method, firstly, the SSD is formatted according to the exFAT format, and a multi-channel data acquisition instruction is received through the CPU; then, according to the multi-channel data acquisition instruction, a cache area is divided for each channel of data in the DDR memory, and at the same time, a plurality of storage cluster areas with continuous addresses are allocated for each channel of data in the formatted SSD according to the cache time granularity; then, the multi-channel data is cached in parallel to the cache area through the FPGA module; finally, based on the parallel synchronous ping-pong cache mechanism and the NVME instruction, the data in the cache area is written in parallel to the storage cluster area through the FPGA module. Among them, the NVME instruction is generated based on the principle that data is written to the SSD at a continuous address, which enables the data corresponding to the acquisition file in the exFAT file format to be written to the SSD at a continuous address, and there is no need to frequently move or address during the data storage process, which improves the data storage speed and realizes efficient data storage. At the same time, due to the versatility of the exFAT file format, the acquisition file corresponding to the stored data can be directly viewed, modified, and migrated and exchanged through the SSD on the device side without taking out the SSD, so that the stored data can be managed more flexibly and conveniently. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a flow chart of a high-speed data storage method provided by the present invention.

[0036] Figure 2 It is a schematic diagram of each module of a high-speed data storage method provided by the present invention.

[0037] Figure 3 It is a structural schematic diagram of a high-speed data storage device provided by the present invention.

[0038] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0039] The present invention belongs to the field of embedded data acquisition and storage processing. In the field of wireless communications, the demand for high-bandwidth, multi-channel high-speed data acquisition and storage processing applications is very extensive, and the data storage processing rate requirement can be as high as several GB per second. The high-speed acquisition processing of FPGA and the high-speed storage capacity of SSD provide possibilities for this technical field.

[0040] In the prior art, portable or vehicle-mounted data collection devices are the main form, and generally, data collection, storage and analysis are two independent systems. Therefore, the collected and stored data need to be imported into offline analysis equipment for analysis.

[0041] For acquisition and storage devices, high-speed acquisition storage and management capabilities are two key functional indicators. In existing systems, both cannot be taken into account at the same time. If implemented through FPGA, it can only be achieved at the expense of management and user experience functions, such as unfriendly viewing of acquisition data and management of acquisition files, weak support for multi-channel acquisition, weak acquisition metadata management, and low efficiency of data migration and export. In addition, due to the primitive storage format of SSD, it cannot be directly read and managed by other systems, such as the analysis system cannot directly read SSD. If implemented through CPU, although the management function is powerful, the performance cannot meet the needs of high-speed data acquisition.

[0042] In the existing technology, using FPGA for data acquisition is the mainstream solution, which mostly adopts raw data storage and does not involve management functions such as file systems. It has poor flexibility and complex FPGA implementation, resulting in weak overall data management functions, insufficient multi-channel acquisition support capabilities and poor user experience.

[0043] In order to solve the above problems, the present invention proposes a high-speed data storage method, which realizes both high-speed acquisition and storage and convenient and friendly data management, supports multi-channel parallel high-speed acquisition and storage, and realizes standardized file system format and automatic preservation of acquisition meta-information, so that the data in the storage disk can be directly read and managed in other systems, supports universalization, and has the advantages of high speed, high efficiency, and high ease of use.

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Combine the following Figures 1 to 4 A high-speed data storage method and device of the present invention are described.

[0046] Figure 1 Schematic diagram of a high-speed data storage method provided by the present invention. Figure 1 As shown, the method comprises the following steps:

[0047] Step 101: Format the SSD according to the exFAT format, and receive data acquisition instructions for multiple channels of data through the CPU.

[0048] Figure 2 Schematic diagram of each module of a high-speed data storage method provided by the present invention, such as Figure 2 shown.

[0049] Specifically, the high-speed data storage method provided by an embodiment of the present invention includes at least a central processing unit (CPU) (including a management program, a file system and an NVME driver), a field programmable gate array (FPGA) module, an analog to digital converter (ADC) channel and an ADC interface, a multi-channel digital down conversion (DDC) channel (existing inside the FPGA), a double data rate (DDR) memory and a DDR interface, and a solid state disk (SSD).

[0050] The data acquisition source is the in-phase quadrature (IQ) data after the ADC channel or the IQ data of multiple digital down conversion (DDC) channels after channelization.

[0051] The management plane is between the CPU and SSD, the data service plane is between the DDR memory and SSD, and the FPGA module provides the underlying access path. The management plane has a small data flow and low real-time requirements, while the service plane has a large data flow and high real-time requirements.

[0052] Through the channel between the CPU and the FPGA module, the CPU can directly or indirectly access the cache resources in the DDR memory. The SSD is connected to the FPGA through the Peripheral Component Interconnect Express (PCIE) bus interface, and then interacts with the DDR memory through direct memory access (DMA).

[0053] In order to achieve high-speed data interaction, the embodiment of the present invention sets the SSD and DDR memory in the same address space. On the one hand, the FPGA module can write the collected IQ data into the DDR memory for caching at high speed, and on the other hand, the SSD can quickly read the data in the DDR memory under the control of the CPU, and can also write the data in the SSD into the DDR memory at high speed, realizing business functions such as data playback, and improving the efficiency and flexibility of data management.

[0054] At the same time, in order to meet the general ultra-high-speed data storage and reading requirements (several GB / s data transmission), the embodiment of the present invention also develops a file system on the CPU side and formats the SSD according to a universal format, and all files in the file system are in a universal format. The universal format can be an extended file allocation table (exFAT), a new technology file system (NTFS), or a 32-bit binary number record management file allocation table (FAT), etc. In the embodiment of the present invention, exFAT is used as an example for explanation, that is, the SSD is formatted according to the exFAT format.

[0055] When performing data collection, the user first sends multiple (or single) channel collection instructions through the control interface. The instructions contain the center frequency, collection bandwidth, and collection description information. The collection description information is input by the user, including location description information or collection vehicle information, and then the CPU receives the single or multiple channel data collection instructions sent by the user.

[0056] Step 102: Generate auxiliary information of the multiple channels of data based on the data acquisition instruction.

[0057] Specifically, after the CPU receives the multi-channel data acquisition instruction, it calculates the sampling rate and generates auxiliary information. The auxiliary information includes the center frequency, bandwidth, sampling rate, sampling start time, sampling end time, sampling location information (depending on whether the device has GPS), acquisition description information, and digital intermediate frequency points. Some information can be filled in at the end of the acquisition, such as the acquisition end time.

[0058] Step 103: Based on the auxiliary information of the multiple data, a cache time granularity is set, and a cache area is divided for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, a plurality of storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit.

[0059] Specifically, the DDR memory is a core cache resource of the FPGA module, that is, the DDR memory is used to cache IQ data collected through the ADC channel and the DDC channel.

[0060] When the CPU receives a single-channel or multi-channel data acquisition instruction, it calculates the sampling rate according to the acquisition bandwidth in the data acquisition instruction, generates auxiliary information for the multi-channel data, and then calculates the storage space requirements for each channel of data based on the number of acquisition channels, thereby dynamically allocating the storage space in the DDR memory and dividing the storage space in the DDR memory into multiple cache areas.

[0061] In order to fully and reasonably utilize the storage space in the DDR memory, the embodiment of the present invention presets the cache time granularity according to parameters such as the minimum storage unit size in the formatted SSD, the data volume of multi-channel data, and the storage space of the cache area, and ensures that the storage space of the cache area is an integer multiple of the minimum storage unit size.

[0062] Before multi-channel data acquisition, a cache area is allocated for each data channel in the DDR memory, and the addresses of multiple cache areas corresponding to the multi-channel data are continuous, so as to realize data parallel caching. At the same time, each minimum storage unit in the formatted SSD is used as a storage cluster area. According to the attached information of the multi-channel data, one or more storage cluster areas with continuous addresses are allocated to each data channel in the formatted SSD according to the cache time granularity and the storage space size required for the multi-channel data, so as to subsequently write the multi-channel data in the cache area to the SSD in parallel.

[0063] Step 104: Cache the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module.

[0064] Specifically, when performing multi-channel data acquisition according to data acquisition instructions, different channels of data usually have different data volumes. Therefore, the size of the cache area corresponding to each channel of data is also different. In order to improve data caching efficiency, it is necessary to cache multiple channels of data in parallel to the cache area according to the preset cache time granularity, that is, to cache multiple channels of data to the cache area at the same time.

[0065] Step 105, based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written into the storage cluster area in parallel according to the cache time granularity through the FPGA module; the NVME instructions are generated based on the principle that data is written at consecutive addresses on the SSD.

[0066] Specifically, when writing the multi-channel data in the cache area to the SSD, the multi-channel data will be presented in the SSD in the form of an acquisition file. The file format of the acquisition file is the exFAT file format. The acquisition file can be named according to key information such as the acquisition time, center frequency, and acquisition bandwidth in the data acquisition instruction, so as to facilitate the later viewing, modification, and migration of the data corresponding to the acquisition file.

[0067] According to the multi-channel data acquisition instruction, the FPGA module first collects the data corresponding to the acquisition file through the data acquisition channel and caches it in the cache area in the DDR memory. Then, based on the parallel synchronous ping-pong cache mechanism, the multi-channel data cached in the cache area of ​​the DDR memory is written in parallel to the storage cluster area in the SSD through the FPGA module according to the cluster information of the storage cluster area corresponding to the multi-channel data pre-allocated and recorded in the FAT.

[0068] In addition, when writing the multi-channel data cached in the cache area of ​​the DDR memory in parallel to the storage cluster area in the SSD, an NVME instruction is also generated based on the principle that data is written at continuous addresses on the SSD. According to the NVME instruction, the multi-channel data can be written to the SSD at one time according to the continuous addresses on the SSD, thereby improving the data writing speed. In this case, since the amount of data written to the multi-channel data at the same time is large, the free area in the SSD (i.e., a larger continuous storage area) is preferred to ensure sufficient continuous storage area. If it is single-channel data, since the amount of data is small, the fragmented area in the SSD (i.e., a smaller independent storage area) is preferred. If the fragmented area cannot meet the capacity requirements, the user can be prompted to organize the fragmented area or select the free area again.

[0069] For example, for the low-speed acquisition of narrowband data, since the amount of data is relatively small, it is preferred to allocate storage clusters in non-continuous storage areas (i.e., fragmented areas); for the high-speed acquisition of broadband data or data acquisition through ADC channels, since the amount of data is relatively large, it is preferred to allocate multiple continuous storage clusters in continuous storage areas (i.e., free areas), and manage the allocation of storage clusters through a bitmap. At the same time, the cluster information of the storage clusters allocated in multiple ways is recorded in the file allocation table (FAT), so that data can be written to the SSD in parallel according to continuous addresses, thereby improving the data writing speed.

[0070] In the embodiment of the present invention, since the multi-channel data is presented in the SSD in the form of an acquisition file, and the file format of the acquisition file is the exFAT file format, and is named according to key information such as the acquisition time, center frequency, and acquisition bandwidth in the data acquisition instruction, rather than the original IQ data, therefore, after the acquisition is completed and stored in the SSD, data reading and writing operations can be performed on a general-purpose computer or server, so that high-capacity offline data can be directly viewed, modified, and migrated and exchanged through the SSD on the device side, while avoiding the problems of low efficiency and easy errors in exporting large-capacity data, reducing the difficulty of data management and improving data management efficiency.

[0071] Based on the above embodiment, when writing multiple data paths in parallel to the SSD, firstly, a cluster space is pre-allocated in the SSD for storing the data, and at least one cluster space (i.e., one storage cluster area) is allocated to each data path. For example, one cluster space is allocated for narrowband data collection, and multiple continuous cluster spaces are allocated for broadband data collection. Then, according to the principle that data is written to the SSD at continuous addresses, the non-volatile memory interface (NVME) driver in the CPU is used (see further). Figure 2 ) generates NVME instructions; then generates a submission queue (SQ) based on the feedback of data reading and writing through the parallel synchronous ping-pong cache mechanism of each data acquisition channel of the FPGA, and stores NVME instructions in the SQ, so that when the SSD accesses the DDR memory through the PCIE channel for data interaction, it can write data to the SSD according to the continuous address on the SSD. Since the address is continuous, there is no need to frequently move or address during the data storage process, thereby improving the data storage speed.

[0072] The invention provides a high-speed data storage method, firstly, the SSD is formatted according to the exFAT format, and a multi-channel data acquisition instruction is received through the CPU; then, according to the multi-channel data acquisition instruction, a cache area is divided for each channel of data in the DDR memory, and at the same time, a plurality of storage cluster areas with continuous addresses are allocated for each channel of data in the formatted SSD according to the cache time granularity; then, the multi-channel data is cached in parallel to the cache area through the FPGA module; finally, based on the parallel synchronous ping-pong cache mechanism and the NVME instruction, the data in the cache area is written in parallel to the storage cluster area through the FPGA module. Among them, the NVME instruction is generated based on the principle that data is written to the SSD at a continuous address, which enables the data corresponding to the acquisition file in the exFAT file format to be written to the SSD at a continuous address, and there is no need to frequently move or address during the data storage process, which improves the data storage speed and realizes efficient data storage. At the same time, due to the versatility of the exFAT file format, the acquisition file corresponding to the stored data can be directly viewed, modified, and migrated and exchanged through the SSD on the device side without taking out the SSD, so that the stored data can be managed more flexibly and conveniently.

[0073] Optionally, the setting of the cache time granularity based on the auxiliary information of the multiple data paths, and dividing a cache area for each data path in the DDR memory includes:

[0074] Based on the auxiliary information of the multiple data channels, a cache time granularity is set, and a cache space required for each data channel is calculated;

[0075] Based on the cache space required for each data path, a cache area is divided for each data path in the DDR memory.

[0076] Specifically, after the CPU receives the data acquisition instruction, it calculates the sampling rate and generates auxiliary information. The auxiliary information includes the center frequency, bandwidth, sampling rate, sampling start time, sampling end time, sampling location information (depending on whether the device has GPS), acquisition description information, and digital intermediate frequency points. Some information can be filled in at the end of the acquisition, such as the acquisition end time. In addition, if the amount of data calculated according to the data acquisition instruction exceeds the SSD write speed, an error message is returned to the user and the execution is terminated so that the user can modify the data acquisition instruction and then restart.

[0077] The amount of data collected from multiple channels is inconsistent. For example, in narrowband data collection scenarios, the amount of data is relatively small, while in broadband data collection scenarios or data collection scenarios through ADC channels, the amount of data is relatively large. Therefore, the cache space required for different channels of collected data can be calculated based on the auxiliary information. Then, according to the cache space required for each channel of data, a cache area can be allocated for each channel of data in the DDR memory.

[0078] At the same time, in an embodiment of the present invention, in order to ensure that data is read and written smoothly, it is necessary to set the cache time granularity to balance the space and time parameters, ensure the effective cache time of the data and the available cache space, and the cache time granularity needs to be set to no less than 100ms.

[0079] For example, for a 100M bandwidth single antenna signal, a 204.8M sampling rate is selected, and IQ data is stored at 16 bits each, then the data volume per second is 819.2MB, and 128M bytes can be allocated to each to meet the demand. Among them, the cache time granularity is set to no less than 100 milliseconds, that is, data is read and written every 100 milliseconds, and data is cached periodically.

[0080] For another example, if the size of the formatted cluster is selected to be 1 MB, in order to balance the space and time parameters, the cache time granularity for narrowband signal data at a sampling rate of 250K can reach 1 second.

[0081] In some embodiments, a portion of fixed storage space needs to be allocated to cache command messages of the NVME driver queue. Generally, 16M or less can meet the demand.

[0082] In some embodiments, for most system-on-chips (SoCs), since the CPU, FPGA module and DDR memory are in the same address space, the programmable logic (PL) unit can directly access the DDR memory on the processing system (PS) side, so the command messages to the NVME driver queue can be directly cached in the DDR memory of the CPU, thereby further improving data processing efficiency.

[0083] After obtaining the data cache space required by different channels of collected data through calculation, the embodiment of the present invention can divide the storage space in the DDR memory into multiple cache areas with continuous cache addresses according to the cache space required by the multiple channels of data, thereby achieving full and reasonable utilization of the storage space in the DDR memory.

[0084] Optionally, the ancillary information includes a center frequency, a bandwidth, a sampling rate, a sampling start time, a sampling end time, sampling location information, acquisition description information, and a digital intermediate frequency.

[0085] Specifically, after the CPU receives the data acquisition instruction, it calculates the sampling rate and generates auxiliary information. The auxiliary information includes the center frequency, bandwidth, sampling rate, sampling start time, sampling end time, sampling location information (depending on whether the device has GPS), acquisition description information, and digital intermediate frequency points. Some information can be filled in at the end of the acquisition, such as the acquisition end time.

[0086] The embodiment of the present invention uses a separate file to store the auxiliary information, and uses the first storage cluster area to store the file data corresponding to the auxiliary information. For example, in a sports scene, data is collected, and the position information is recorded once per second. When the collection is completed, the auxiliary information and the position information are written together into the first storage cluster area corresponding to the collection file corresponding to the position information, so that the auxiliary information corresponding to the data can be quickly viewed after the data storage is completed, thereby improving the convenience of data management.

[0087] Optionally, the caching the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module includes:

[0088] Obtaining a buffer address corresponding to each channel of data in the multiple channels of data;

[0089] Based on the cache area address, each path of data is cached in parallel in the cache area corresponding to each path of data according to the cache time granularity.

[0090] Specifically, according to the auxiliary information of multiple data channels, the cache time granularity is set, and after a cache area is allocated for each data channel in the DDR memory, the CPU controls the FPGA module to collect IQ data of each channel, and caches the data corresponding to the collection file into the cache area through the data acquisition channel (such as ADC channel or DDC channel).

[0091] In order to improve the cache speed and ensure the continuity of data, the addresses of the corresponding multiple cache areas divided for multiple data channels are continuous. When caching data, the FPGA module can cache multiple data channels to the corresponding cache areas at one time according to the pre-divided cache area addresses, thereby improving the data acquisition speed.

[0092] Optionally, based on the parallel synchronous ping-pong cache mechanism and the NVME instruction, writing the data in the cache area into the storage cluster area in parallel according to the cache time granularity through the FPGA module includes:

[0093] Based on the parallel synchronous ping-pong cache mechanism, each cache area is divided into two cache sub-areas;

[0094] Determining the state of each cache sub-region in each cache region;

[0095] When the state of the cache sub-area is a readable state, based on the NVME instruction, the data in the cache sub-area is written into the storage cluster area in parallel through the FPGA module according to the cache time granularity.

[0096] Specifically, based on the parallel synchronous ping-pong cache mechanism, each cache area is divided into two cache sub-areas, and the two cache sub-areas are read and written alternately.

[0097] Assume that The data of each channel is collected in parallel, with a time granularity of (Unit: milliseconds), the buffer area corresponding to each data channel is divided into , There are two cache sub-areas. The size of the smallest storage unit in the SSD is S (in bytes). The data volume of each channel is Bytes / second (determined by bandwidth and sampling rate), where The choice must ensure that each path , The value of is an integer multiple of S. Considering that S is usually selected in units of at least 1MB or larger during formatting, we have:

[0098]

[0099]

[0100] When data is written into the cache sub-area When the CPU can Read the data and process it. After writing is full, switch to writing to the cache sub-area , and the CPU starts to read from the cache sub-area Read the data.

[0101] At the same time, the CPU obtains the status of each cache sub-area through real-time query or interruption. When the cache is valid, that is, it is in a readable state, the data of the cache sub-area is read and written to the SSD. When the writing is completed, the cache valid flag of the cache sub-area is cleared, and the status of another cache sub-area is obtained through real-time query or interruption.

[0102] In some embodiments, it is assumed that two cache sub-areas are being read and written alternately. The reading and writing process is performed alternately between the two cache sub-areas: at the beginning, the cache states of the two cache sub-areas are invalid and cannot be read, the cache state is in a writable state, and the flag is 0; then the data is written to a cache sub-area, which can be read after the writing is completed, the cache state is in a readable state, the flag becomes 1, and then the data is written to another cache sub-area, and its state becomes 1; at the same time, the CPU reads the data in the cache sub-area with a flag of 1 and writes it to the SSD. After the writing is completed, the valid flag of the cache sub-area is cleared, and it becomes a writable state, and the flag becomes 0.

[0103] When it is determined that the state of each cache sub-area in each cache area is a readable state, based on the NVME instruction, the data in the cache sub-area is written in parallel to the storage cluster area through the FPGA module according to the cache time granularity, that is, the data is written to the SSD at one time according to the continuous address on the SSD.

[0104] The embodiment of the present invention obtains the status of the cache sub-area by real-time query or interruption, determines that the cache status of the cache sub-area is a readable state, and then reads the cache data of the cache sub-area through the CPU and writes it in parallel to the SSD, thereby ensuring the smooth progress of parallel synchronous ping-pong cache, realizing continuous writing and reading of data, and improving the data processing reading and writing speed.

[0105] Optionally, the method further comprises:

[0106] When all the data in the buffer area are written into the storage cluster area, the file entry information is updated, or when a data acquisition end instruction is received through the CPU, the file entry information is updated.

[0107] Specifically, during data collection and storage, the CPU generates file entry information for each data path in the SSD, and records the storage cluster number corresponding to the storage cluster area of ​​each data in the SSD in each file entry information. The CPU usually updates the corresponding file entry information regularly, usually about 10 seconds.

[0108] In an embodiment of the present invention, after the FPGA module completes the data collection of the current cache of the DDR memory, the process can be automatically terminated. At this time, the CPU queries and obtains the relevant identifier, records the end time of the collection, writes the current cached data into the SSD, and updates the file entry information, thereby realizing automatic control.

[0109] In some embodiments, the user can also send an acquisition end instruction. At this time, regardless of whether the data acquisition currently cached in the DDR memory is completed, the CPU will send the received acquisition end instruction to the FPGA module to terminate the data acquisition process, and then obtain the relevant identifier through CPU query, record the end time, write the currently cached data to the SSD, and update the file entry information, thereby meeting the user's diverse data storage needs.

[0110] In the embodiment of the present invention, the CPU caches the cluster information and file information of each write data, and updates the exFAT or bitmap after the acquisition is completed, thereby updating the file entry information. Since the file entry information is updated only once at the end, storage space is saved and storage performance overhead is reduced. In addition, this method of generating files after data acquisition is completed reduces the time and space required for regular updates of file entry information during the acquisition process, thereby improving data writing efficiency.

[0111] Based on the above embodiment, after the data is written, the CPU can receive instructions, read out the data stream according to the universal file format (ie, exFAT format), and transmit it through the protocol group package, thereby achieving flexible, convenient and efficient management of the data.

[0112] A high-speed data storage device provided by the present invention is described below. The high-speed data storage device described below and the high-speed data storage method described above can be referenced to each other.

[0113] Based on any of the above embodiments, Figure 3 This is a schematic diagram of the structure of a high-speed data storage device provided in this article. Figure 3 The embodiment of the present invention provides a high-speed data storage device, including a receiving module 301, a generating module 302, a partitioning module 303, a cache module 304 and a storage module 305, wherein:

[0114] The receiving module 301 is used to format the SSD according to the exFAT format, and receive the data acquisition instruction of multiple data through the CPU; the generating module 302 is used to generate the auxiliary information of the multiple data based on the data acquisition instruction; the partitioning module 303 is used to set the cache time granularity based on the auxiliary information of the multiple data, and divide a cache area for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, multiple storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit; the cache module 304 is used to cache the multiple data in parallel to the cache area through the FPGA module according to the cache time granularity; the storage module 305 is used to write the data in the cache area in parallel to the storage cluster area through the FPGA module according to the cache time granularity based on the parallel synchronous ping-pong cache mechanism and the NVME instruction; the NVME instruction is generated based on the principle that data is written at continuous addresses on the SSD.

[0115] The present invention provides a high-speed data storage device, firstly formats the SSD according to the exFAT format, and receives a multi-channel data acquisition instruction through the CPU; then, according to the multi-channel data acquisition instruction, a cache area is divided for each channel of data in the DDR memory, and at the same time, multiple storage cluster areas with continuous addresses are allocated for each channel of data in the formatted SSD according to the cache time granularity; then, the multi-channel data is cached in parallel to the cache area through the FPGA module; finally, based on the parallel synchronous ping-pong cache mechanism and the NVME instruction, the data in the cache area is written in parallel to the storage cluster area through the FPGA module. Among them, the NVME instruction is generated based on the principle that data is written to the SSD at a continuous address, which enables the data corresponding to the acquisition file in the exFAT file format to be written to the SSD at a continuous address, and there is no need to frequently move or address during the data storage process, which improves the data storage speed and realizes efficient data storage. At the same time, due to the versatility of the exFAT file format, the acquisition file corresponding to the stored data can be directly viewed, modified, and migrated and exchanged through the SSD on the device side without taking out the SSD, so that the stored data can be managed more flexibly and conveniently.

[0116] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the high-speed data storage method, which includes:

[0117] Format the SSD according to the exFAT format, and receive data acquisition instructions for multiple channels of data through the CPU;

[0118] Based on the data acquisition instruction, generate the auxiliary information of the multiple channels of data;

[0119] Based on the auxiliary information of the multiple data, a cache time granularity is set, and a cache area is divided for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, a plurality of storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit;

[0120] Cache the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module;

[0121] Based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written into the storage cluster area in parallel according to the cache time granularity through the FPGA module; the NVME instructions are generated based on the principle that data is written at continuous addresses on the SSD.

[0122] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0123] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the high-speed data storage method provided by the above methods, the method comprising:

[0124] Format the SSD according to the exFAT format, and receive data acquisition instructions for multiple channels of data through the CPU;

[0125] Based on the data acquisition instruction, generate the auxiliary information of the multiple channels of data;

[0126] Based on the auxiliary information of the multiple data, a cache time granularity is set, and a cache area is divided for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, a plurality of storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit;

[0127] Cache the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module;

[0128] Based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written into the storage cluster area in parallel according to the cache time granularity through the FPGA module; the NVME instructions are generated based on the principle that data is written at continuous addresses on the SSD.

[0129] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the high-speed data storage method provided by the above methods, the method comprising:

[0130] Format the SSD according to the exFAT format, and receive data acquisition instructions for multiple channels of data through the CPU;

[0131] Based on the data acquisition instruction, generate the auxiliary information of the multiple channels of data;

[0132] Based on the auxiliary information of the multiple data, a cache time granularity is set, and a cache area is divided for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, a plurality of storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit;

[0133] Cache the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module;

[0134] Based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written into the storage cluster area in parallel according to the cache time granularity through the FPGA module; the NVME instructions are generated based on the principle that data is written at continuous addresses on the SSD.

[0135] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0137] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0138] "Determine B based on A" in the embodiments of the present application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "Determine B based on A and C", "Determine B based on A, C and E", "Determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "When A meets the first condition, use the first method to determine B"; for another example, "When A meets the second condition, determine B", etc.; for another example, "When A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "When A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0139] It should also be noted that the terms "target", "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0140] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to them.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-speed data storage method, characterized in that: include: Format the SSD according to the exFAT format, and receive data acquisition instructions for multiple channels of data through the CPU; Based on the data acquisition instruction, generate the auxiliary information of the multiple channels of data; Based on the auxiliary information of the multiple data, a cache time granularity is set, and a cache area is divided for each data in the DDR memory, and the addresses of the multiple cache areas corresponding to the multiple data are continuous; and based on the auxiliary information of the multiple data, a plurality of storage cluster areas with continuous addresses are allocated to each data in the formatted SSD according to the cache time granularity; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit; Cache the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module; Based on the parallel synchronous ping-pong cache mechanism and NVME instructions, the data in the cache area is written into the storage cluster area in parallel according to the cache time granularity through the FPGA module; the NVME instructions are generated based on the principle that data is written at continuous addresses on the SSD.

2. The high-speed data storage method according to claim 1, characterized in that: The method of setting a cache time granularity based on the auxiliary information of the multiple data paths and dividing a cache area for each data path in the DDR memory includes: Based on the auxiliary information of the multiple data channels, a cache time granularity is set, and a cache space required for each data channel is calculated; Based on the cache space required for each data path, a cache area is divided for each data path in the DDR memory.

3. The high-speed data storage method according to claim 2, characterized in that: The ancillary information includes the center frequency, bandwidth, sampling rate, sampling start time, sampling end time, sampling location information, acquisition description information and digital intermediate frequency.

4. The high-speed data storage method according to claim 1, characterized in that: The step of caching the multiple data in parallel to the cache area according to the cache time granularity through the FPGA module includes: Obtaining a buffer address corresponding to each channel of data in the multiple channels of data; Based on the cache area address, each path of data is cached in parallel in the cache area corresponding to each path of data according to the cache time granularity.

5. The high-speed data storage method according to claim 1, characterized in that: The method of writing the data in the cache area in parallel to the storage cluster area according to the cache time granularity based on the parallel synchronous ping-pong cache mechanism and the NVME instruction through the FPGA module includes: Based on the parallel synchronous ping-pong cache mechanism, each cache area is divided into two cache sub-areas; Determining the state of each cache sub-region in each cache region; When the state of the cache sub-area is a readable state, based on the NVME instruction, the data in the cache sub-area is written into the storage cluster area in parallel through the FPGA module according to the cache time granularity.

6. The high-speed data storage method according to claim 1, characterized in that: The method further comprises: When all the data in the buffer area are written into the storage cluster area, the file entry information is updated, or when a data acquisition end instruction is received through the CPU, the file entry information is updated.

7. A high-speed data storage device, characterized in that: include: A receiving module is used to format the SSD according to the exFAT format and receive data acquisition instructions for multiple channels of data through the CPU; A generating module, used for generating the auxiliary information of the multiple channels of data based on the data acquisition instruction; A partition module, configured to set a cache time granularity based on the auxiliary information of the multiple data paths, and to divide a cache area for each data path in the DDR memory, wherein the addresses of the multiple cache areas corresponding to the multiple data paths are continuous; and, based on the auxiliary information of the multiple data paths, to allocate multiple storage cluster areas with continuous addresses according to the cache time granularity for each data path in the formatted SSD; the storage cluster area is the smallest storage unit in the formatted SSD, and the storage space of the cache area is an integer multiple of the storage space of the smallest storage unit; A cache module, used for caching the multiple data in parallel to the cache area according to the cache time granularity through an FPGA module; The storage module is used to write the data in the cache area into the storage cluster area in parallel according to the cache time granularity through the FPGA module based on the parallel synchronous ping-pong cache mechanism and NVME instructions; the NVME instructions are generated based on the principle that data is written at consecutive addresses on the SSD.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the high-speed data storage method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the high-speed data storage method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the high-speed data storage method according to any one of claims 1 to 6 is implemented.

Images