Efficient data storage method and system

Abstract

The invention discloses an efficient data storage method and system, and belongs to the technical field of data storage. The method comprises the steps of obtaining to-be-stored data; partitioning the data storage recorder into a plurality of storage blocks; and storing the to-be-stored data into the storage block according to the type of the to-be-stored data. According to the invention, the storage space of the recorder is divided into a plurality of areas for storing different types of data, so that the storage address of the data to be stored and read can be more effectively and quickly positioned when the data is stored and read, and the storage and reading efficiency of the data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to an efficient data storage method and system. Background Art

[0002] With the development of science and technology, the methods of storing data used by the collection equipment in the telecommunications industry today mainly include database storage, file storage and data stream storage. When using the data stream storage method, there is no unified standard for the storage and reading of data. Generally, the data is stored and read according to the specified data length. When the data length is not fixed and the data volume is large, the data storage efficiency is low and the memory space utilization rate is low.

[0003] The existing storage method occupies a large amount of storage space and has low storage efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide an efficient data storage and analysis method, partitioning a data storage recorder, dividing the recorder into multiple areas for storing different types of data; when a receiving end receives data, first determine the type of data, find the storage area of ​​this type of data according to the data type, then frame and segment the data, mark each segment of data with different marking symbols, and store the marked data in the recorder; when reading the stored data, first determine the type of data, locate the storage area corresponding to this type of data according to the data type, parse the data according to the marking symbol of each segment of data, and restore the data information.

[0005] In order to solve the above technical problems, the present invention provides an efficient data storage method, comprising the following steps:

[0006] Get the data to be stored;

[0007] Partitioning the data storage recorder into multiple storage blocks;

[0008] According to the type of data to be stored, the data to be stored is stored in the storage block.

[0009] Preferably, partitioning the data storage recorder into a plurality of storage blocks specifically comprises the following steps:

[0010] Dividing the data storage recorder into a plurality of storage blocks, dividing the storage blocks into a plurality of storage pages, and marking the address of each storage block;

[0011] An area is defined for each storage block.

[0012] Preferably, the number of the storage blocks is 10;

[0013] The 0th to 3rd storage blocks are defined as reserved areas;

[0014] The 4th to 6th storage blocks are defined as the key area;

[0015] The 7th to 9th storage blocks are defined as the address definition area;

[0016] The space of 0 to 100M in the 10th storage block is defined as a reserved area, and the space after 100M is defined as a data storage area, which includes a fixed data storage area, a collection data storage area, and a video data storage area;

[0017] Preferably, the seventh storage block is a fixed data area address definition area, the eighth storage block is a collection data area address definition area, and the ninth storage block is a video data area address definition area.

[0018] Preferably, the method further comprises the following steps:

[0019] The addresses of the fixed data area address definition area, the acquisition data area address definition area and the video data area address definition area are saved in the address definition area.

[0020] Preferably, according to the type of data to be stored, storing the data to be stored in the storage block specifically includes the following steps:

[0021] According to the type of data to be stored, the address of the corresponding data storage area is obtained from the address definition area;

[0022] According to the address of the data storage area, an empty storage block is obtained from the corresponding data storage area;

[0023] Preprocess the data to be stored to obtain preprocessed data;

[0024] Store the preprocessed data into an empty memory block.

[0025] Preferably, the preprocessing includes framing, segmenting and marking.

[0026] Preferably, the types of the mark include coding mark, time mark, information body mark and frame header and frame tail mark;

[0027] The marking symbols of the marking include time symbols, segment symbols and frame symbols.

[0028] Preferably, when the time symbol appears in a data frame, it indicates that the time from the frame header of the data frame to the time before the time symbol is the time of data record;

[0029] The segmentation symbol is used as a spacer between information bodies, marking the end of each information body. The number of segmentation symbols that appear in a data frame indicates the number of information bodies.

[0030] The framing symbol is used as the end of a data frame. When the framing symbol appears at the end of a data frame, it indicates that the data is stored successfully and recorded.

[0031] The present invention also provides an efficient data storage method, comprising:

[0032] An acquisition module, used to acquire data to be stored;

[0033] A partition module, used for partitioning the data storage recorder into a plurality of storage blocks;

[0034] The storage module is used to store the data to be stored in the storage block according to the type of the data to be stored.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The present invention divides the storage space of the recorder into multiple areas for storing different types of data, so that the storage address of the data to be stored and read can be located more effectively and quickly when storing and reading data, thereby improving the storage and reading efficiency of data.

[0037] 2. Before storing data, the present invention first marks each framed data segment so that specific data and specific bytes can be stored more quickly at one time, and different information body storage spaces can be opened up according to the size of each information body. For data storage blocks of the same size in the data recorder, more data information can be stored, which greatly improves the utilization rate of the storage space and extends the service life of the data recorder. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0039] Figure 1 It is a schematic diagram of the data format;

[0040] Figure 2 It is a schematic diagram for storing data according to a fixed length;

[0041] Figure 3 It is a schematic diagram of data framing and segmentation;

[0042] Figure 4 It is a flow chart of an efficient data storage method. DETAILED DESCRIPTION

[0043] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0044] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0045] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0047] like Figure 4 As shown, the present invention provides an efficient data storage method, comprising the following steps:

[0048] Get the data to be stored;

[0049] Partitioning the data storage recorder into multiple storage blocks;

[0050] According to the type of data to be stored, the data to be stored is stored in the storage block.

[0051] Preferably, partitioning the data storage recorder into a plurality of storage blocks specifically comprises the following steps:

[0052] Dividing the data storage recorder into a plurality of storage blocks, dividing the storage blocks into a plurality of storage pages, and marking the address of each storage block;

[0053] An area is defined for each storage block.

[0054] Preferably, the number of the storage blocks is 10;

[0055] The 0th to 3rd storage blocks are defined as reserved areas;

[0056] The 4th to 6th storage blocks are defined as the key area;

[0057] The 7th to 9th storage blocks are defined as the address definition area;

[0058] The space of 0 to 100M in the 10th storage block is defined as a reserved area, and the space after 100M is defined as a data storage area, which includes a fixed data storage area, a collection data storage area, and a video data storage area;

[0059] Preferably, the seventh storage block is a fixed data area address definition area, the eighth storage block is a collection data area address definition area, and the ninth storage block is a video data area address definition area.

[0060] Preferably, the method further comprises the following steps:

[0061] The addresses of the fixed data area address definition area, the acquisition data area address definition area and the video data area address definition area are saved in the address definition area.

[0062] Preferably, according to the type of data to be stored, storing the data to be stored in the storage block specifically includes the following steps:

[0063] According to the type of data to be stored, the address of the corresponding data storage area is obtained from the address definition area;

[0064] According to the address of the data storage area, an empty storage block is obtained from the corresponding data storage area;

[0065] Preprocess the data to be stored to obtain preprocessed data;

[0066] Store the preprocessed data into an empty memory block.

[0067] Preferably, the preprocessing includes framing, segmenting and marking.

[0068] Preferably, the types of the mark include coding mark, time mark, information body mark and frame header and frame tail mark;

[0069] The marking symbols of the marking include time symbols, segment symbols and frame symbols.

[0070] Preferably, when the time symbol appears in a data frame, it indicates that the time from the frame header of the data frame to the time before the time symbol is the time of data record;

[0071] The segmentation symbol is used as a spacer between information bodies, marking the end of each information body. The number of segmentation symbols that appear in a data frame indicates the number of information bodies.

[0072] The framing symbol is used as the end of a data frame. When the framing symbol appears at the end of a data frame, it indicates that the data is stored successfully and recorded.

[0073] The present invention also provides an efficient data storage method, comprising:

[0074] An acquisition module, used to acquire data to be stored;

[0075] A partition module, used for partitioning the data storage recorder into a plurality of storage blocks;

[0076] The storage module is used to store the data to be stored in the storage block according to the type of the data to be stored.

[0077] In the prior art, Figure 1 As shown, this is the format of a piece of data.

[0078] If the data is stored in a fixed length, such as Figure 2 As shown, the specific process includes the following steps:

[0079] First, organize the data to be stored into a data stream: 2010010110111;

[0080] Secondly, the data is framed and segmented into time segments, information coding segments and information body segments, such as Figure 3 As shown;

[0081] Then the data is stored in corresponding fields in the storage space in segments. In the storage space, the length of each field has a fixed definition, and the length of the information body segment is set to the longest length;

[0082] Figure 1 The specific storage process of the data stream of the data shown is as follows: the time period length is 8 bytes, the information coding segment length is 4 bytes, the first information body length is 100 bytes, and the second information body length is 200 bytes. When storing, first determine the actual length of the data segment. If the length exactly matches the length of the corresponding field defined in the storage space, store it. If the length is less than the length of the corresponding field defined in the storage space, store the data segment first, and then fill the blank bits bit by bit in a loop until the fixed length of the storage space is filled before cyclically storing the next data segment. This storage method occupies more storage space and has low storage efficiency.

[0083] In order to better illustrate the technical effect of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:

[0084] Embodiment 1: An efficient data storage method, comprising:

[0085] The types of marking for each segment of data include: coding mark, time mark, information body mark and frame header and frame tail mark, and the marking symbols used include: time symbol, segmentation symbol and framing symbol; among them, when the time symbol appears in a data frame, it indicates that the time from the frame header of the data frame to the time symbol is the time of data record; the segmentation symbol is used as the interval between information bodies, marking the end of each information body. The number of segmentation symbols that appear in a data frame indicates the number of information body contents; the framing symbol is used as the end of a data frame. When the framing symbol appears at the end of a data frame, it indicates that the data is stored successfully and the recording is completed.

[0086] The specific method of partitioning the data storage recorder is as follows: divide the recorder into multiple storage blocks, mark the bad storage blocks, divide the good storage blocks into multiple storage pages, and mark the address of each storage block, and then define the area for each storage block. The size of each storage block is 4M, the size of each storage page is 32K, and when defining the area for each storage block, the 0th to 3rd storage blocks are defined as the reserved area, the 4th to 6th storage blocks are defined as the key area, the 7th to 9th storage blocks are defined as the address definition area, the space of 0 to 100M of the 10th storage block is defined as the reserved area, and the space after 100M is defined as the fixed data storage area, the acquisition data storage area, and the video data storage area; among them, the 7th storage block is the fixed data area address definition area, the 8th storage block is the acquisition data area address definition area, and the 9th storage block is the video data area address definition area. After receiving data and determining the data type, locate the address definition area according to the data type, read the address of the corresponding data storage area, jump to the storage block pointed to by the address according to the address, find the empty storage block, frame, segment and mark the data, and then store the data in the empty storage block.

[0087] This embodiment provides a highly efficient storage of streaming data from an acquisition device, partitions a data storage recorder, and divides the recorder into multiple areas for storing different types of data; when the receiving end receives the data, it first determines the type of data, finds the storage area for the type of data according to the data type, then frames and segments the data, marks each segment of data with different marking symbols, and stores the marked data in the recorder.

[0088] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules, modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0089] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0091] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0092] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An efficient data storage method, characterized in that: The following steps are involved: Get the data to be stored; Partitioning the data storage recorder into multiple storage blocks; According to the type of data to be stored, the data to be stored is stored in the storage block.

2. The efficient data storage method according to claim 1, characterized in that: Partitioning the data storage recorder into multiple storage blocks specifically includes the following steps: Dividing the data storage recorder into a plurality of storage blocks, dividing the storage blocks into a plurality of storage pages, and marking the address of each storage block; An area is defined for each storage block.

3. The efficient data storage method according to claim 2, characterized in that: The number of storage blocks is 10; The 0th to 3rd storage blocks are defined as reserved areas; The 4th to 6th storage blocks are defined as the key area; The 7th to 9th storage blocks are defined as address definition areas; The space of 0 to 100M in the 10th storage block is defined as a reserved area, and the space after 100M is defined as a data storage area. The data storage area includes a fixed data storage area, an acquisition data storage area, and a video data storage area.

4. The efficient data storage method according to claim 3, characterized in that: The 7th storage block is the fixed data area address definition area, the 8th storage block is the acquisition data area address definition area, and the 9th storage block is the video data area address definition area.

5. The efficient data storage method according to claim 4, characterized in that: The following steps are also included: The addresses of the fixed data area address definition area, the acquisition data area address definition area and the video data area address definition area are saved in the address definition area.

6. The efficient data storage method according to claim 5, characterized in that: According to the type of data to be stored, the data to be stored is stored in the storage block, which specifically includes the following steps: According to the type of data to be stored, the address of the corresponding data storage area is obtained from the address definition area; According to the address of the data storage area, an empty storage block is obtained from the corresponding data storage area; Preprocess the data to be stored to obtain preprocessed data; Store the preprocessed data into an empty memory block.

7. The efficient data storage method according to claim 6, characterized in that: The preprocessing includes framing, segmenting and marking.

8. The efficient data storage method according to claim 7, characterized in that: The types of the marks include coding marks, time marks, information body marks and frame header and frame tail marks; The marking symbols of the marking include time symbols, segment symbols and frame symbols.

9. The efficient data storage method according to claim 8, characterized in that: When the time symbol appears in a data frame, it indicates that the time from the frame head of the data frame to the time before the time symbol is the time of data record; The segmentation symbol is used as a spacer between information bodies, marking the end of each information body. The number of segmentation symbols that appear in a data frame indicates the number of information bodies. The framing symbol is used as the end of a data frame. When the framing symbol appears at the end of a data frame, it indicates that the data is stored successfully and recorded.

10. An efficient data storage system, used to implement the efficient data storage as claimed in any one of claims 1 to 8, characterized in that: include: An acquisition module, used to acquire data to be stored; A partition module, used for partitioning the data storage recorder into a plurality of storage blocks; The storage module is used to store the data to be stored in the storage block according to the type of the data to be stored.

Images