Industrial database message storage method

A database and file storage technology, applied in electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of access speed limitation, occupying large storage space, etc., to reduce storage space, shorten processing time, and speed up retrieval time. Effect

Active Publication Date: 2012-04-18
ASAT CHINA TECH
3 Cites 19 Cited by

AI-Extracted Technical Summary

Problems solved by technology

[0004] The storage of massive messages requires a large amount of sto...
View more

Method used

And, by belonging to the message of identical source identification in same data file, make no need to preserve the corresponding source identification of every message again...
View more

Abstract

The invention discloses an industrial database message storage method, which comprises the following steps of: dividing a retained file storage region into an index region and a data region; during storage of a message, storing index information of the message into the index region from the head of the index region according to a positive sequence, and storing information of other parts of the message into the data region from the bottom of the data region according to an inverted sequence; and automatically changing division of the index region and the data region according to data volumes stored in the index region and the data region so that the retained space for the index region cannot be wasted. The message can be stored in the file storage region with a fixed size efficiently, so the utilization rate of the space is maximized and the storage space occupied by the message is reduced; furthermore, a starting time stamp and an ending time stamp when each file storage space stores the message are stored, so during search of the message, the corresponding file storage space can be positioned quickly and the message can be positioned quickly according to the index information; therefore, the search time of the message is increased and the processing time is shorter.

Application Domain

Special data processing applications

Technology Topic

Start timeUtilization rate +3

Image

  • Industrial database message storage method
  • Industrial database message storage method
  • Industrial database message storage method

Examples

  • Experimental program(1)

Example Embodiment

[0036] In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.
[0037] A preferred embodiment of the present invention relates to a method for storing industrial database messages. The industrial message in this embodiment generally includes at least three parts: BLOB message data, source identification, and time stamp. It can also optionally include the access point attributes of the message, the quality code of the physical or logical link of the message transmission, and the message. Other attributes such as the type of physical or logical link for text transmission.
[0038] In this embodiment, the messages with the same source point identifier are stored in a file storage area, that is, in a data file. In the industrial database, corresponding data files are reserved for different source IDs. The size of the data file is determined according to the number of messages to be stored and the estimated message length, and the data file is divided into an index area and a data area. After the data file is full, another data file is automatically reserved for the source point identification, that is, one source point identification can correspond to multiple data files. Each source point identifier also contains a uniquely corresponding index file. The index file can be named after the source point identifier. The index file contains the names of all data files corresponding to the source point identifier and the reports stored in each data file. The start timestamp and end timestamp of the document can also contain information such as the index status of each data file.
[0039] The specific message storage process is as follows figure 1 Shown.
[0040] In step 101, the message is cached in the memory, and the source identification of the message and the timestamp of the message are divided into fields independently, and the rest of the attribute part will be merged with the message data into an independent storage part without distinction. ,Such as figure 2 Shown. Compress and encode message data and other attributes, such as using LZW (Lempel-Ziv-Welch) algorithm for compression and coding. The compression coding and reverse analysis are both defined and implemented by the application upper layer. After a certain number of messages are cached, they are stored in the industrial database.
[0041] In step 102, the corresponding data file in the database is found according to the source point identifier and the time stamp interval of the message.
[0042] Specifically, first find the corresponding index file according to the source point identifier, and the general index file name is the source point identifier, such as [source point identifier].pridx. Then read the data file name in the index file and the corresponding start time stamp and end time stamp (ie, time stamp interval), and compare the time stamp interval of the message to be stored with the time stamp interval corresponding to each data file, Find the name of the data file closest to the time stamp interval of the message to be stored. In actual operation, in order to speed up the search, several common search algorithms can be used, such as dichotomy, hash table and B and B+ trees.
[0043] After the corresponding data file is retrieved, step 103 is entered to determine whether the remaining space of the data file is greater than the space required by the message to be stored. If it is less, then step 104 is entered; if it is greater, step 107 is entered.
[0044] In step 104, there is not enough space for storing messages in the data file corresponding to the source point identifier, and a new data file is generated for the source point identifier. The data file is also divided into an index area and a data area. Then go to step 105.
[0045] In step 105, the physical storage location of the data part (including the message data and other attribute parts) of each message in the data file is calculated, and the physical storage location and time stamp of the message constitute index information of the message. Then go to step 106.
[0046] In step 106, the message buffered in the memory is stored in the new data file, starting from the head of the index area, and the index information of the message is stored in the index area in positive order, starting from the bottom of the data area, Store other parts of the message in the data area in reverse order, such as image 3 Shown.
[0047] Among them, the index information is composed of the time stamp of the message and the physical storage location of other parts of the information in the data area. The other part information of the message is information other than the source point identification and time stamp, including the access point attributes and the message Other attributes such as the quality code of the physical or logical link for message transmission, and the type of physical or logical link for message transmission.
[0048] As a further improvement, before storing the index information of the message in the index area, an optimized algorithm (such as a hash algorithm) can be used to sort the index information in the memory to form a corresponding mapping table (such as hash-map), Store the sorted index information mapping table (such as hash-map) in the index area so that the index information can be quickly located when retrieving messages.
[0049] After the message is stored, the name of the new data file, the start timestamp and the end timestamp of the message stored in the data file are saved in the index file corresponding to the source point identifier.
[0050] In step 107, the current data file has enough storage space, the current storage location of the data area in the data file is determined, and the data part (including the message data and other attribute parts) of each message is calculated according to the storage location in the data file The physical storage location of the message, the physical storage location and timestamp of the message constitute the index information of the message. Then go to step 108.
[0051] In step 108, according to the current storage positions of the index area and the data area in the data file, the index information of the message is stored in the index area in the forward order, and the other part of the message is stored in the data area in the reverse order. And update the start time stamp and end time stamp of the data file in the corresponding index file.
[0052] Similarly, before storing the index information of the message in the index area, you can use an optimized algorithm (such as a hash algorithm) to sort the index information in the memory to form a corresponding mapping table (such as hash-map), and sort The obtained index information mapping table (such as hash-map) is stored in the index area, so that the index information can be quickly located when retrieving messages.
[0053] In summary, due to large changes in industrial message length and unpredictability, when reserving message storage space, space is often wasted. It is easy to happen that there is still space in the index area and the data area is full, or the data area is still There is space and the index area is full. Through the above storage method, the data volume of the index area and the data area both increase from the two ends to the middle part of the data file. The middle part of the data file is a blank area. There is no fixed division between the index area and the data area, so it can be based on the stored report. The text needs to be changed automatically, so that there is no waste of storage space due to the reserved index area, and the continuous storage of data can be maintained for fast reading. When the amount of message data to be stored is greater than the blank part of the data file, stop storing the message in the data file, so as to store messages in a fixed-size file storage area with the highest efficiency and maximize space utilization , To reduce the storage space occupied by messages.
[0054] Moreover, by storing the messages belonging to the same source point identifier in the same data file, it is no longer necessary to save the source point identifier corresponding to each message, saving storage space, and in the industrial field there are tens of millions of industrial messages. This significantly improves data storage efficiency.
[0055] According to the message storage method of this embodiment, when a message is retrieved, the corresponding index file can be found directly according to the source identifier of the message to be retrieved. Then compare the time stamp interval of the message to be retrieved, the start time stamp and the end time stamp corresponding to each data file name in the index file, and find the data file containing the time stamp interval to be retrieved. In actual operation, the same Several common search algorithms can be used, such as dichotomy, hash table and B, B+ tree, etc. Then read the index area in the data file, perform a secondary search from the index area, and locate the actual storage location of the message in the data file. During data locating, if the index information has been sorted by an optimization algorithm, the corresponding index information can be quickly located according to the sorted mapping table to find the required message. For example, a hash-map can be used to improve retrieval efficiency. Through the above storage method, the retrieval of the message is more convenient and the processing time is shorter.
[0056] Although the present invention has been illustrated and described by referring to certain preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the present invention. The spirit and scope of the invention.

PUM

no PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.

Similar technology patents

Portable communication apparatus having triple-axis hinge folder and rotation locking device thereof

InactiveUS20050192066A1convenient to usemaximize space utilization
Owner:SAMSUNG ELECTRONICS CO LTD

Interlocking container

ActiveUS7628310B2maximize space utilization
Owner:PAPER SYST

Data distribution method and device based on heterogeneous storage

PendingCN113467700ABest access performanceMaximize space utilization
Owner:ALIBABA GRP HLDG LTD

Center floor console

InactiveUS20050035618A1easily storemaximize space utilization
Owner:FREDRICKS DESIGN

Rack realizing adjustability through insertion connection for bowels and dishes

PendingCN107432694Aeasy to useMaximize space utilization
Owner:ZHEJIANG MASON KITCHEN

Classification and recommendation of technical efficacy words

  • save storage space
  • Maximize space utilization

Method of encoding structured low density check code

InactiveCN101141133AGood frame error rate performancesave storage space
Owner:BEIJING UNIV OF POSTS & TELECOMM +1

De-block effect filtering device and method

InactiveCN101409833Asave storage spaceloose timing
Owner:昆山杰得微电子有限公司

Distributed cache method and system

ActiveCN103019960AAvoid redundant storagesave storage space
Owner:浙江杭海新城控股集团有限公司

Center floor console

InactiveUS20050035618A1easily storemaximize space utilization
Owner:FREDRICKS DESIGN

Portable communication apparatus having triple-axis hinge folder and rotation locking device thereof

InactiveUS20050192066A1convenient to usemaximize space utilization
Owner:SAMSUNG ELECTRONICS CO LTD

Interlocking container

ActiveUS7628310B2maximize space utilization
Owner:PAPER SYST

Data distribution method and device based on heterogeneous storage

PendingCN113467700ABest access performanceMaximize space utilization
Owner:ALIBABA GRP HLDG LTD

Rack realizing adjustability through insertion connection for bowels and dishes

PendingCN107432694Aeasy to useMaximize space utilization
Owner:ZHEJIANG MASON KITCHEN
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products