A fast search system and method based on B-Tree data structure

Abstract

The present invention relates to the technical field of highway network data retrieval, and discloses a fast retrieval system and method based on a B-Tree data structure, including a data control center, a retrieval module, a data compression module, a storage hard disk and a cache area. The fast retrieval system and method based on the B-Tree data structure uses the ordered characteristics of the Disk B-Tree data structure and the long-hash algorithm to generate the list parameters of the corresponding data structure, so as to ensure the integrity and accuracy of data retrieval while occupying less memory, and optimize the query efficiency of the list parameters of the toll lane software, solve the problem of excessive memory occupation and long retrieval time caused by the increase of data content such as the list parameters loaded by the highway toll software, and ensure that each list index is loaded into the memory without exceeding 4K, and each list query takes no more than 1 millisecond, thereby improving the traffic efficiency of the toll lane vehicles.

Description

Technical Field

[0001] The present invention relates to the technical field of highway networking data retrieval, and in particular to a fast retrieval system and method based on a B-Tree data structure. Background Art

[0002] When the highway network toll collection system is performing ETC intelligent toll collection, it can identify the license plates of vehicles at the entrance of the lane, query the blacklist vehicles through data retrieval, and monitor the special behaviors and abnormal events of vehicles in real time. After the cancellation of provincial boundary toll stations, the highway network toll collection system has become a development trend. The system reduces the time-consuming manual confirmation operation and gives full play to the advantages of the intelligent highway monitoring and toll collection system. Among them, the fast retrieval method based on the Disk B-Tree data structure can effectively reduce the retrieval time and the list index loading memory when searching the software list directory, helping the system to quickly and accurately identify blacklist vehicles. Its application has greatly improved the accuracy of abnormal payment behavior inspection and the efficiency of toll station traffic.

[0003] At present, there are few precedents for applying Disk B-Tree data structure to highway traffic data retrieval at home and abroad. In the context of abolishing provincial boundary toll stations, the retrieval system based on traditional database is difficult to support the query and access of huge data volumes. Some methods have optimized the database structure and adopted a more efficient data retrieval platform, which has improved the efficiency of data query and access to a certain extent. However, such methods have high requirements on hardware performance, and the index loading process requires sufficient memory support, which affects the integrity and accuracy of data retrieval when a single point failure occurs in the database.

[0004] Therefore, how to ensure data retrieval efficiency when the data content such as the list parameters loaded by the paid software increases dramatically, reduce the requirements for hardware equipment for the use of the retrieval system, and ensure the accuracy of the data retrieval results has become a technical problem that researchers in this field need to solve urgently. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the deficiencies in the prior art, the present invention provides a rapid retrieval system and method based on a B-Tree data structure, which has the advantages of solving the problems of low data retrieval efficiency and high latency in the process of highway traffic data retrieval, and solves the above-mentioned technical problems.

[0007] (II) Technical solution

[0008] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a rapid retrieval system based on a B-Tree data structure, comprising a data control center for sending highway data to various ETC toll stations and receiving feedback information from the ETC toll stations, a retrieval module for searching and querying blacklisted vehicles, a data compression module for compressing the highway data sent by the data control center, and a storage hard disk for storing compressed data and storing an updated database.

[0009] As a preferred technical solution of the present invention, the data control center collects daily highway data and sends it to each ETC toll station to provide a data support source for data retrieval.

[0010] As a preferred technical solution of the present invention, the internal storage structure of the retrieval module is a Disk B-Tree data structure, in which one data block has a multi-branch structure, one data block contains multiple value records, all LeaF nodes form a bidirectional linked list, and the LeaF nodes are used for pointing when traversing the array.

[0011] As a preferred technical solution of the present invention, the retrieval module queries data based on the internal storage structure, and the query steps are as follows: first, starting from the root node of the internal storage structure, compare the query value with the key value between the nodes, and then, based on the comparison result, decide to move from the current node to the left subtree or the right subtree, and repeat the above steps again until a matching node is found or it is determined that there is no matching node, and finally return the query result.

[0012] As a preferred technical solution of the present invention, the data compression module adopts a long-hash algorithm to generate a list parameter for the data sent by the data control center and compress the data.

[0013] As a preferred technical solution of the present invention, the storage bytes of the data compression module are 64 bits.

[0014] As a preferred technical solution of the present invention, the data compression module stores the highway data sent by the data control center in a storage hard disk so as not to occupy the memory running space.

[0015] As a preferred technical solution of the present invention, when the ETC toll station detects that the vehicle information has not been entered, the data is fed back to the data control center, the new data is stored in the storage hard disk, and the retrieval module adopts a concurrent query mode when searching.

[0016] As a preferred technical solution of the present invention, it also includes a cache area. When the data control center detects that data is frequently called, the data is transferred from the storage hard disk to the cache area for IO saving.

[0017] The present invention provides a fast retrieval method based on a B-Tree data structure. The fast retrieval system based on a B-Tree data structure comprises the following steps:

[0018] S1. The data control center sends highway-related data to each ETC toll station to ensure data synchronization;

[0019] S2, the data compression module compresses the highway-related data into 64-bit ordered data through the long-hash algorithm and places it in the first library of the storage hard disk;

[0020] S3. After the vehicle enters the ETC station, the data is scanned through the retrieval module. The retrieval module calls the data of the first library of the hard disk through the Disk B-Tree data structure for comparison;

[0021] S4. If the vehicle information already exists in the first library, execute S7, otherwise execute S5;

[0022] S5. If the vehicle information does not exist in the first library, the vehicle is intercepted and the subsequent libraries are searched in sequence. If the vehicle information does not exist in any of the libraries, a new library is created to store the vehicle data.

[0023] S6, determining whether the same vehicle information in a certain area is called and retrieved by the search module multiple times, and if so, retrieving the vehicle data into a cache area, which is located in the memory;

[0024] S7, the vehicle passes and scans the next vehicle

[0025] Compared with the prior art, the present invention provides a fast retrieval system and method based on the B-Tree data structure, which has the following beneficial effects:

[0026] The present invention uses the ordered characteristics of the Disk B-Tree data structure and the long-hash algorithm to generate the list parameters of the corresponding data structure, so as to ensure the integrity and accuracy of data retrieval while occupying less memory, and at the same time optimizes the query efficiency of the list parameters of the toll lane software, and solves the problem of excessive database memory occupation and long retrieval time caused by the increase of data content such as the list parameters loaded by the highway toll software. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the Disk B-tree data structure of the present invention;

[0028] Figure 2 This is a schematic diagram of the data storage structure of the present invention;

[0029] Figure 3A schematic diagram of the principle of the data structure of the present invention is formed;

[0030] Figure 4 A schematic diagram of the principle of the data structure of the present invention is formed;

[0031] Figure 5 This is a schematic diagram of the data storage process of the present invention;

[0032] Figure 6 It is a schematic diagram of the file header structure of the present invention.

[0033] Figure 7 This is a schematic diagram of the basic node structure of the present invention;

[0034] Figure 8 This is a schematic diagram of the leaf node structure of the present invention;

[0035] Fig. 9 It is a schematic diagram of the data file structure of the present invention;

[0036] Fig.10 It is a schematic diagram of the system flow of the present invention;

[0037] Fig.11 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figure 1-11 A fast retrieval system based on a B-Tree data structure includes a data control center for sending highway data to various ETC toll stations and receiving feedback information from the ETC toll stations, a retrieval module for searching and querying blacklisted vehicles, a data compression module for compressing the highway data sent by the data control center, and a storage hard disk for storing compressed data and storing updated databases.

[0040] Furthermore, the data control center collects daily highway data and sends it to each ETC toll station to provide a data support source for data retrieval.

[0041] Furthermore, the internal storage structure of the retrieval module is a Disk B-Tree data structure, in which a data block has a multi-branch structure, a data block contains multiple value records, and all LeaF nodes form a bidirectional linked list. The LeaF nodes are used to point when traversing the array.

[0042] Furthermore, the retrieval module queries data based on the internal storage structure, and the query steps are as follows: first, starting from the root node, compare the query value with the key value between the nodes, and then decide to move to the left or right subtree based on the comparison result, and repeat the above steps again until a matching node is found or it is determined that there is no matching node, and finally return the query result.

[0043] Furthermore, the data compression module uses a long-hash algorithm to generate a list parameter for the data sent by the data control center and compresses the data.

[0044] Furthermore, the storage bytes of the data compression module are 64 bits.

[0045] Furthermore, the data compression module stores the highway data sent by the data control center in a storage hard disk so as not to occupy the memory running space.

[0046] Furthermore, when the ETC toll station detects that the vehicle information has not been entered, the data is fed back to the data control center, and the new data is stored in the storage hard disk. The retrieval module adopts a concurrent query mode when searching.

[0047] Furthermore, it also includes a cache area. When the data control center detects that data is frequently called, the data is transferred from the storage hard disk to the cache area for IO saving.

[0048] like Figure 1 The improved Disk B-Tree shown in the figure is a multi-branch structure. A data block can have multiple branches, and a data block contains multiple value records. At the same time, all Leaf nodes form a bidirectional linked list. Figure 2 The file header definition file format shown is 4096. This system retrieves and verifies the underlying file data through an ordered hash index in the hard disk, such as Figure 6 The file header uses 4k of space to store basic information about the file, such as software version, starting node address, and other non-critical attributes. Figure 7 The basic node shown uses 4k space to store hash tables, page number tables, and other non-key attribute tables. The hash table is the intermediate value of all keys and is used to locate child nodes. The page number table is the page number of the child node, which is used for addressing. Other non-key attribute tables are other information of the child node, such as data size, data page number, etc. Figure 8The leaf node shown uses 4k space to store a hash table, a page number table, a table of other non-key attributes, and a data block. The hash table is the full key of the branch and is used for matching queries. The page number table is the page number of the data block, and the data block is where the actual data is stored. The other non-key attribute table is some additional information, such as the data offset address, the next data page number, etc. Fig. 9 The data file shown uses 4k space to store data, the next data page number, offset address, etc. Data is the actual content, such as text, pictures, etc. The next data page number is the page number pointing to the next data page, and if there is none, it is 0. The offset address is the location of the data in the data page and is used for positioning.

[0049] The present invention provides a fast retrieval method based on a B-Tree data structure. The fast retrieval system based on a B-Tree data structure comprises the following steps:

[0050] S1. The data control center sends highway-related data to each ETC toll station to ensure data synchronization;

[0051] S2, the data compression module compresses the highway-related data into 64-bit ordered data through the long-hash algorithm and places it in the first library of the storage hard disk;

[0052] S3. After the vehicle enters the ETC station, the data is scanned through the retrieval module. The retrieval module calls the data of the first library of the hard disk through the Disk B-Tree data structure for comparison;

[0053] S4. If the vehicle information already exists in the first library, execute S7, otherwise execute S5;

[0054] S5. If the vehicle information does not exist in the first library, the vehicle is intercepted and the subsequent libraries are searched in sequence. If the vehicle information does not exist in any of the libraries, a new library is created to store the vehicle data.

[0055] S6, determining whether the same vehicle information in a certain area is called and retrieved by the search module multiple times, and if so, retrieving the vehicle data into a cache area, which is located in the memory;

[0056] S7, the vehicle passes and scans the next vehicle.

[0057] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fast retrieval system based on B-Tree data structure, characterized by: It includes a data control center for sending highway data to various ETC toll stations and receiving feedback information from ETC toll stations, a retrieval module for searching and querying blacklisted vehicles, a data compression module for compressing highway data sent by the data control center, and a storage hard disk for storing compressed data and storing updated databases; The data control center collects daily highway data and sends it to each ETC toll station to provide a data support source for data retrieval; The internal storage structure of the retrieval module is a Disk B-Tree data structure, in which a data block has a multi-branch structure, a data block contains multiple value records, and all LeaF nodes form a bidirectional linked list. The LeaF nodes are used to point when traversing the array; The retrieval module searches for data based on the internal storage structure, and the query steps are as follows: first, starting from the root node of the internal storage structure, compare the query value with the key value between the nodes, and then, according to the comparison result, decide to move from the current node to the left subtree or the right subtree, and repeat the above steps again until a matching node is found or it is determined that there is no matching node, and finally return the query result; The retrieval module is used to call the data in the storage hard disk for comparison with the vehicles entering the ETC toll station. If the vehicle information does not exist in the first library of the storage hard disk, the vehicle is intercepted and the libraries at the rear positions are searched in sequence. If the vehicle information does not exist in any of them, a new library is created to store the vehicle data. If the vehicle information already exists in the first library, the vehicle passes and scans the next vehicle. At the same time, it is determined whether the same vehicle information in a certain area is called and retrieved by the retrieval module multiple times. If so, the vehicle data is retrieved to the cache area, which is located in the memory. The data compression module uses the long-hash algorithm to generate list parameters for the data sent by the data control center and compress the data; The storage bytes of the data compression module are 64 bits; The data compression module stores the highway data sent by the data control center into the storage hard disk so as not to occupy the memory operation space; When the ETC toll station detects that the vehicle information has not been entered, the data is fed back to the data control center, and the new data is stored in the storage hard disk. The retrieval module uses the concurrent query mode when searching; It also includes a cache area. When the data control center detects that data is frequently called, the data is transferred from the storage hard disk to the cache area for IO saving.

2. A fast search method based on a B-Tree data structure, comprising a fast search system based on a B-Tree data structure as claimed in claim 1, characterized in that: The following steps are involved: S1. The data control center sends highway-related data to each ETC toll station to ensure data synchronization; S2, the data compression module compresses the highway-related data into 64-bit ordered data through the long-hash algorithm and places it in the first library of the storage hard disk; S3. After the vehicle enters the ETC station, the data is scanned through the retrieval module. The retrieval module calls the data of the first library of the hard disk through the Disk B-Tree data structure for comparison; S4. If the vehicle information already exists in the first library, execute S7, otherwise execute S5; S5. If the vehicle information does not exist in the first library, the vehicle is intercepted and the subsequent libraries are searched in sequence. If the vehicle information does not exist in any of the libraries, a new library is created to store the vehicle data. S6, determining whether the same vehicle information in a certain area is called and retrieved by the search module multiple times, and if so, retrieving the vehicle data into a cache area, which is located in the memory; S7, the vehicle passes and scans the next vehicle.

Images