16 results about "Radix tree" patented technology

The invention relates to a data reading and writing method and device, electronic equipment and a computer storage medium, and belongs to the technical field of data indexing.The data reading and writing method comprises the steps that a key of indexed data is obtained, a preset position prediction function is adopted in a learning index layer to predict the key, and a first storage position corresponding to the key is predicted; when the first storage position does not conflict with the read operation or the write operation of the indexed data, performing the read operation or the write operation on the indexed data at the first storage position; when the first storage position conflicts with the read operation or the write operation of the indexed data, performing adaptive retrieval on the key in an adaptive cardinal number tree layer, and determining a second storage position corresponding to the key; and performing read operation or write operation on the indexed data at the second storage position. The data reading and writing efficiency of the database can be improved.

This invention discloses a general method for calculating the effective domain of forwarding rules, relating to the field of network device configuration technology. The method includes: inserting all forwarding rules into a ternary radix tree based on matching; querying all matches in the ternary radix tree that intersect with forwarding rule M, and determining the forwarding rule corresponding to each match to form a rule set; wherein the priority of forwarding rule M is P; and subtracting the matches corresponding to all forwarding rules with a priority higher than P in the rule set from the matches corresponding to forwarding rule M to obtain the effective domain of forwarding rule M. This calculation method is not only adaptable to prefix-based matching but can also quickly calculate the effective domain under any wildcard pattern, exhibiting good compatibility; the provided ternary radix tree can efficiently support intersection matching queries under any wildcard pattern, while also possessing good compatibility.

The invention relates to a separate memory architecture-oriented swap space cache optimization system and method, and the method comprises the steps: receiving a page swap request sent by a host memory space, and converting the page swap request into a swap space memory access request; a cardinal number tree cache index is generated according to the logic block number in the exchange space memory access request, searching is conducted according to the index, and if searching succeeds, a data block is obtained from a cache space and sent to a host memory space; if the search is not successful, adopting an asynchronous active cache replacement strategy to select a data block from the original cache region, writing the data block back to the remote memory pool, and meanwhile, adopting a dynamic pre-fetching strategy based on historical access to read pre-fetching data from the remote memory, and storing the pre-fetching data into a pre-fetching cache region. On the premise that an existing switching mechanism of an operating system is not changed, the access efficiency of the switching space of the separated memory is improved, the access frequency of the remote memory is reduced, and therefore the memory access time delay of the whole system is reduced, and the I / O performance of the system is improved.

The application discloses an importance-aware large model prefix KV rearrangement method and system, comprising the following steps: judging whether all tokens in each node in the radix tree are important or not according to the calculation results of historical queries and prefixes through an important information acquisition module; reading the data blocks containing K or V of the Imp sequence into the CPU memory through a prefix KV rearrangement module, and then placing the K or V corresponding to the important tokens in a data block and placing the K or V corresponding to the unimportant tokens in other data blocks; and updating the sequence in each radix tree node according to the token order after rearrangement, and placing the important tokens in front and the unimportant tokens at the back through a metadata update module. The application alleviates the read amplification problem, further reduces the data reading amount of the disk, reduces the loading time during prefix KV multiplexing, and thus shortens the time for generating the first token during large model reasoning.

The application relates to the technical field of storage, and provides a request processing method and device, a storage device and a storage medium, the method comprises the following steps: acquiring a base number tree, the base number tree comprises a plurality of leaf nodes, each leaf node comprises a preset number of index values arranged in sequence; if a target leaf node exists in the base number tree, then performing to-be-written data merging according to effective index values in the target leaf node to generate a writing request, the target leaf node is a leaf node with adjacent effective index values, and the effective index values are index values corresponding to to-be-written data; and the writing request is sent to a hard disk, so that the hard disk writes the merged to-be-written data into the hard disk according to the writing request. The application reduces the IO interaction times of the storage device and the hard disk, thereby improving the IO request processing capacity of the storage device.

The embodiment of the invention provides a vector hybrid retrieval method and system, and is applied to the technical field of data processing. The method is applied to a two-layer vector hybrid index (TLHI), and the TLHI comprises a cardinal tree-based index (RTBI) and a hierarchical navigation small world (HNSW) index. Vector data and attribute information corresponding to the to-be-retrieved data are obtained, the attribute information is traversed in sequence through RTBI, and a target partition comprising the attribute information is obtained. And performing vector retrieval on the target partition by using the HNSW index to obtain a retrieval result matched with the vector data. The retrieval result comprises Top K pieces of data similar to the vector data. According to the method, the hybrid retrieval is performed based on the double-layer vector hybrid index TLHI, the first layer of the index uses RTBI to realize efficient attribute filtering, and the second layer of the index uses HNSW to perform vector similarity retrieval, so that the requirements of vector similarity retrieval and attribute filtering can be met at the same time, and the retrieval efficiency of vector hybrid retrieval is improved.

The present invention discloses an index storage method, system, and storage medium. The method comprises: determining a subtree in an original index whose number of key-value pairs is less than a set number, storing the data of each subtree in a corresponding page, and obtaining a corresponding SPRT index. The method can optimize range queries, which radix trees are not good at, and effectively reduce the depth of the tree, thereby comprehensively improving the performance of index operations. A runtime recovery setting is also proposed, so that during runtime, SPRT only needs to persist data and a portion of metadata that is rarely modified, effectively reducing persistence overhead.

The application relates to the technical field of computers, and particularly discloses a multi-cluster resource permission management method and system based on a radix tree. The application realizes delay minimization and consistency guarantee in the permission synchronization process by dynamically adjusting cluster synchronization strategies and monitoring cluster performance in real time, automatically judges the dynamic weight of each cluster by comprehensively considering the network delay, CPU usage, memory occupancy and disk I / O load of each cluster, divides the clusters into a priority synchronization group and a delay synchronization group according to the weight values, so that the permission synchronization request of a cluster with heavy load or large delay is preferentially processed, the lag of permission synchronization is effectively reduced, a hierarchical permission management structure is constructed by using a radix tree model, permission change information can be accurately recorded and synchronized, and the consistency of permission change data in different clusters is ensured through hash verification, and the accuracy and reliability of permission management in a multi-cluster environment are significantly improved.

The invention relates to the technical field of computers, and particularly discloses a multi-cluster resource authority management method and system based on a cardinal number tree. By dynamically adjusting the cluster synchronization strategy and monitoring the cluster performance in real time, delay minimization and consistency guarantee in the authority synchronization process are achieved, by comprehensively considering the network delay, the CPU utilization rate, the memory occupancy rate and the disk I / O load of each cluster, the dynamic weight of the cluster is automatically judged, and the authority synchronization efficiency is improved. The cluster is divided into the priority synchronization group and the delay synchronization group according to the weight value, so that the authority synchronization request of the cluster with heavy load or large delay is preferentially processed, the hysteresis of authority synchronization is effectively reduced, the hierarchical authority management structure is constructed by using the cardinal tree model, authority change information can be accurately recorded and synchronized, and the authority synchronization efficiency is improved. And the consistency of authority change data in different clusters is ensured through Hash verification, so that the accuracy and reliability of authority management in a multi-cluster environment are remarkably improved.

Provided in the embodiments of the present application are a data processing method and a computer device, which can be applied to the field of computer data processing. The computer device runs a first process in a kernel mode and a second process in a user mode, and runs a code segment by means of the first process, so as to perform the following operations: acquiring first information, the first information comprising the second process requesting from the kernel mode a control memory of a first type, a first permission and a first length; if the first permission is a target permission, creating a control memory on the basis of a created dynamic radix tree, so that the second process operates on the control memory on the basis of the first permission; and once the use of the control memory is completed, releasing same. By combining the created control memory with the dynamic radix tree, the present application achieves communication between two processes in a separate state, so that a userspace service operates on the created control memory on the basis of corresponding permissions, thereby ensuring the independence of the userspace service without increasing the kernel complexity, and making the userspace service more flexible and secure.

The application provides a persistent variable radix tree access system supporting lock-free concurrent access, comprising: a data storage module: organizing and managing the persistent storage of prefix nodes, leaf nodes and data of the variable radix tree, and supporting a general variable radix tree read-write access interface; a local access module: receiving read-write requests of local applications on the variable radix tree and putting the requests into a processing queue to wait for processing by a background processing module; the background processing module: processing the read-write requests in the processing queue, updating the prefix nodes and leaf nodes of the variable radix tree, and ensuring that the metadata and data of the variable radix tree are in a consistent state; and a data recovery module: detecting fault consistency after the persistent variable radix tree access system is restarted, and recovering all data in the variable radix tree to a consistent state when the data is inconsistent.

The application provides a configuration method of a Kong gateway. The method comprises: encapsulating a radix tree routing collection function, a radix tree construction function and a radix tree query function; embedding the radix tree routing collection function and the radix tree construction function in an initialization function of the Kong gateway, so that in the initialization process of the Kong gateway, when traversing native routing information in a database, the radix tree routing collection function is used to obtain radix tree routing information corresponding to each native routing information, and then the radix tree construction function is used to obtain a target radix tree according to each radix tree routing information; and replacing routing matching code in a routing query function of the Kong gateway with the radix tree query function, so that when the Kong gateway receives a query request, if no successful matching is found in the cache, the radix tree query function is used to process the query request. The scheme enables the Kong gateway to use a radix tree as a core data structure for routing matching, optimizes memory usage and significantly improves overall performance.

The invention relates to an access control method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: searching in a pre-constructed first cardinal number tree data structure based on a source address of a session first packet to obtain a first target bitmap; searching in a pre-constructed second cardinal number tree data structure based on the destination address of the session first packet to obtain a second target bitmap; performing logic AND operation on the first target bitmap and the second target bitmap to obtain a third target bitmap; and traversing a set of target access control rules indicated by the third target bitmap according to a preset priority sequence, and executing a corresponding action based on the target access control rules under the condition that the currently traversed target access control rules are matched with the transport layer protocol and / or port information of the session first packet. By adopting the method, the session establishment rate of the cloud firewall can be improved.

The invention relates to a data-centered adaptive radix tree hardware accelerator and an acceleration method. The hardware accelerator comprises an on-chip memory, a control unit, a merging unit, a distributor and an operation unit. The control unit is used for sending the operation information to the scanning buffer area; the merging unit is used for reading the prefixes of the keys corresponding to the operation information from the scanning buffer area and distributing the operation information to different buckets according to a mode that the prefixes of the keys are matched with the prefixes of the buckets, so that the same operation information of the ART nodes is merged and combined operation information is formed; sending each bucket to a bucket buffer area; the dispatcher is used for calling buckets with different prefixes from the bucket buffer area and randomly sending the buckets to the idle operation unit; and the operation unit is used for searching each piece of operation information in the bucket according to the key value sequence, and searching a target node corresponding to the operation information by traversing the tree buffer area or indexing the shortcut buffer area, so as to trigger all operations related to the target node in the bucket to be executed at the same time.

The invention discloses a general calculation method for a forwarding rule effective domain, and relates to the technical field of network equipment configuration. The calculation method comprises the following steps: inserting all forwarding rules into a ternary cardinal number tree according to matching; querying all matches intersecting with the forwarding rule M in the ternary cardinal number tree, and determining the forwarding rule corresponding to each match to form a rule set; wherein the priority of the forwarding rule M is P; and sequentially subtracting the matching corresponding to all the forwarding rules with the priorities higher than P in the rule set from the matching corresponding to the forwarding rule M to obtain an effective domain of the forwarding rule M. The calculation method not only adapts to matching in a prefix form, but also can quickly realize effective domain calculation in any wildcard character mode, and has better compatibility; the provided ternary cardinal number tree can efficiently support intersection matching query in any wildcard character mode, and meanwhile has good compatibility.

This invention relates to the field of database systems and storage systems, and in particular to a writing method and apparatus based on a computable storage device. The method includes receiving a write transaction for a database; when the database is under non-write-intensive load, writing the flushed data corresponding to the write transaction to the actual storage area based on a B+ tree index structure; dividing the entire key space of the database into multiple partitions according to the range of keys, and for each partition, storing the key value falling into the partition and a pointer to the actual data in the leaf nodes of an adaptive radix tree, obtaining a multi-instance adaptive radix tree index structure; when the database is under write-intensive load, writing the flushed data corresponding to the write transaction to the virtual storage area of ​​the computable storage device based on the multi-instance adaptive radix tree index structure, and having the computable storage device complete the process of migrating the flushed data from the virtual storage area to the actual storage area. This invention can improve write performance.