59 results about "Skip list" patented technology

The invention relates to the technical field of data processing, in particular to an intelligent document retrieval and generation system based on metadata driving, which comprises a metadata extraction module, a dynamic index construction module, an intelligent routing module, a distributed retrieval module and a result generation module. The metadata extraction module extracts organization names, timestamps and technical classification tags from an input document, maps the organization names, the timestamps and the technical classification tags into structured key value pairs, and transmits the structured key value pairs to the dynamic index construction module through a streaming pipeline; and the dynamic index construction module constructs a skip list index based on discrete fields to realize rapid matching, fuses metadata features and document semantics by adopting a multi-head attention mechanism, and outputs a structured response through topic clustering. According to the method, through a metadata pre-filtering, path optimization and semantic analysis cooperation mechanism, the multi-dimensional composite query efficiency in a distributed environment is improved, and noise interference is suppressed.

The invention discloses an RDMA Chain-based LSM Tree key value type database access method. The method comprises the following steps: a database client sends a request containing a version number, a key and an operation type to a database server main node; the server main node analyzes the request and processes the request according to the operation type; during write operation, a new serial number is generated, and data is inserted into a skip list of a memory table MemTable; when reading operation is carried out and the version number is smaller than 0, whether a key exists or not is checked through a Bloom filter, if yes, query is carried out at a secondary node through the RDMA technology, and if not, an SSTable file in a disk is directly accessed; and when the reading operation is performed and the version number is greater than or equal to 0, preferentially searching the data of the corresponding version in the MemTable, and if the data of the corresponding version fails, querying the SSTable. Through the RDMA technology, CPU occupation and network round-trip time are reduced, and the response speed and concurrent processing capacity of the system are improved.

The invention belongs to the technical field of cache prefetching optimization of a firmware layer of a computer storage system, and particularly provides a memory control method, a storage device, a medium and equipment.The instantaneous flow is effectively filtered and persistent hotspot data is accurately recognized through a three-layer structure of combining a first-stage hotspot filtering module with a window cache, a view queue and a protection queue; the second-stage continuous access filtering module dynamically judges a continuous access mode by means of mean-variance statistics of an annular buffer area, and accidental continuous triggering of prefetching is avoided; and the third-stage random access judgment module prohibits invalid prefetching and effectively distinguishes access modes. The pre-fetching bandwidth is limited through the token bucket algorithm, the global priority queue of the skip list is combined, it is ensured that the priority of the write task is higher than that of the pre-fetching task, pre-fetching and foreground I / O resources are isolated, delay jitter control over foreground key I / O can be achieved during full-load pre-fetching, and the problem that delay soaring is caused when pre-fetching preempts the resources is solved.

The invention relates to the technical field of databases, and discloses a time sequence database processing method and system based on artificial intelligence. A data acquisition module is used for acquiring multi-source heterogeneous time sequence data, and a data processing module adopts a skip list structure and an out-of-order optimization technology; the data storage module is used for optimizing fragmentation storage and compression through hardware acceleration, the query analysis module is used for multi-dimensional query of a user-defined time window, and the anomaly detection module is used for detecting an abnormal mode in time series data through a machine learning algorithm. And the data indexing module is used for generating an index based on a timestamp and a multi-dimensional condition. According to the method, real-time processing and storage of the time series data can be achieved under the conditions of high concurrency and large data volume, and by using a skip list structure and an out-of-order optimization technology, the system can still keep efficient data writing efficiency under the out-of-order and cut-off environments; and the effect of the requirement for real-time access and storage of mass data in multiple scenes is achieved.

An embodiment provides new joiners to a conversation with access to historical keys using an encrypted skip list that can be safely stored on untrusted servers. This list provides random access to conversation content with logarithmic complexity (queries and data size), and imposes small (log-scale) overhead on linear / full access to the conversation content.

The present application relates to the technical field of data processing, and more particularly to an intelligent document retrieval and generation system based on metadata driving, comprising a metadata extraction module, a dynamic index construction module, an intelligent routing module, a distributed retrieval module and a result generation module. The metadata extraction module extracts the agency name, timestamp and technical classification label from the input document and maps them into structured key-value pairs, which are transmitted to the dynamic index construction module through a streaming pipeline. The dynamic index construction module constructs a skip list index based on discrete fields to achieve fast matching, adopts a multi-head attention mechanism to fuse metadata features and document semantics, and outputs structured responses through topic clustering. The present application improves the efficiency of multi-dimensional compound queries in a distributed environment and suppresses noise interference through a metadata pre-filtering, path optimization and semantic analysis collaborative mechanism.

The invention provides a skip list supporting dynamic real-time data range query, a maintenance method and system and a data range query method and system.The query method comprises the steps that from a high-level index, backward traversal is conducted layer by layer through index nodes, and if the key of a data node corresponding to the current index node is smaller than start, backward traversal continues; if the key of the data node corresponding to the current index node is greater than end, descending to the next layer of index, and if the data link table is the bottommost layer of data link table, traversing the bottom layer of data nodes by utilizing a basic range query process; and if the unique identifier of the data node corresponding to the current index node is between start and end, processing by using a recursive divide-and-conquer process, and finally accumulating query results obtained by recursive divide-and-conquer. The method has the advantages that the statistical information is maintained in the index node, so that the statistical information can be directly used during query, and the query efficiency is greatly improved.

The invention discloses a data backup method and device based on a skip list, and the method comprises the steps: carrying out the partitioning operation of initial backup data, and obtaining a plurality of initial backup data blocks; calculating a hash value of each initial backup data block, storing each initial backup data block, and generating storage description information of each initial backup data block; constructing skip list index information based on the hash value of each initial backup data block and the storage description information; wherein the skip list index information is composed of index nodes organized in a skip list form, and each index node takes a hash value of the corresponding initial backup data block as a sorting key and stores storage description information of the corresponding initial backup data block; and performing incremental backup operation on the updated backup data based on the skip list index information. According to the technical scheme, the backup efficiency and the utilization rate of the backup storage space are improved, and meanwhile the reliability, accuracy and operation convenience of data backup and recovery are guaranteed.

The invention relates to the technical field of springboard control, and discloses a jointed board Badmark point automatic identification and springboard control method and system, and the method comprises the steps: analyzing a Gerber file, and determining the coordinate range and the number of each jointed board; identifying a bad board identification template image of a first jointed board in the jointed boards and calculating a position offset vector; array data of the remaining jointed boards except the first jointed board is calculated; calculating an RGB threshold range of a Badmark area in a good board state, and establishing a bad board judgment interval and a good board judgment interval; when the template matching score falls into the bad board judgment interval, the corresponding jointed board number is written into the skipping list.According to the method, the Badmark detection frame coordinates of all the remaining jointed boards can be automatically calculated, an operator does not need to manually calibrate the jointed boards one by one, and the accuracy and reliability of springboard control are ensured.

Techniques are disclosed that relate to skip lists. A computer system maintains a skip list having towers of varying depths and entries storing pointers to other towers. A first tower includes an entry at a particular depth storing a pointer to access an entry of a second tower. The pointer includes first similarity information indicating an amount of similarity between a key of the first tower and a key of the second tower. The computer system performs a traversal of the skip list for a search key. The computer system generates second similarity information indicating an amount of similarity between the first tower's key and the search key. Based on a comparison involving the first and second similarity information and without accessing the second tower to obtain information about its key, the computer system determines whether to traverse to the second tower using the pointer or descend the first tower.

The present invention discloses a skip list structure based on persistent memory and an access method thereof, allocating persistent memory to the bottom-level linked list of the skip list, and persisting the bottom-level linked list of the skip list; designing a skip list structure optimized for CPU cache; inserting data into the skip list based on persistent memory; deleting data from the skip list based on persistent memory; and managing the free space of persistent memory. Through the present invention, when inserting, searching, and deleting operations are performed in the skip list based on persistent memory, the CPU cache and persistent memory hardware characteristics can be fully utilized, thereby improving the performance of the operation.

The invention discloses a time sequence data retrieval method and device, terminal equipment and a storage medium, and belongs to the field of data retrieve.The method comprises the steps that a time sequence data set is obtained and evenly sampled, and a plurality of check points are obtained; calculating an actual interval between the check points, and judging a data distribution state of the time sequence data set; if the time sequence data set is uniformly distributed, performing interpolation positioning on the time sequence data set according to a target timestamp, and outputting a pointer corresponding to a target data element; and if the time sequence data set is non-uniformly distributed, carrying out secondary sampling on the time sequence data set, constructing a skip list of breadth-first storage layout according to the sampled elements, carrying out tree search, positioning to a target data interval, carrying out binary search, and outputting a pointer corresponding to a target data element. The problems that in the prior art, the whole scene cannot be covered to give play to the optimal performance, and adaptability is poor can be solved.

Embodiments of the present disclosure provide a data access and index establishment method, a device, a storage medium and a program. The method comprises: on the basis of index layer numbers of a plurality of data nodes arranged in sequence, generating a skip list comprising a plurality of index linked lists. One index linked list comprises a plurality of pieces of index information arranged according to the sequence of corresponding data nodes, and the index information comprises index feature values and a position identifier of a successor data node. The index feature value corresponding to a predecessor data node comprises the length of a common prefix character string of the predecessor data node and the successor data node. During querying of target data, a first index feature value corresponding to a first data node for comparison with the target data and a second index feature value of the target data relative to the first data node are determined, and the target data is queried on the basis of a comparison result between the second index feature value and the first index feature value. On the basis of the design of the skip list structure, the data query overhead can be reduced, and the query efficiency is improved.

This application discloses a data block management method, system, and storage medium. The data block management method effectively improves the data block lookup efficiency by using a data block ID query queue, an LRU management queue, and management nodes in combination. Furthermore, by traversing or using the LRU management queue and / or the data block ID query queue in a skip list manner, the performance problems of data movement caused by using arrays are avoided, thereby improving data processing efficiency.

An embodiment provides new joiners to a conversation with access to historical keys using an encrypted skip list that can be safely stored on untrusted servers. This list provides random access to conversation content with logarithmic complexity (queries and data size), and imposes small (log-scale) overhead on linear / full access to the conversation content.

The application relates to a FPGA Jacobian matrix structure reordering method, which comprises the following steps: selecting a node with the minimum metric value from unprocessed nodes in a Jacobian matrix as an optimal pivot, and calculating an optimal pivot variable skip list; for all variables in the current optimal pivot variable skip list, sequentially performing node variable skip list updating calculation, node element skip list updating calculation and node metric value updating calculation; detecting and updating node super variables based on all variables in the current optimal pivot variable skip list; removing the current optimal pivot from the unprocessed nodes, and repeating the above steps until all nodes of the Jacobian matrix are covered. The application can improve the response speed and effectiveness of power flow calculation.

The invention relates to an FPGA power flow Jacobian matrix bidirectional skip list incremental calculation method. The method comprises the steps that a Jacobian matrix non-zero element structure skip list is designed; reading a binary power grid branch data sequence, and splicing the read binary power grid branch data sequence by adopting a shift register to obtain binary power grid branch data; and calculating the non-zero element structure of the Jacobian matrix according to the binary power grid branch data and the Jacobian matrix non-zero element structure skip list dynamic increment. According to the method, the calculation efficiency of the Jacobian matrix can be improved.

The invention discloses a log structure merging tree compression state scanning method based on an order-preserving dictionary, and belongs to the field of data compression. The method comprises the following steps: receiving new data write-in based on a memory line memory storage structure of a skip list, and inserting line memory new data into a memory list according to a key of the write-in data; when the space allocated to the memory table reaches a set threshold value, performing key-value separation row-to-column conversion on the memory data, compressing the data by using an order-preserving dictionary, storing the compressed data into a hierarchical external memory of the LSM tree, and residing the order-preserving dictionary table into the memory; when the levels of the external storage are combined, combining the order-preserving dictionary tables of the corresponding data so as to directly carry out compression state combination on the order-preserving dictionary compressed data of the levels; and when load query is carried out in the external memory, reading the compressed data of the order-preserving dictionary in the external memory into the internal memory by taking a block as a unit, and directly carrying out query calculation on the compressed data based on the order-preserving dictionary table and the query condition. According to the scheme, merging and query scanning can be carried out in a compressed state, disk I / O overhead is reduced, and resource consumption is reduced.

This invention relates to a dynamic bidirectional skip list storage structure and processing method for matrix data sets. The dynamic bidirectional skip list storage structure includes a data field at the bottom and an index field above the data field. The data field stores the original data, and the index field stores index data. The data field includes multiple data unit structures at the same level, and the index field includes multiple levels, each level containing multiple data unit structures. Data unit structures within the same level are linked from smallest to largest using a doubly linked list, and data unit structures at different levels are linked to the same data range using inter-level linked lists. Based on this dynamic bidirectional skip list storage structure, the calculation processes of dataset merging, difference, searching, and deletion during the optimal reordering of the power flow Jacobian matrix can be efficiently implemented, thus achieving a balance between storage and computational efficiency.

The invention relates to the technical field of big data storage and processing, and discloses a storage scheduling and analysis processing method for high-concurrency Internet of Things data, and the method comprises the steps: firstly building a writing end discrete tension model based on the disorder degree and fluctuation amplitude of a data stream, and building a reading end aggregation tension model based on a query intention and access popularity; secondly, substituting the two-way tension value into a tension field model, dynamically selecting and instantiating a row skip list, a column prepolymerization array or a sparse index aggregation tree as a memory table structure; when data is written into a disk, frequency domain transformation coding is performed on periodic data, and a heterogeneous sorting character string table is generated; and finally, performing frequency domain calculation in combination with the Parseval theorem in the query process, and dynamically correcting the tension weight by using performance feedback data. Through structure variation and frequency domain compression during operation, the writing performance and the query efficiency are effectively balanced, and the storage cost is reduced.

The present disclosure provides a page jump method and device, equipment and storage medium, which relates to the technical field of internet technology, application processing, page display, data storage and other technical fields in the computer technology field. The page jump method comprises: determining the page number of a to-be-jumped page; determining the logical serial number of the starting node of the to-be-jumped page in the skip list according to the page number, the skip list comprising a plurality of nodes, at least one node in the plurality of nodes being a node corresponding to the to-be-jumped page, the data in the node containing page data and a jump auxiliary value, the jump auxiliary value contained in the node being used to reflect the number of nodes located before the node in the skip list; finding the starting node in the skip list according to the logical serial number and the jump auxiliary value contained in the node; obtaining the page data of the to-be-jumped page from the starting node; and jumping to the to-be-jumped page according to the page data of the to-be-jumped page. Thus, the node corresponding to the page is quickly found by using the skip list, and the page jump efficiency is improved.

The invention provides a vector database construction method and system, and relates to the technical field of databases. The method comprises the steps of performing normalization processing on an original vector so as to limit a sudden change value and zoom the sudden change value to a unified range; recursive dichotomy is carried out on the value range of each dimension, binary codes are allocated to each interval, and mapping of the intervals and the codes is dynamically recorded through a skip list; alternately combining codes of all dimensions of the normalized vector to generate a global binary string; and finally, taking the binary string and the original vector as a joint primary key, and constructing a B + tree index. According to the method, a high-dimensional proximity problem is converted into a prefix matching problem of a one-dimensional space, and excellent characteristics of a B + tree and a skip list are utilized, so that efficient insertion, deletion, modification, accurate query and fuzzy query of high-dimensional vector data are realized; the problems of complicated calculation, high storage overhead, difficulty in dynamic updating and low query speed of the traditional method on high-dimensional data are effectively solved.

The application belongs to the technical field of cache prefetch optimization of the firmware layer of a computer storage system, and specifically provides a memory control method, a storage device, a medium and equipment, through a three-layer structure of a primary hot spot filtering module combined with a window cache, an apprentice queue and a protection queue, transient traffic is effectively filtered, and persistent hot spot data is accurately identified; a secondary continuous access filtering module uses mean-variance statistics of a ring buffer to dynamically determine a continuous access mode, and avoids accidental continuous triggering of prefetch; a tertiary random access judgment module prohibits invalid prefetch, and effectively distinguishes access modes. The application limits prefetch bandwidth through a token bucket algorithm, and combines a skip list global priority queue to ensure that the priority of a write task is higher than that of a prefetch task, so that resources of prefetch and foreground I / O are isolated, when full-load prefetch is performed, foreground critical I / O delay jitter can be controlled, and the problem of delay soaring caused by resource preemption of prefetch is avoided.

The present application relates to a kind of optimal column ordering method and device of tide Jacobian matrix, wherein, method includes: the maximum non-zero element number of optimal pivot column in lower triangular matrix in Jacobian matrix decomposition process is calculated, and the non-zero element upper bound of optimal pivot column is obtained;With the same non-zero element structure of optimal pivot column, pivot row mode skip list and original row mode skip list are updated to empty set;The numerical value of each column number value corresponding to row mode counter in the non-zero element upper bound mode skip list of pivot row of the column number value pointed to by the column number value of optimal pivot column is updated, and the numerical value of pivot row mode counter and original row mode counter is obtained;The column metric value of each column number value in the non-zero element upper bound mode skip list of pivot row of the column number value pointed to by the column number value of optimal pivot column is updated and calculated.The present application can improve the response speed and effectiveness of power flow calculation.

This invention provides a method and system for incremental time-series data updates using adaptive hierarchical indexes, belonging to the field of database indexing technology. The method includes: constructing a three-level hierarchical index structure comprising a hot layer, a warm layer, and a cold layer; the hot layer using an in-memory skip list, the warm layer using an LSM tree, and the cold layer using a segmented B+ tree; configuring an incremental merge scheduler to dynamically trigger inter-layer data sinking and index merging based on data volume, query frequency, and write rate; employing a lock-free concurrency strategy to avoid merging blocking foreground read / write operations; embedding time-series feature summaries in index nodes to implement query pruning; and introducing a pre-write log and checkpoint mechanism to ensure atomicity and crash recovery. This invention can maintain stable query latency under continuous high-throughput write conditions.

Techniques are disclosed that relate to skip lists. A computer system maintains a skip list having towers of varying depths and entries storing pointers to other towers. A first tower includes an entry at a particular depth storing a pointer to access an entry of a second tower. The pointer includes first similarity information indicating an amount of similarity between a key of the first tower and a key of the second tower. The computer system performs a traversal of the skip list for a search key. The computer system generates second similarity information indicating an amount of similarity between the first tower's key and the search key. Based on a comparison involving the first and second similarity information and without accessing the second tower to obtain information about its key, the computer system determines whether to traverse to the second tower using the pointer or descend the first tower.

The application can be used in the technical field of big data technology application in finance, and provides a non-relational database generation and query method and device based on skip list data, the non-relational database generation method based on skip list data comprises the following steps: generating a query index array according to a plurality of historical query expressions of a relational source database; determining data corresponding to the query index array in the relational source database; and generating a non-relational database according to the data corresponding to the query index array and a skip list model.The non-relational database generation and query method and device based on skip list data provided by the application make full use of the characteristics of the non-relational database, and under the premise of guaranteeing efficient query and sacrificing only a small part of local storage space, the specific interface is used to complete fast retrieval, classification, range query and other operations on the local dump data set, so that the data analysis and other tasks can be efficiently completed.

The application discloses a microgrid resource visualized topology index generation method and system, the method comprises the following steps: receiving geographical boundary coordinate data of a microgrid system to establish a microgrid physical area coordinate system; constructing a hexagonal honeycomb grid structure in the microgrid physical area coordinate system; generating a honeycomb grid level hash code according to the honeycomb grid; constructing a skip list chain of a network topology structure and a skip list chain access path cache table based on the hash code; finally, performing cross-level jump retrieval by using the cache table and generating a microgrid resource visualized topology index; by combining the hexagonal honeycomb grid and the skip list chain structure, efficient cross-level retrieval is realized, the speed and accuracy of device positioning in a complex microgrid system are significantly improved, and the fault processing efficiency and operation and maintenance response speed are effectively ensured.

The invention discloses a transaction data convergence processing method and device, electronic equipment and a storage medium, and relates to the technical field of databases. The method comprises the steps of dividing a log file into a plurality of continuous and non-overlapped log segments based on key parameters of the log file of an obtained source end database, and distributing segment numbers for the log segments; submitting the log segments with segment numbers to a multi-thread scheduling component, scheduling a plurality of independent threads by the multi-thread scheduling component, and performing data pulling on each log segment in parallel; when each independent thread pulls one transaction record in the log segment, converging the pulled transaction record in the constructed transaction convergence table according to segment attribution information or transaction life cycle information of the transaction record; and in response to a table scanning event for the transaction convergence table, scanning a transverse skip table and a longitudinal skip table of the transaction convergence table, forming transaction records meeting a preset transaction integrity condition into complete transaction data, and sending the complete transaction data to a downstream analysis disk.

This application discloses a patching method, apparatus, and computer device, applicable to the field of computer technology. In this method, the target patch function required by a first process and its corresponding dependent function skip list are loaded into the memory space of the first process, with the target patch function occupying shared memory space. The addresses of the dependent functions of the target patch function in the first process's memory are obtained, and the jump addresses corresponding to the dependent functions in the dependent function skip list are modified according to the obtained addresses, so that when executing the dependent functions that the target patch function depends on, a jump to the dependent function skip list is initiated, and a jump to the location of the dependent function is initiated according to the jump address. A jump instruction is set in the original function in the first process's memory space, so that when executing the original function, a jump to execute the target patch function is initiated. Because the target patch function is loaded into the shared memory space, other processes can also access it, eliminating the need for each process to load it separately, thus avoiding memory waste.