31 results about "Skip list" patented technology

The invention belongs to the technical field of cache prefetching optimization of a firmware layer of a computer storage system, and particularly provides a memory control method, a storage device, a medium and equipment.The instantaneous flow is effectively filtered and persistent hotspot data is accurately recognized through a three-layer structure of combining a first-stage hotspot filtering module with a window cache, a view queue and a protection queue; the second-stage continuous access filtering module dynamically judges a continuous access mode by means of mean-variance statistics of an annular buffer area, and accidental continuous triggering of prefetching is avoided; and the third-stage random access judgment module prohibits invalid prefetching and effectively distinguishes access modes. The pre-fetching bandwidth is limited through the token bucket algorithm, the global priority queue of the skip list is combined, it is ensured that the priority of the write task is higher than that of the pre-fetching task, pre-fetching and foreground I / O resources are isolated, delay jitter control over foreground key I / O can be achieved during full-load pre-fetching, and the problem that delay soaring is caused when pre-fetching preempts the resources is solved.

The invention provides a skip list supporting dynamic real-time data range query, a maintenance method and system and a data range query method and system.The query method comprises the steps that from a high-level index, backward traversal is conducted layer by layer through index nodes, and if the key of a data node corresponding to the current index node is smaller than start, backward traversal continues; if the key of the data node corresponding to the current index node is greater than end, descending to the next layer of index, and if the data link table is the bottommost layer of data link table, traversing the bottom layer of data nodes by utilizing a basic range query process; and if the unique identifier of the data node corresponding to the current index node is between start and end, processing by using a recursive divide-and-conquer process, and finally accumulating query results obtained by recursive divide-and-conquer. The method has the advantages that the statistical information is maintained in the index node, so that the statistical information can be directly used during query, and the query efficiency is greatly improved.

Techniques are disclosed that relate to skip lists. A computer system maintains a skip list having towers of varying depths and entries storing pointers to other towers. A first tower includes an entry at a particular depth storing a pointer to access an entry of a second tower. The pointer includes first similarity information indicating an amount of similarity between a key of the first tower and a key of the second tower. The computer system performs a traversal of the skip list for a search key. The computer system generates second similarity information indicating an amount of similarity between the first tower's key and the search key. Based on a comparison involving the first and second similarity information and without accessing the second tower to obtain information about its key, the computer system determines whether to traverse to the second tower using the pointer or descend the first tower.

The invention discloses a time sequence data retrieval method and device, terminal equipment and a storage medium, and belongs to the field of data retrieve.The method comprises the steps that a time sequence data set is obtained and evenly sampled, and a plurality of check points are obtained; calculating an actual interval between the check points, and judging a data distribution state of the time sequence data set; if the time sequence data set is uniformly distributed, performing interpolation positioning on the time sequence data set according to a target timestamp, and outputting a pointer corresponding to a target data element; and if the time sequence data set is non-uniformly distributed, carrying out secondary sampling on the time sequence data set, constructing a skip list of breadth-first storage layout according to the sampled elements, carrying out tree search, positioning to a target data interval, carrying out binary search, and outputting a pointer corresponding to a target data element. The problems that in the prior art, the whole scene cannot be covered to give play to the optimal performance, and adaptability is poor can be solved.

This application discloses a data block management method, system, and storage medium. The data block management method effectively improves the data block lookup efficiency by using a data block ID query queue, an LRU management queue, and management nodes in combination. Furthermore, by traversing or using the LRU management queue and / or the data block ID query queue in a skip list manner, the performance problems of data movement caused by using arrays are avoided, thereby improving data processing efficiency.

An embodiment provides new joiners to a conversation with access to historical keys using an encrypted skip list that can be safely stored on untrusted servers. This list provides random access to conversation content with logarithmic complexity (queries and data size), and imposes small (log-scale) overhead on linear / full access to the conversation content.

The application relates to a FPGA Jacobian matrix structure reordering method, which comprises the following steps: selecting a node with the minimum metric value from unprocessed nodes in a Jacobian matrix as an optimal pivot, and calculating an optimal pivot variable skip list; for all variables in the current optimal pivot variable skip list, sequentially performing node variable skip list updating calculation, node element skip list updating calculation and node metric value updating calculation; detecting and updating node super variables based on all variables in the current optimal pivot variable skip list; removing the current optimal pivot from the unprocessed nodes, and repeating the above steps until all nodes of the Jacobian matrix are covered. The application can improve the response speed and effectiveness of power flow calculation.

The invention relates to an FPGA power flow Jacobian matrix bidirectional skip list incremental calculation method. The method comprises the steps that a Jacobian matrix non-zero element structure skip list is designed; reading a binary power grid branch data sequence, and splicing the read binary power grid branch data sequence by adopting a shift register to obtain binary power grid branch data; and calculating the non-zero element structure of the Jacobian matrix according to the binary power grid branch data and the Jacobian matrix non-zero element structure skip list dynamic increment. According to the method, the calculation efficiency of the Jacobian matrix can be improved.

This invention relates to a dynamic bidirectional skip list storage structure and processing method for matrix data sets. The dynamic bidirectional skip list storage structure includes a data field at the bottom and an index field above the data field. The data field stores the original data, and the index field stores index data. The data field includes multiple data unit structures at the same level, and the index field includes multiple levels, each level containing multiple data unit structures. Data unit structures within the same level are linked from smallest to largest using a doubly linked list, and data unit structures at different levels are linked to the same data range using inter-level linked lists. Based on this dynamic bidirectional skip list storage structure, the calculation processes of dataset merging, difference, searching, and deletion during the optimal reordering of the power flow Jacobian matrix can be efficiently implemented, thus achieving a balance between storage and computational efficiency.

The invention relates to the technical field of big data storage and processing, and discloses a storage scheduling and analysis processing method for high-concurrency Internet of Things data, and the method comprises the steps: firstly building a writing end discrete tension model based on the disorder degree and fluctuation amplitude of a data stream, and building a reading end aggregation tension model based on a query intention and access popularity; secondly, substituting the two-way tension value into a tension field model, dynamically selecting and instantiating a row skip list, a column prepolymerization array or a sparse index aggregation tree as a memory table structure; when data is written into a disk, frequency domain transformation coding is performed on periodic data, and a heterogeneous sorting character string table is generated; and finally, performing frequency domain calculation in combination with the Parseval theorem in the query process, and dynamically correcting the tension weight by using performance feedback data. Through structure variation and frequency domain compression during operation, the writing performance and the query efficiency are effectively balanced, and the storage cost is reduced.

The present disclosure provides a page jump method and device, equipment and storage medium, which relates to the technical field of internet technology, application processing, page display, data storage and other technical fields in the computer technology field. The page jump method comprises: determining the page number of a to-be-jumped page; determining the logical serial number of the starting node of the to-be-jumped page in the skip list according to the page number, the skip list comprising a plurality of nodes, at least one node in the plurality of nodes being a node corresponding to the to-be-jumped page, the data in the node containing page data and a jump auxiliary value, the jump auxiliary value contained in the node being used to reflect the number of nodes located before the node in the skip list; finding the starting node in the skip list according to the logical serial number and the jump auxiliary value contained in the node; obtaining the page data of the to-be-jumped page from the starting node; and jumping to the to-be-jumped page according to the page data of the to-be-jumped page. Thus, the node corresponding to the page is quickly found by using the skip list, and the page jump efficiency is improved.

The invention provides a vector database construction method and system, and relates to the technical field of databases. The method comprises the steps of performing normalization processing on an original vector so as to limit a sudden change value and zoom the sudden change value to a unified range; recursive dichotomy is carried out on the value range of each dimension, binary codes are allocated to each interval, and mapping of the intervals and the codes is dynamically recorded through a skip list; alternately combining codes of all dimensions of the normalized vector to generate a global binary string; and finally, taking the binary string and the original vector as a joint primary key, and constructing a B + tree index. According to the method, a high-dimensional proximity problem is converted into a prefix matching problem of a one-dimensional space, and excellent characteristics of a B + tree and a skip list are utilized, so that efficient insertion, deletion, modification, accurate query and fuzzy query of high-dimensional vector data are realized; the problems of complicated calculation, high storage overhead, difficulty in dynamic updating and low query speed of the traditional method on high-dimensional data are effectively solved.

The application belongs to the technical field of cache prefetch optimization of the firmware layer of a computer storage system, and specifically provides a memory control method, a storage device, a medium and equipment, through a three-layer structure of a primary hot spot filtering module combined with a window cache, an apprentice queue and a protection queue, transient traffic is effectively filtered, and persistent hot spot data is accurately identified; a secondary continuous access filtering module uses mean-variance statistics of a ring buffer to dynamically determine a continuous access mode, and avoids accidental continuous triggering of prefetch; a tertiary random access judgment module prohibits invalid prefetch, and effectively distinguishes access modes. The application limits prefetch bandwidth through a token bucket algorithm, and combines a skip list global priority queue to ensure that the priority of a write task is higher than that of a prefetch task, so that resources of prefetch and foreground I / O are isolated, when full-load prefetch is performed, foreground critical I / O delay jitter can be controlled, and the problem of delay soaring caused by resource preemption of prefetch is avoided.

The present application relates to a kind of optimal column ordering method and device of tide Jacobian matrix, wherein, method includes: the maximum non-zero element number of optimal pivot column in lower triangular matrix in Jacobian matrix decomposition process is calculated, and the non-zero element upper bound of optimal pivot column is obtained;With the same non-zero element structure of optimal pivot column, pivot row mode skip list and original row mode skip list are updated to empty set;The numerical value of each column number value corresponding to row mode counter in the non-zero element upper bound mode skip list of pivot row of the column number value pointed to by the column number value of optimal pivot column is updated, and the numerical value of pivot row mode counter and original row mode counter is obtained;The column metric value of each column number value in the non-zero element upper bound mode skip list of pivot row of the column number value pointed to by the column number value of optimal pivot column is updated and calculated.The present application can improve the response speed and effectiveness of power flow calculation.

This invention provides a method and system for incremental time-series data updates using adaptive hierarchical indexes, belonging to the field of database indexing technology. The method includes: constructing a three-level hierarchical index structure comprising a hot layer, a warm layer, and a cold layer; the hot layer using an in-memory skip list, the warm layer using an LSM tree, and the cold layer using a segmented B+ tree; configuring an incremental merge scheduler to dynamically trigger inter-layer data sinking and index merging based on data volume, query frequency, and write rate; employing a lock-free concurrency strategy to avoid merging blocking foreground read / write operations; embedding time-series feature summaries in index nodes to implement query pruning; and introducing a pre-write log and checkpoint mechanism to ensure atomicity and crash recovery. This invention can maintain stable query latency under continuous high-throughput write conditions.

The application can be used in the technical field of big data technology application in finance, and provides a non-relational database generation and query method and device based on skip list data, the non-relational database generation method based on skip list data comprises the following steps: generating a query index array according to a plurality of historical query expressions of a relational source database; determining data corresponding to the query index array in the relational source database; and generating a non-relational database according to the data corresponding to the query index array and a skip list model.The non-relational database generation and query method and device based on skip list data provided by the application make full use of the characteristics of the non-relational database, and under the premise of guaranteeing efficient query and sacrificing only a small part of local storage space, the specific interface is used to complete fast retrieval, classification, range query and other operations on the local dump data set, so that the data analysis and other tasks can be efficiently completed.

The invention discloses a transaction data convergence processing method and device, electronic equipment and a storage medium, and relates to the technical field of databases. The method comprises the steps of dividing a log file into a plurality of continuous and non-overlapped log segments based on key parameters of the log file of an obtained source end database, and distributing segment numbers for the log segments; submitting the log segments with segment numbers to a multi-thread scheduling component, scheduling a plurality of independent threads by the multi-thread scheduling component, and performing data pulling on each log segment in parallel; when each independent thread pulls one transaction record in the log segment, converging the pulled transaction record in the constructed transaction convergence table according to segment attribution information or transaction life cycle information of the transaction record; and in response to a table scanning event for the transaction convergence table, scanning a transverse skip table and a longitudinal skip table of the transaction convergence table, forming transaction records meeting a preset transaction integrity condition into complete transaction data, and sending the complete transaction data to a downstream analysis disk.

The application provides a processing method and device of a skip list based on range partitioning, which can be used in the financial field or other technical fields. The method comprises the following steps: obtaining an operation request for a target element, wherein the request comprises information of the target element; determining a partition in which the target element is located according to the information of the target element, wherein each partition is divided according to the range of element key values; and performing a corresponding operation on a skip list corresponding to the partition by using the information of the target element according to the request type of the operation request. The processing method and device of the skip list based on range partitioning provided in the application embodiment can solve the problem that the skip list is blocked for a long time when updating the skip list due to the large capacity of the skip list.

The present application relates to big data processing and edge computing technical field, specifically to information big data management system based on edge computing, including mutation detection module, skip list management module and decision execution module, the mutation detection module carries out "noise-silence" dual-state detection to big data stream in edge node, only intercepts mutation micro-slice at state conversion and calculates 128 bit SimHash mutation fingerprint; the skip list management module weaves the "mutation fingerprint-physical block" bidirectional skip list with reversible fuse mark on local PCM, realizes the index of mutation fingerprint to physical block address; the decision execution module takes the skip list root handle as the entrance, drives the "write once" type edge inference machine to generate the immutable decision frame, writes back the data source terminal through the low delay channel and updates the fuse state, so as to complete the closed loop processing of cold mutation data.

The invention relates to a kernel data collecting and processing method and device based on multilayer cache optimization, and the method comprises the steps: collecting kernel data from an operating system kernel, and carrying out the deduplication processing of the kernel data; controlling the switching frequency of AB double caches by adopting a time wheel, writing and reading the de-duplicated kernel data through a double-cache mechanism, performing batch processing on the de-duplicated kernel data, and merging to obtain operation data; performing batch processing queue classification and sorting on the operation data through the prefix tree to obtain a sorting result; according to a sorting result, writing the operation data into an annular buffer area with a corresponding priority in the shared memory; and on the basis of a set condition, target data are retrieved in the annular buffer area by adopting a skip list and are sorted, and the target data are exported to a corresponding process according to the service priority. According to the invention, the collection speed and the export speed of the kernel data can be matched, and high-performance and low-disturbance real-time operation data collection and processing are realized while multiple priorities are ensured.

Techniques related to efficiently managing skip list data structures are disclosed. In some embodiments, a computing system stores a skip list comprising a plurality of key-value records, the key-value records comprising one or more pointers to other key-value records in the plurality of key-value records. The computing system scans the plurality of key-value records in order of keys to identify key-value records to purge from the skip list. The scanning includes maintaining a list of key-value records comprising pointers to key-value records that have not yet been scanned by the scanning. In response to identifying a key-value record for purging, the computing system purges the key-value record by replacing pointers included in the key-value records of the list with pointers included in the key-value record that is being purged.

The invention relates to a cost consistency checking method and system for a power transmission and transformation project, and the method comprises the following steps: obtaining cost text data of a power transmission and transformation project, and carrying out the preprocessing of the cost text data, and obtaining a text character string T; the text character string T is retrieved and matched through a multi-mode character string matching algorithm based on a skip list and a double hash table, cost information is obtained, a submitted version cost inventory is further formed, and fields related to equipment materials and called project definition fields are included in the submitted version cost inventory; obtaining an approved-version cost inventory, wherein the submitted-version cost inventory is an approved-version cost inventory after being reviewed and adjusted; and performing consistency checking on the submitted version cost inventory and the approved version cost inventory to complete the checking process. Compared with the prior art, the method has the advantages of being intelligent, accurate in checking and the like.

This invention discloses a high-efficiency blockchain query system and its dynamic construction method based on Gskiplist, belonging to the field of computer science. The Gskiplist-based high-efficiency blockchain query system includes a blockchain subsystem, a "grouped skip list" subsystem, and a database. The blockchain subsystem contains all transactions and blocks, the "grouped skip list" subsystem includes a multi-level Node tree, multiple MetaD node sets, and a parameter file, and the database is used to persistently store all data. The dynamic construction method of the Gskiplist-based high-efficiency blockchain query system includes: reading the parameter file; reading blockchain data and constructing an initially preset "grouped skip list" subsystem; opening a query service externally, while simultaneously listening for new blocks and user query requests; responding to user query requests and dynamically maintaining the structure of the "grouped skip list" subsystem. This invention can effectively accelerate blockchain query efficiency.

The present application relates to the technical field of jump control, and discloses a kind of Badmark point automatic identification and jump control method and system, wherein the method comprises: parsing Gerber file, determining the coordinate range and panel number of each panel;Identify the first panel in each panel Bad board identification template image and calculate the position offset vector;Calculate the array data of the remaining panels except the first panel;Calculate the RGB threshold range of the Badmark region under the good board state and establish the bad board determination interval and the good board determination interval;When the template matching score falls into the bad board determination interval, the corresponding panel number is written into the skip list, the method can automatically calculate the Badmark detection frame coordinates of all remaining panels, without manual calibration by operating personnel one by one panel, ensuring the accuracy and reliability of jump control.

A system includes storage of job data describing a computing job in a job data location, creation of a queue entry associated with the computing job, the queue entry comprising a pointer to the job data location and a subset of the job data, storage of the queue entry in a job scheduler queue of a lock-free skiplist at a position based on a priority of the job, and reading of data from the queue entry and from a plurality of other queue entries of the job scheduler queue.

This specification provides one or more embodiments of a skip list-based data read / write method, system, device, and storage medium. Nodes in the skip list store keys and first characteristic values. The first characteristic value records the fork position of the node's key relative to the key of its predecessor node, and at least a portion of the value of the node's key after the fork position. The method includes: obtaining the target key of the element to be read / written; if the target key is different from the key of the first node in the skip list, determining the next node pointed to by the first node at the highest index level as the comparison node, and repeatedly executing the following process until a preset condition is met: comparing the second characteristic value of the target key with the first characteristic value of the comparison node; if they are different, determining the comparison node for the next iteration based on the size relationship indicated by the comparison result; if the preset condition is met, performing read / write operations on the element to be read / written based on the comparison node from the last iteration. This method helps reduce comparison overhead.

This invention relates to an incremental calculation method for a bidirectional skip list of the Jacobian matrix in FPGA power flow, comprising: designing a skip list of non-zero elements of the Jacobian matrix; reading binary power grid branch data sequences, concatenating the read binary power grid branch data sequences using a shift register to obtain binary power grid branch data; and dynamically incrementally calculating the non-zero element structure of the Jacobian matrix based on the binary power grid branch data and the skip list of non-zero elements of the Jacobian matrix. This invention can improve the calculation efficiency of the Jacobian matrix.

The invention relates to a matrix data set dynamic bidirectional skip list storage structure and a processing method, and the structure comprises a data field part located at a bottom layer and an index field part located above the data field part. The data field part is used for storing original data, and the index field part is used for storing index data; the data field part comprises a plurality of data unit structures located at the same hierarchy, the index field part comprises a plurality of hierarchies, and each hierarchy comprises a plurality of data unit structures; wherein the data unit structures in the same hierarchy are linked from small to large through a double linked list, and the data unit structures among different hierarchies are linked with the same data interval range through inter-hierarchy linked lists. Based on the dynamic bidirectional skip list storage structure, calculation processes such as union set, difference set, search and deletion of data sets in the optimal reordering process of the load flow Jacobian matrix can be efficiently realized, so that the balance of storage and calculation efficiency is realized.

The invention relates to a power flow Jacobian matrix optimal column sorting method and device, and the method comprises the steps: calculating the maximum number of non-zero elements of an optimal principal element column in a lower triangular matrix in a Jacobian matrix decomposition process, and obtaining the upper bound of the non-zero elements of the optimal principal element column; updating all principal element row mode skip lists and original row mode skip lists which have the same non-zero element structure as the optimal principal element column into null sets; updating the numerical value of a row mode counter corresponding to each column number value in a principal element row non-zero element upper bound mode skip list pointed by the column number value of the optimal principal element column to obtain the numerical values of a principal element row mode counter and an original row mode counter; and updating and calculating the column metric value of each column number value in the principal element row non-zero element upper bound mode skip list pointed by the column number value of the optimal principal element column. According to the invention, the response speed and effectiveness of load flow calculation can be improved.

The application provides a database index construction method and device, belonging to the technical field of information storage, and the method comprises the following steps: in the case of receiving a data insertion request, inserting the to-be-inserted data into a to-be-inserted position; if the detection result of first detection is that a node of a global index is added, allocating the required address space for the added node based on the address space of a dynamic random access memory; if the detection result of second detection is that the parent node of the added node is located in a nonvolatile memory, inserting the added node into a quick index; if the used space proportion of the dynamic random access memory reaches a preset proportion, migrating the added node from the address space of the dynamic random access memory to the address space of the nonvolatile memory; wherein the global index and the quick index both adopt a skip list data structure. The application adopts the skip list data structure, optimizes the index structure of the in-memory database, and reduces the space consumption of the index on the DRAM under the premise of ensuring high data response speed.