173 results about "Key-value database" patented technology

The invention discloses a method and a device for data partitioning in a key-value database. The method comprises the following steps of: resolving partitioning number information and partitioning algorithm information from a pre-stored partitioning information basic list; and running a corresponding partitioning algorithm by a partitioning callback function according to the partitioning algorithm information, wherein the corresponding partitioning algorithm determines information of partitioned sub databases of key-value data to be partitioned according to the partitioning number information and a calculation value; and the calculation value is a value relevant to the key-value data to be partitioned. The partitioning number information and the partitioning algorithm information are recorded into the partitioning information basic list; when the key-value data are subjected to operation such as storage and query, the partitioned sub databases are determined for the key-value data according to contents recorded in the partitioning information basic list; and therefore, the partitioning mode can be flexibly changed by modifying the contents in the partitioning information basic list.

A method for bursting a hash table of a key-value database comprises receiving a key and a value, traversing trie nodes of the key-value database from a root node to a leaf node by recursively dividing the key into a prefix and a suffix, reaching the leaf node, the leaf node being the hash table, determining that the key is not stored in the hash table, determining that the hash table is not able to store the key and the value, removing the hash table, associating a new trie node with a parent trie node of the hash table, associating two or more new hash tables with the new trie node, moving all keys and associated values from the hash table into one of the two or more new hash tables, and inserting the key and the associated value into one of the two or more new hash tables.

An embodiment of the invention discloses a storage method for data in a Key-Value system and a related device. By means of the method and the device, the storage operation efficiency of Value data can be improved. The method comprises steps as follows: whether the data volume of Value data in the Key-Value exceeds the threshold value of data is judged; if the data volume of Value data doesn't exceed the threshold value of data, the Value data is sliced, and M slice contents are obtained; slice information is generated for N slices according to the M slice contents and comprises the number of Value data slices, offset addresses of slice contents in the N slice contents, serial numbers ID of the slice contents in the N slice contents and (M-N) slice contents; and Key data and slice information are stored in an LSM-Tree (the log-structured merge-tree), the N slice contents are stored in a Key-Value database, and the Key data correspond to the slice contents.

The invention discloses a generation method of a pre-join table on a ring-shaped schema database, belonging to the field of distributed data storage. The method comprises the following steps of: establishing a Key-Value database, forming a column value index of a key column and storing the column value index in the database; generating a pre-join table; if an original data table is updated, updating the corresponding pre-join table; and executing data query, and outputting query result. With the pre-join table generation method provided by the invention, the pre-join table can be efficiently generated in the ring-shaped schema Key-Value database, and a quick join query function is provided; by optimizing a computational process, time and space cost for generating the pre-join table is reduced, and calculation speed is accelerated; and simultaneously, query delay is reduced according to an updating and maintaining method, and real-time join query result can be provided.

The invention provides a saving method and system for time series data based on a key value database. The method comprises the steps that S1, original time series data collected by a sensor is obtained, the original time series data comprises a plurality of attributes, and the attributes comprise an equipment number, a sensor number, a sampling time, a receiving time and a measuring value; S2, according to the sampling time, the original time series data is divided into a corresponding time partition, and time partition information of the original time series data is obtained; S3, the multipleattributes and the time partition information are combined to generate target time series data; S4, the equipment number, the sensor number and the time partition information are taken as first levelindexes, the sampling time is taken as a second level index, and the target time series data is saved in the key value database. According to the saving method and system for the time series data based on the key value database, the time partition information is introduced, the method and the system can adapt to different data sizes, data reading of non-query target is reduced, and a better timeseries data query performance is obtained on the basis of assurance of high-performance data writing.

The invention provides a data processing method and data processing device applied to a data warehouse. The data processing method includes: when a first data base is operated, rectifying records of a mirror database in the first database in the data warehouse according to an incremental log corresponding to the operation and then storing the rectified records in a key/value database; fetching latest entries from the key/value database, additionally storing the entries in a first data table of the data warehouse and enabling the first data table to include historical versions of the records of the mirror database; searching data in the first data table. Therefore, timeliness of the data in the data warehouse can be improved.

Embodiments of the present invention provide a method and a system of file access. The method comprises that: a file system driving module receives a first access request sent by an upper application; the file system driving module, according to the corresponding relation between an access interface corresponding to the upper application and an access interface of the key-value database, converts the first access request into a second access request which is suitable for access to the key-value database; the file system driving module sends the second access request to a client of the key-value database; and the client of the key-value database, according to the second access request, searches corresponding file content or directory entry content or file attributes or directory entry attributes from the key-value database which is stored in a server of the key-value database. When the technical schemes in the embodiments are compared with technical schemes of layer-by-layer access of metadata in file systems of POSIX in the prior art, access cost is relatively small and access efficiency is relatively high.

The invention discloses an information persistence and query method and device applied to a key-value database. Aiming at any business entity object, the attribute values of the business entity object are respectively stored in different columns of the same row of a datasheet; a row key is generated based on the attribute values of the business entity object. As information persistence of the database, such as a Hbase, is realized in a column-based storage mode, and according to the scheme, the attribute values corresponding to the same business entity object are stored in different columns of the same row, compared with an information persistence scheme, aiming at a relational database, provided by the prior art, the scheme provided by the embodiment of the invention can be matched with the characteristic, about the storage mode, of the key-value database. The invention also discloses a query method and device aiming at objects.

The invention discloses a method and device for processing geographic information vector data. The method includes the steps of converting the vector data in an external format into vector image layer data, wherein the vector image layer data comprise image layer basic information, image layer metadata information and geographic target data, and the geographic target data comprise geographic target attribute information, geographic target index information and geographic target vector data; using the vector image layer data as a storage unit, and computing an outer covering frame range of the storage unit; storing the image layer basic information, the image layer metadata information, the geographic target attribute information, the geographic target index information and the outer covering frame range of the storage unit into a relation database; storing the geographic target data in the storage unit into a distribution key value database. According to the method and device for processing the geographic information vector data, the geographic information vector data storage efficiency and the geographic information vector data inquiring efficiency can be effectively improved.

The invention discloses block chain node equipment and a data reading and writing-in method of a distributed database. By launching a transaction for binary data of keys and/or values on a block chain, and constructing a local index from the keys to transaction target information, the decentralized key-value database based on the block chain is achieved, and data writing-in and reading based on the key-value database are supported. The decentralized key-value database achieved through a block chain technology can run based on common network nodes, and any amendment records of data can be traced and inquired.

The invention discloses a layering storage and inquiring method based on a key value database. The method includes the following implementation steps: constructing a layering key code naming rule and determining a comparison method between key codes; when executing a data storage request, determining the key code sequencing position of the data storage request according to the key code comparison method and inserting data into the key value database; when executing the data inquiring request, searching for a first greatest lower bound larger than or equal to all possible key codes under a target key code catalogue and utilizing the bound as a starting key code and returning all key values with the target key code as the beginning and results of sub catalogues as the data inquiring request from the starting key code. The method has high speed layering data storage and inquiring capability on the basis of traditional characteristics of the key value database, similar tree-shaped complex data can be effectively organized and stored, and the method can be used for storing data in the catalogue mode supporting massive files of a light distributive network file system.

The embodiment of the present invention provides a method for constructing and mining a monitoring-oriented video map, equipment, by constructing graph database and key-value database as data persistence storage of knowledge map, the depth convolution neural network is used for face recognition automatically, greatly reducing the intensity of manual scrutiny, Using a combination of shared convolution neural network and deep deconvolution network to automatically label the attributes of pedestrian objects, the unstructured video data is converted into structured graph data storage, it can satisfy the character retrieval based on censorship surveillance video, Trajectory mining, group center mining and other security requirements, can efficiently query node attribute information, can effectively visualize the relationship between nodes, using backbone information database and detailed information database storage methods, play their respective advantages, data storage and retrieval efficiency is high.

The invention belongs to the field of computer technology and relates to an information inquiry method, a device, a computer storage medium and an electronic device based on a distributed database. The information query method based on the distributed database includes: acquiring the first data and aggregating the first data at least once to generate at least one level index; Sorting the aggregated first data according to an index field corresponding to the index, and importing the sorted first data into a distributed key value database; Receiving a query request initiated by a user, and querying the distributed key value database according to the query request to obtain a second data; Returning the second data to the user. On the one hand, the invention improves the efficiency of data query by storing data in a distributed key value database; On the other hand, the distributed key-value database is scalable and can improve the storage capacity.

A method, article of manufacture, and apparatus for protecting data, comprising receiving a changed block, reading a previous backup to get a file list in change block tracking, checking a file in a modified master file table list for a reference flag, wherein the modified master file table list is based on the changed block, and modifying a database entry based on the reference flag.

The present invention provides a data storage structure conversion method. The method comprises: extracting a first data unit and a second data unit in to-be-converted data based on an original storage structure; combining the first data unit and the second data unit to determine a key in a converted-into storage structure; then based on the original storage structure of the to-be-converted data, determining a value in the converted-into storage structure according to a value setting rule; and according to the converted-into storage structure, storing a data key-value pair consisting of the key and the value into a key-value database. According to the method provided by the present invention, the data based on the original storage structure is converted into and stored as the data based on the converted-into storage structure, so as to store various types of data by using a uniform converted-into storage structure. Meanwhile, with relatively high data storage processing efficiency being ensured, a storage mode limited by a physical memory is converted into a storage mode in which a magnetic disk is the destination of data storage and that is not limited by a physical memory.

The invention provides a message processing method, device and system. The system comprises a message receiving module which is used for receiving an issuing request sent by an issuing client; a message processing module which is used for converting the issuing request into a message content key-value pair, and transmitting the message content key-value pair to a key-value database for storage, wherein the issuing request comprises to-be-issued message contents and theme information, the keywords of the message content key-value pair comprises the theme information and number information representing the issuing sequence of the message contents, and the key values of the message content key-value pair comprises the message contents in the issuing request. According to the invention, the system enables the issued message to be converted into the key-value pair stored in the key-value database, and improves the message storage reliability and durability. Moreover, the keywords of the generated key-value pair comprise the number information representing the issuing sequence of the message contents, thereby facilitating the subsequent management of a message pushing state.

The invention discloses a write-in method of a key/value database memory log. The method comprises the following steps that according to the number N of disks capable of being practically used by a system, N log files are written in the disks in parallel, and in addition, each file in the N log files is put in different disks. According to the method, the key/value database log file is split into a plurality of sub files by adopting a Hash algorithm, the multi-disc characteristics are sufficiently utilized, and meanwhile, the sub files are written into different disks in parallel, so that the I/O (input/output) seek time is effectively reduced, the write-in concurrency of the key/value database log is improved, and the fast and efficient log write-in is realized.

The invention provides a method for realizing the multi-key finding of a key value database. The method comprises the following steps of: registering a dividing structure of a first key for a main data table in a key value databank, and registering a dividing structure of a second key for a main data table index table, wherein a dividing method of each line of data and a comparison rule of each row of data of the main data table and the index table are respectively provided by the dividing structure of the first key and the dividing structure of the second key of the main data table and the index table, so that an index table which meets multi-row finding is established when indexes are established for the main data table. The technical scheme of the invention can effectively and reasonably improve the efficiency for finding multiple key values, is simple and efficient, and the expense can be neglected, so that the resource is effectively saved; and moreover, by utilizing the method provided by the technical scheme of the invention to realize the multi-key value finding, all items which meet querying conditions can be obtained through once index query, so that the method is convenient and efficient to use.