175 results about "Key-value database" patented technology

The invention discloses a distributed key value database system, which comprises a client end and a data server cluster, wherein the client end comprises a Hash module, a positioning module and a forwarding module; the data server cluster comprises a plurality of data intervals; each data interval comprises a host node and a plurality of slave nodes; the host node comprises a first read module, a write module, a first recovery module, a first log module, a first decision module and a first storage engine module; the Hash module is used for receiving a write request sent from a client, and positions the write request to the data intervals according to a key of the write request; each write request is provided with a number distributed by a system automatically; the positioning module is used for positioning the write request to the host nodes of the data intervals; and the forwarding module is used for sending the write request to the write module of the main node. According to the distributed key value database system, strong consistency between data copies is provided under the condition of smaller expenditure.

The invention discloses a method and a device for data partitioning in a key-value database. The method comprises the following steps of: resolving partitioning number information and partitioning algorithm information from a pre-stored partitioning information basic list; and running a corresponding partitioning algorithm by a partitioning callback function according to the partitioning algorithm information, wherein the corresponding partitioning algorithm determines information of partitioned sub databases of key-value data to be partitioned according to the partitioning number information and a calculation value; and the calculation value is a value relevant to the key-value data to be partitioned. The partitioning number information and the partitioning algorithm information are recorded into the partitioning information basic list; when the key-value data are subjected to operation such as storage and query, the partitioned sub databases are determined for the key-value data according to contents recorded in the partitioning information basic list; and therefore, the partitioning mode can be flexibly changed by modifying the contents in the partitioning information basic list.

A method for bursting a hash table of a key-value database comprises receiving a key and a value, traversing trie nodes of the key-value database from a root node to a leaf node by recursively dividing the key into a prefix and a suffix, reaching the leaf node, the leaf node being the hash table, determining that the key is not stored in the hash table, determining that the hash table is not able to store the key and the value, removing the hash table, associating a new trie node with a parent trie node of the hash table, associating two or more new hash tables with the new trie node, moving all keys and associated values from the hash table into one of the two or more new hash tables, and inserting the key and the associated value into one of the two or more new hash tables.

An embodiment of the invention discloses a storage method for data in a Key-Value system and a related device. By means of the method and the device, the storage operation efficiency of Value data can be improved. The method comprises steps as follows: whether the data volume of Value data in the Key-Value exceeds the threshold value of data is judged; if the data volume of Value data doesn't exceed the threshold value of data, the Value data is sliced, and M slice contents are obtained; slice information is generated for N slices according to the M slice contents and comprises the number of Value data slices, offset addresses of slice contents in the N slice contents, serial numbers ID of the slice contents in the N slice contents and (M-N) slice contents; and Key data and slice information are stored in an LSM-Tree (the log-structured merge-tree), the N slice contents are stored in a Key-Value database, and the Key data correspond to the slice contents.

The embodiment of the invention discloses a method for realizing real-time retrieval. After extending a Key-Value Hash database to form a relational database structure, the method comprises the following steps of: receiving information carrying Key-Value data and delivered by a client; writing data into memory object caches and extended Key-Value Hash database according to the information; receiving query commands delivered by the client; and retrieving according to the query commands. Simultaneously, the embodiment of the invention also correspondingly discloses a system and a server for realizing the real-time retrieval. Adding the Key-Value database served as persistent storage for the memory object caches, the embodiment of the invention is capable of realizing quickly reading and writing the data in the database under the condition of high concurrent access and prevents data loss in the retrieving process. Simultaneously, the Key-Value database is extended to form the relational database structure, thus being capable of realizing the queries on complex conditions.

The invention discloses a generation method of a pre-join table on a ring-shaped schema database, belonging to the field of distributed data storage. The method comprises the following steps of: establishing a Key-Value database, forming a column value index of a key column and storing the column value index in the database; generating a pre-join table; if an original data table is updated, updating the corresponding pre-join table; and executing data query, and outputting query result. With the pre-join table generation method provided by the invention, the pre-join table can be efficiently generated in the ring-shaped schema Key-Value database, and a quick join query function is provided; by optimizing a computational process, time and space cost for generating the pre-join table is reduced, and calculation speed is accelerated; and simultaneously, query delay is reduced according to an updating and maintaining method, and real-time join query result can be provided.

The invention discloses a dynamic user behavior-based cloud forensics method and a dynamic user behavior-based cloud forensics system; the formalized definition of dynamic user behavior is put forward as basis; and the method comprises the steps of: collecting dynamic user behavior and behavior data, and storing the behavior data as primary evidence data; carrying out data integration, data cleaning and data mining to form forensic analysis data, and storing the forensic analysis data into a data analysis library (key value database); using Map Reduce to carry out correlation analysis, sequence pattern analysis and isolated point analysis on the forensic analysis data, and digging out the potential user behavior patterns and the possible aggressive behavior to form forensic evidence; and displaying the forensic evidence in a visual way. The high performance computing power of cloud computing and large-scale distributed-memory environment are applied into computer forensic analysis, so that various problems in cloud computing forensics can be solved.

The invention provides a saving method and system for time series data based on a key value database. The method comprises the steps that S1, original time series data collected by a sensor is obtained, the original time series data comprises a plurality of attributes, and the attributes comprise an equipment number, a sensor number, a sampling time, a receiving time and a measuring value; S2, according to the sampling time, the original time series data is divided into a corresponding time partition, and time partition information of the original time series data is obtained; S3, the multipleattributes and the time partition information are combined to generate target time series data; S4, the equipment number, the sensor number and the time partition information are taken as first levelindexes, the sampling time is taken as a second level index, and the target time series data is saved in the key value database. According to the saving method and system for the time series data based on the key value database, the time partition information is introduced, the method and the system can adapt to different data sizes, data reading of non-query target is reduced, and a better timeseries data query performance is obtained on the basis of assurance of high-performance data writing.

The invention provides a data processing method and data processing device applied to a data warehouse. The data processing method includes: when a first data base is operated, rectifying records of a mirror database in the first database in the data warehouse according to an incremental log corresponding to the operation and then storing the rectified records in a key / value database; fetching latest entries from the key / value database, additionally storing the entries in a first data table of the data warehouse and enabling the first data table to include historical versions of the records of the mirror database; searching data in the first data table. Therefore, timeliness of the data in the data warehouse can be improved.

The invention provides a multi-device linkage control method and device and a storage medium, and the method comprises the steps: storing an obtained target request in a preset key value database, wherein the target request comprises a control instruction for at least two target devices; splitting the control instructions included in the currently executed target request in a key value database, and determining the priority of each control instruction in the currently executed target request; and sequentially sending all the control instructions in the currently executed target request to thetarget equipment corresponding to the control instructions according to the priorities from high to low so as to instruct the corresponding target equipment to execute preset corresponding operation based on the control instructions. According to the invention, the problems of long consumed time, control failure, abnormal control and the like during multi-device linkage are solved, and the effectsof realizing linkage control of the smart home device, improving the efficiency of controlling the smart home device and improving the user experience are achieved.

Embodiments of the present invention provide a method and a system of file access. The method comprises that: a file system driving module receives a first access request sent by an upper application; the file system driving module, according to the corresponding relation between an access interface corresponding to the upper application and an access interface of the key-value database, converts the first access request into a second access request which is suitable for access to the key-value database; the file system driving module sends the second access request to a client of the key-value database; and the client of the key-value database, according to the second access request, searches corresponding file content or directory entry content or file attributes or directory entry attributes from the key-value database which is stored in a server of the key-value database. When the technical schemes in the embodiments are compared with technical schemes of layer-by-layer access of metadata in file systems of POSIX in the prior art, access cost is relatively small and access efficiency is relatively high.

The invention discloses an information persistence and query method and device applied to a key-value database. Aiming at any business entity object, the attribute values of the business entity object are respectively stored in different columns of the same row of a datasheet; a row key is generated based on the attribute values of the business entity object. As information persistence of the database, such as a Hbase, is realized in a column-based storage mode, and according to the scheme, the attribute values corresponding to the same business entity object are stored in different columns of the same row, compared with an information persistence scheme, aiming at a relational database, provided by the prior art, the scheme provided by the embodiment of the invention can be matched with the characteristic, about the storage mode, of the key-value database. The invention also discloses a query method and device aiming at objects.

The invention discloses a method and device for processing geographic information vector data. The method includes the steps of converting the vector data in an external format into vector image layer data, wherein the vector image layer data comprise image layer basic information, image layer metadata information and geographic target data, and the geographic target data comprise geographic target attribute information, geographic target index information and geographic target vector data; using the vector image layer data as a storage unit, and computing an outer covering frame range of the storage unit; storing the image layer basic information, the image layer metadata information, the geographic target attribute information, the geographic target index information and the outer covering frame range of the storage unit into a relation database; storing the geographic target data in the storage unit into a distribution key value database. According to the method and device for processing the geographic information vector data, the geographic information vector data storage efficiency and the geographic information vector data inquiring efficiency can be effectively improved.

The invention discloses a cloud smart controller APP and a key value learning and assignment method. The APP can be used for any smart phone and achieves the functions of remote controller classification, keyboard layout, user registration and equipment login, keyboard assignment and the like. The smart phone adopting the APP can replace various remote controllers to control any smart home or household appliances. The key value learning and assignment method comprises the steps of recognition of a replaced remote controller, cloud key assignment database retrieval assignment, learning and assignment of key values of the replaced remote controller and the like. The APP obtained after keyboard assignment can directly control the corresponding smart home or household appliances through the smart phone. The method comprises the specific steps of downloading the remote controller APP to the smart phone, selecting a default keyboard consistent with the controlled equipment, conducting learning assignment, controlling the smart home equipment through WiFi / Bluetooth or converting the smart home equipment into infrared control signals through WiFi / Bluetooth, and replacing the infrared remote controller to control the household appliances.

During a search technique, indexes associated with user accounts of users that are using the communication application are opened in memory from a transactional key-value database. These indexes encompass messages (such as emails) communicated using the communication application, and each of the users has at least one separate, associated index. When a search query associated with a target user account is received from the communication application, a search based on the search query is performed by reading the associated index in the memory from the transactional key-value database without managing the index using a file system. Then, a result for the search query is returned.

A method for bursting a hash table of a key-value database comprises receiving a key and a value, traversing trie nodes of the key-value database from a root node to a leaf node by recursively dividing the key into a prefix and a suffix, reaching the leaf node, the leaf node being the hash table, determining that the key is not stored in the hash table, determining that the hash table is not able to store the key and the value, removing the hash table, associating a new trie node with a parent trie node of the hash table, associating two or more new hash tables with the new trie node, moving all keys and associated values from the hash table into one of the two or more new hash tables, and inserting the key and the associated value into one of the two or more new hash tables.

The invention discloses block chain node equipment and a data reading and writing-in method of a distributed database. By launching a transaction for binary data of keys and / or values on a block chain, and constructing a local index from the keys to transaction target information, the decentralized key-value database based on the block chain is achieved, and data writing-in and reading based on the key-value database are supported. The decentralized key-value database achieved through a block chain technology can run based on common network nodes, and any amendment records of data can be traced and inquired.

The invention discloses a layering storage and inquiring method based on a key value database. The method includes the following implementation steps: constructing a layering key code naming rule and determining a comparison method between key codes; when executing a data storage request, determining the key code sequencing position of the data storage request according to the key code comparison method and inserting data into the key value database; when executing the data inquiring request, searching for a first greatest lower bound larger than or equal to all possible key codes under a target key code catalogue and utilizing the bound as a starting key code and returning all key values with the target key code as the beginning and results of sub catalogues as the data inquiring request from the starting key code. The method has high speed layering data storage and inquiring capability on the basis of traditional characteristics of the key value database, similar tree-shaped complex data can be effectively organized and stored, and the method can be used for storing data in the catalogue mode supporting massive files of a light distributive network file system.

The invention discloses a data storage method and system, and relates to the field of data storage. According to the method, data and index records are written according to the storage characteristicsof an SMR disk and a front-to-back order of storage space in Zone, and are respectively stored according to the characteristics of a Key-Value database, combination of the Key-Value database and theSMR disk is realized, the advantages of high storage density and high storage quantity of the SMR disk can be exerted, the characteristic of convenient reading of the Key-Value database can also be effectively utilized, and the data input by a user are stored in real time. In addition, Zone of which idle storage space is greater than data length is selected to store the data of each time of writing, fragmentation of the data is avoided, and reading efficiency of the data is improved.

The embodiment of the present invention provides a method for constructing and mining a monitoring-oriented video map, equipment, by constructing graph database and key-value database as data persistence storage of knowledge map, the depth convolution neural network is used for face recognition automatically, greatly reducing the intensity of manual scrutiny, Using a combination of shared convolution neural network and deep deconvolution network to automatically label the attributes of pedestrian objects, the unstructured video data is converted into structured graph data storage, it can satisfy the character retrieval based on censorship surveillance video, Trajectory mining, group center mining and other security requirements, can efficiently query node attribute information, can effectively visualize the relationship between nodes, using backbone information database and detailed information database storage methods, play their respective advantages, data storage and retrieval efficiency is high.

The invention provides a data processing method and apparatus. The method comprises the steps of receiving a query operation request, wherein the query operation request comprises query parameters, and the query parameters include a first attribute identifier and a first attribute value; sending the first query request to a memory database; receiving a first query response returned by the memory database, wherein the first query response comprises a corresponding first content identifier queried in a index list corresponding to the first attribute identifier according to the query parameters by the memory database; sending a second query request to a key value database, wherein the second query request comprises a first content identifier; receiving a second query response returned by thekey value database, wherein the second query response comprises first data contents corresponding to the first content identifier and found in a key value record by taking the first content identifieras a key value through the key value database; and sending the first data contents to a requester of the query operation request. The method combines the advantages of the memory database and the keyvalue database, and has the advantages of good performance and low cost.

The invention provides a method and a device for processing data as well as a data storage system based on a key value data base. The method, the device and the system can improve the file access efficiency of the key value data base and can prolong the service life of a magnetic disc. The method comprises the steps as follows: slots corresponding to key names of data items to be stored at present are looked up in an index file according to corresponding relations between preset slots and the key names of the data items; metadata blocks of the data items to be stored at present are stored in the slots; in the corresponding relations, one slot corresponds to key names of a plurality of data items; the number of data items which correspond to one slot is preset, each slot comprises a recording area used for recording addresses of expansion slots connected to the slot; and under the condition that the slot is full, metadata blocks of data items to be stored at present are stored in expansion slots, which are not full, of the slot.

The invention belongs to the field of computer technology and relates to an information inquiry method, a device, a computer storage medium and an electronic device based on a distributed database. The information query method based on the distributed database includes: acquiring the first data and aggregating the first data at least once to generate at least one level index; Sorting the aggregated first data according to an index field corresponding to the index, and importing the sorted first data into a distributed key value database; Receiving a query request initiated by a user, and querying the distributed key value database according to the query request to obtain a second data; Returning the second data to the user. On the one hand, the invention improves the efficiency of data query by storing data in a distributed key value database; On the other hand, the distributed key-value database is scalable and can improve the storage capacity.

A method, article of manufacture, and apparatus for protecting data, comprising receiving a changed block, reading a previous backup to get a file list in change block tracking, checking a file in a modified master file table list for a reference flag, wherein the modified master file table list is based on the changed block, and modifying a database entry based on the reference flag.

The present invention provides a data storage structure conversion method. The method comprises: extracting a first data unit and a second data unit in to-be-converted data based on an original storage structure; combining the first data unit and the second data unit to determine a key in a converted-into storage structure; then based on the original storage structure of the to-be-converted data, determining a value in the converted-into storage structure according to a value setting rule; and according to the converted-into storage structure, storing a data key-value pair consisting of the key and the value into a key-value database. According to the method provided by the present invention, the data based on the original storage structure is converted into and stored as the data based on the converted-into storage structure, so as to store various types of data by using a uniform converted-into storage structure. Meanwhile, with relatively high data storage processing efficiency being ensured, a storage mode limited by a physical memory is converted into a storage mode in which a magnetic disk is the destination of data storage and that is not limited by a physical memory.

The invention provides a message processing method, device and system. The system comprises a message receiving module which is used for receiving an issuing request sent by an issuing client; a message processing module which is used for converting the issuing request into a message content key-value pair, and transmitting the message content key-value pair to a key-value database for storage, wherein the issuing request comprises to-be-issued message contents and theme information, the keywords of the message content key-value pair comprises the theme information and number information representing the issuing sequence of the message contents, and the key values of the message content key-value pair comprises the message contents in the issuing request. According to the invention, the system enables the issued message to be converted into the key-value pair stored in the key-value database, and improves the message storage reliability and durability. Moreover, the keywords of the generated key-value pair comprise the number information representing the issuing sequence of the message contents, thereby facilitating the subsequent management of a message pushing state.

An embodiment of the invention relates to a multi-tenant expanding method based on Key-Value database. The method includes acquiring initial business entity metadata; a customizing phase, an instance generating phase and an instance operating phase, wherein the customizing phase is expanding business attributes based on the initial business entity metadata, and defining business entities and displaying labels of the business attributes to generate specific business entity metadata meeting multi-tenant requirements; the instance generating phase is establishing a Key-Value database according to the specific business entity metadata; the instance operating phase is inserting tenant identifications as RowKey prefixes to the business entity data of Key-Value database, generating a query scope of RowKey to query business entity data through the tenant identifications, extracting the business entity data from the Key-Value database in the query scope and generating an interactive interface according to the specific business entity metadata, so as to display the business entity data.

The invention discloses a write-in method of a key / value database memory log. The method comprises the following steps that according to the number N of disks capable of being practically used by a system, N log files are written in the disks in parallel, and in addition, each file in the N log files is put in different disks. According to the method, the key / value database log file is split into a plurality of sub files by adopting a Hash algorithm, the multi-disc characteristics are sufficiently utilized, and meanwhile, the sub files are written into different disks in parallel, so that the I / O (input / output) seek time is effectively reduced, the write-in concurrency of the key / value database log is improved, and the fast and efficient log write-in is realized.

The invention provides a method for realizing the multi-key finding of a key value database. The method comprises the following steps of: registering a dividing structure of a first key for a main data table in a key value databank, and registering a dividing structure of a second key for a main data table index table, wherein a dividing method of each line of data and a comparison rule of each row of data of the main data table and the index table are respectively provided by the dividing structure of the first key and the dividing structure of the second key of the main data table and the index table, so that an index table which meets multi-row finding is established when indexes are established for the main data table. The technical scheme of the invention can effectively and reasonably improve the efficiency for finding multiple key values, is simple and efficient, and the expense can be neglected, so that the resource is effectively saved; and moreover, by utilizing the method provided by the technical scheme of the invention to realize the multi-key value finding, all items which meet querying conditions can be obtained through once index query, so that the method is convenient and efficient to use.

The invention discloses a high-performance computing-oriented distributed data organization method. Data access and data management are separated in an interface layer; and at a computing node end, a client provides a simplified data access interface for an application only, and file access operation, in a POSIX form, completely transparent for the application is supported. According to the method, for a data storage demand of a high-performance computing application, POSIX file system access semantics is simplified, and the data access and the data management are decoupled, so that an efficient and lightweight programming interface is provided for the application; a distributed object storage technology is adopted for performing data organization, so that data organization limitation of a conventional file system is broken through, a more efficient data access protocol is realized, and system expandable capability is further improved; and a memory-based metadata management mode is provided, and a high-performance non-relational key-value database is introduced in a high-performance computer system for the first time, so that an efficient and expandable metadata service can be provided for outside by utilizing excellent characteristics of the high-performance computer system.