42 results about "Index compression" patented technology

A data storage caching architecture supports using native local memory such as host-based RAM, and if available, Solid State Disk (SSD) memory for storing pre-cache delta-compression based delta, reference, and independent data by exploiting content locality, temporal locality, and spatial locality of data accesses to primary (e.g. disk-based) storage. The architecture makes excellent use of the physical properties of the different types of memory available (fast r/w RAM, low cost fast read SSD, etc) by applying algorithms to determine what types of data to store in each type of memory. Algorithms include similarity detection, delta compression, least popularly used cache management, conservative insertion and promotion cache replacement, and the like.

An apparatus and method is provided for pruning an index of a corpus of text documents by creating an inverted index of terms appearing in the documents, wherein the index includes postings of the terms in the documents, ranking the postings in the index, and pruning from the index the postings below a given level in the ranking.

A method for matrix by vector multiplication, applied in an artificial neural network system, is disclosed. The method comprises: compressing a plurality of weight values in a weight matrix and indices of an input vector into a compressed main stream; storing M sets of synapse values in M memory devices; and, performing reading and MAC operations according to the M sets of synapse values and the compressed main stream to obtain a number M of output vectors. The step of compressing comprises: dividing the weight matrix into a plurality of N×L blocks; converting entries of a target block and corresponding indices of the input vector into a working block and an index matrix; removing zero entries in the working block; shifting non-zero entries row-by-row to one of their left and right sides in the working block; and, respectively shifting corresponding entries in the index matrix.

A method and apparatus for searching compressed nucleic acid sequences are disclosed. In the method, a reference sequence is compared with a subject sequence to be encoded, the subject sequence is compressed, an index is created with respect to the reference sequence and the compressed subject sequence, a position corresponding to a query is searched for in the compressed subject sequence using the index, a character found at the position within the compressed sequence is converted into a sequence, and the sequence is output as the response to the query.

Systems, methods and computer program products for compression of database indexes are described herein. A system embodiment includes a sequence determiner to scan a database index and to determine a start of a range and end of a range of consistently changing values in one or more index pages of said database index and an index updater to update said database index based on a sequence determined by said sequence determiner, while suspending writing of one or more values that lie within start of said range and end of said range of values. A method embodiment includes scanning an index, determining a pattern of changing values in one or more index pages of said index and selectively updating said index based on said determining step to minimize index insertions. The method embodiment further includes determining a start of a range of values and an end of said range of values in an index page, setting appropriate bits to identify said start of range of values and end of range of values, determining if an entry to be inserted can appended to at the end of said range of values, and compressing said index by suspending writing of one or more values that occur between said start of range of values and said end of range of values.

A technique for reducing the total storage used in representing data having spatial extents. The data is represented in a tree structure having a plurality of nodes, wherein each of the nodes has parent and child relationship to one or more others of the nodes in the tree structure. An encoded representation of the relation of a child node's extents with respect to its parent is stored in the node. A preorder traversal of the tree structure is performed to store it compactly in an output file.

The invention discloses an index data storage method and relates to the field of data processing. By compression of index pages, system storage space is reduced, and number of index records on each index page is increased. The index data storage method includes: acquiring all raw index data sets in a single page, creating an index bitmap in the single page, and determining and storing a minimum index value in the single page; subtracting the minimum index value from each index value to calculate difference values delta1, setting a flag bit of the delta1th position in the index bitmap, then deleting the index values in each raw index data set, and after compressed index data sets including only tuple addresses are generated, restoring the compressed index data set corresponding to each raw index data set in the single page according to the higher-lower bit sequence of the delta1, corresponding to the index values, in the index bitmap so as to complete index compression. The n index data storage method is mainly applied to the process of index data storage.

The invention discloses a quantum safety communication method based on synchronous random number information base information retrieval, which avoids channel loss and noise, has high efficiency, is safe and reliable, and is low in cost. The method comprises the following steps of: 1) establishing a synchronous random number information base with large data volume at a sending end and a receiving end of a quantum communication system; 2) determining the unit length of information units according to the data volume of the information base at the sending end, and establishing an information maintenance table for the information base simultaneously; 3) preprocessing communication data at the sending end; 4) searching an information unit which is the same as the preprocessed communication data in the information base at the sending end, and recording the searched index communication data; 5) compressing the index communication data to acquire index compressed data, and sending the index compressed data to the receiving end in a classical communication mode; 6) performing reverse operation on the received index compressed data by using the receiving end; and 7) updating the information base.

A method for compressing index identification includes fetching the first index identification judging scope existed by value of the first index identification, confirming out length unit and label by utilizing said scope, using the first index identification value presented by confirmed out length unit as the second index identification, using the second index identification and confirmed out label commonly to replace said first index identification.

The present invention is a method and apparatus for adaptively compressing and transmitting streaming video, preferably implemented in hardware as part of a codec, that adaptively applies a variable rate of compression to images in a video stream to optimize the quality attainable for a given transport bandwidth. In one or more embodiments, image frames from the video stream are divided into blocks, and a variable compression ratio compression engine compresses the blocks for transmission over a network. Depending on their compressed sizes, one or more of the compressed blocks are packaged into fixed size data packets for transmission over a network. The data packets are fed into a packet buffer. A packet servo releases packets from the buffer to achieve a desired transport data rate. The amount of compression applied by the compression engine is controlled based on the number of packets in the buffer. In one or more embodiments, a plurality of indexed compression vectors are predefined for the compression engine that provide increasing amounts of compression. In one or more embodiments, the index of the compression vector used by the compression engine is incremented when the buffer level of the packet buffer increases, and decremented when the buffer level decreases. In this manner the compression applied by the compression engine automatically adapts to changing entropy within the images of the video stream to optimize the video quality for a given transport data rate.

Systems and methods for compressing an index are described. In one exemplary method, the results of a search are annotated and then encoded into one or more chunks of compressed data in accordance with the annotations of the results. The annotations include an indication of a best encoding method selected from a set of available encoding methods, and an indication of whether to switch to a new chunk during encoding or to continue encoding in the current chunk. Other methods are described and data processing systems and machine readable media are also described.

The invention discloses a compressing method and a device of internal storage index. The method comprises that an index hash chain table space is created in the internal storage; when an internal storage index key value needs to be stored in the internal storage, a hash calculation is conducted to the internal storage index key value to obtain a hash key value through a first hash function; whether the same valve as the hash key value exists in the index hash chain table space or not is checked; when the same valve as the hash key value exists in the index hash chain table space, no operation is implemented, when the same valve as the hash key value does not exists in the index hash chain table space, the hash key values is stored in the index hash chain table space. The compressing method and the device of the internal storage index can compress the internal storage index effectively and can reduce the occupation of the internal storage index to the internal storage.

The invention provides an OBJ file lossless compression method for regular buildings, which comprises the following steps: reading geometric data files and material files of the regular buildings, andconstructing memory objects in a memory; traversing the material in the material file, and constructing the material parameter dictionary in the memory, wherein the key of the material parameter dictionary is the name of the material parameter, and the value of the material parameter dictionary is the list of optional values of the material parameter; searching the value of the current material parameter in the material parameter dictionary, and replacing the value of the current material parameter with the corresponding index value to generate the data block of the material block; generatingcompact geometric data blocks by vertex shrinkage and tree index compression, and outputting the indexed material files in memory to the merged file in the order of material parameter dictionary anddata blocks. The method can play an effective compression effect on regular buildings, can effectively compress data volume and improve data transmission efficiency.

Apparatus and method for compression of acceleration structure build data in a ray tracing implementation. For example, one embodiment of an apparatus comprises: traversal hardware logic to traverse rays through a graphics scene comprising a plurality of primitives; and an acceleration data structure processing unit comprising: a bounding box compressor to compress a set of bounding boxes to generate a plurality of bounding box compression blocks, and an index compressor to compress a set of indices to generate a plurality of index compression blocks, and an acceleration data structure builder for constructing acceleration structures based on bounding box compression blocks and index compression blocks

The embodiment of the invention discloses a Chinese word entry index compression method based on a mobile terminal and the mobile terminal. The quantity of data stored in an ROM or an SD of the mobile terminal is greatly reduced, so that the processing speed of subsequent index retrieving is increased, and the technical problem that for a traditional database index scheme, due to the fact that Chinese word entries are not optimized, the efficiency is low when information of word entries containing partial characters or words is retrieved is solved. The Chinese word entry index compression method based on the mobile terminal comprises the steps that all classified word groups are associated according to keywords, and a corresponding associated word group list is established; the associated word group list is subjected to coding processing based on a position relationship of the keywords in the word groups, and a corresponding compression character string is generated; the compression character string is stored.

The invention relates to an index compression method and device, a computer readable storage medium and electronic equipment. The method comprises the steps of dividing an original key of an index file into a high-order part and a low-order part; multiplexing the high-order part and the low-order part to represent a new offset; wherein when the offset is used as a new offset, the high-order part is used for representing an interval range of atom.dat in base.dat; the low-order part is used for representing an offset address relative to the atomic.dat; compressing the size of the original size into two bytes, and taking the two bytes as a new size; and forming a new index file based on the new key, the new offset, the new size and the original crc16. According to the method, the original key is segmented, only the low-order part is reserved and stored in the memory, the high-order part and the low-order part are reused as the offset, and the original size is compressed into two bytes to serve as the new size, so that the size of the index file is greatly reduced, and the size of the memory occupied by the index file is effectively reduced.

The invention discloses a method and a device for forming a data micro index in an on-chain contract and electronic equipment. The method comprises the following steps: establishing a micro index for data under a chain according to a preset index compression method; calculating a hash value of the micro index under the chain; and storing the micro index and the hash value thereof in an on-chain contract. According to the method, the data provided by the oracle machine forms the micro index under the chain and then is stored in the on-chain contract, so that the occupied space on the chain is greatly reduced, a large amount of complete data can be inquired by storing a small amount of necessary data in the extremely small on-chain space, and the smooth completion of on-chain transactions is further ensured.