15 results about "Suffix array" patented technology

The invention discloses a self-adaptive range query filtering method, system and device and a storage medium. The method comprises the steps that each key in an input data set is segmented into a prefix and a suffix, the keys with the same prefix are divided into the same partition, and each partition comprises the number of the prefixes, the number of the suffixes and suffix arrays arranged in an ascending order; a self-adaptive range filter is constructed, in the construction process, coding is carried out with partitions as units, prefixes are inserted into a cuckoo filter, suffixes are inserted into a bridge structure, suffix arrays are continuously and compactly stored according to the sparse degree, and compression processing is carried out; according to the query range, the left boundary and the right boundary are segmented into two prefix-suffix pairs, the corresponding partition is rapidly positioned according to the prefix, range query is executed in the suffix array of the partition, and the suffix meeting the condition is returned. According to the invention, the self-adaptive range query filter based on fingerprints is realized through key segmentation and a specific coding mode, the inclined workload can be adaptively coped, and the query performance is improved.

The invention discloses a large data BWT construction method and device and terminal equipment, and relates to the technical field of computers, and the method comprises the steps: scanning a character string T, calculating L type and S type substring precursor sequences LPre and SPre according to the ranking of S * type characters in the T, calculating the ranking of the L type substring of the T by adopting a small top heap H1, calculating the ranking of the S type substring of the T by adopting a large top heap H2, and calculating the ranking of the S type substring of the T according to the ranking of the S * type characters in the T; sorting and naming the S * type substring of the T to generate a new contracted character string T1; calculating a suffix array SA (T1) of the T1; calculating the ranking of S * type suffixes of the T according to the SA (T1), calculating L type and S type suffix precursor tuple sequences LPre and SPre according to the ranking of the S * type suffixes of the T, calculating an L type BWT sequence Lbwt by using a small top heap H1, and calculating an S type BWT sequence Sbwt by using a large top heap H2; and combining the two types of BWT sequences to generate a final BWT sequence of the character string T.

The invention discloses a redundant field filtering method based on an improved BM algorithm and a suffix array, and relates to the technical field of pattern recognition and data cleaning, and the method comprises the following steps: S1, obtaining a target text, preprocessing the target text to obtain a standard target text, and obtaining a text character string corresponding to the standard target text, the method comprises the steps of S1, obtaining a text string, S2, obtaining a redundant field text, and obtaining multiple segments of redundant fields corresponding to the redundant field text, and S3, taking the text string as a main string T, constructing a suffix array S based on the main string T, and updating the suffix array S to obtain a new suffix array SA. According to the method, the suffix array and the longest common prefix array are constructed, so that possible matching positions can be quickly positioned in the matching process, unnecessary character comparison operation is avoided, and the number of times of character comparison is reduced.

The invention discloses a multi-sequence parallel comparison method based on maximum precision matching, and relates to the technical field of crossing of bioinformatics and computational biology. The method comprises the following steps: splicing character strings of a plurality of sequences to obtain a long sequence, and determining a suffix array and a longest common prefix array of the long sequence by adopting a mode of combining a bucket dividing strategy and induced sequencing; according to the suffix array, searching a maximum accurate matching fragment from the longest common prefix array, and screening an ordered and non-overlapped maximum accurate matching fragment set through a dynamic programming technology; expanding each maximum accurate matching fragment in the maximum accurate matching fragment set to all sequences to form a chain; segmenting the long sequence through a chain to obtain a plurality of sub-sequence sets; and calling third-party multi-sequence comparison software to compare the plurality of sub-sequence sets, and splicing the comparison result of each sub-sequence set to obtain a multi-sequence comparison result. According to the method, the time complexity of ultra-long or large-scale sequence data comparison is reduced.

The invention relates to the technical field of computers, and provides a correctness verification method and device for a large-scale text sequence BWT, equipment and a medium. The method comprises the following steps of: blocking a character string X and a suffix number group of the character string X, and shrinking the character string X by recursively calling block induction sorting until a character in the shrunk character string X1 is unique; in the backtracking stage of the recursive 0 layer, calculating the sequence of S * type suffixes of X according to the suffix array of X1, and calculating the hash value of a descending S * type suffix sequence; according to the sequence of the S * type suffixes of the character string X, calculating L type and S type suffix preamble character sequences of each block and hash values of the L type and S type suffix preamble character sequences; calculating an L * type suffix sequence and the hash value of each block of the L type BWT sequence by adopting a small top heap; calculating the hash value of each block of the S * type suffix sequence and the S type BWT sequence by adopting a large top heap; and combining the two types of BWT sequences, and outputting a final verification result, so that the space-time overhead of BWT correctness verification is reduced.

The invention relates to a ship increment updating method based on an improved Bsdiff algorithm, and the method is characterized in that the method comprises the following steps: 1, generating a suffix array based on an old file through employing an EM-SA-DS algorithm; 2, initializing a difference file; 3, comparing a new file by using the generated suffix array, and generating an approximately matched region; 4, determining the boundary of the difference file; 5, writing in a control file, a difference file and a newly added file; an incremental update file is generated by the old file and the new file at the shore end and transmitted to the ship end, and the new file is generated by the old file and the incremental update file at the ship end; the problem that the memory space required for generating the increment file is large when the Bsdiff algorithm is used for remote updating of the ship-end equipment is solved, the size of an updating package can be effectively reduced, the transmitted data volume is reduced, rollback caused by upgrading failure is reduced, memory use is reduced, and efficiency is improved.

The invention provides a differential package generation method, a differential package reduction method, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring first version data and second version data; the second version data is used for updating the first version data; performing lexicographical order rule sorting on the suffix array of the first version data to obtain a shortest suffix array set of the first version data; traversing the second version data according to the shortest suffix array set to obtain a coverage line set; and generating a differential packet of the first version data and the second version data according to the overlay line set. According to the method, the differential data packet between the new version data and the old version data is generated according to the overlay line set for firmware upgrading, the differential packet occupying small storage space is obtained, and the firmware upgrading efficiency is improved.

The invention discloses a sparse data index construction method and device, electronic equipment and a medium. The method comprises the steps that a target character string and a position array are acquired; performing multi-layer scanning processing on the target character string to obtain a virtual character string; sorting the target type substrings corresponding to the two-tuples in the virtual character string to obtain a suffix sequence; based on the position array, scanning from the target character string to obtain a target character of the target position; based on a preset association rule, obtaining an associated character of the target character from the virtual character string; according to the position index numerical value corresponding to each target character, the position index of the tail character of the associated character and the suffix sequence, constructing a triple; and sorting all the triads, outputting a position index value of each triad in sequence based on a sorting result, and sorting to obtain a sparse suffix array of the target character string. The sparse data index can be constructed efficiently, and the method can be widely applied to the technical field of data processing.

Embodiments of the invention provide a query cardinal number determination method and apparatus, relate to the technical field of databases, and can reduce the space overhead when determining a query cardinal number. The method comprises the following steps: determining a maximum target suffix in suffix data of target data; and determining a query cardinal number of the target data based on the target suffix, a suffix array of the character string in the database and a target function used for indicating a character distribution condition.

The application relates to the technical field of server content management, in particular to a server content updating and rollback method based on static technology; the application detects storage hot area changes through an entropy value difference matrix, combines a time stamp marking feature moment, realizes accurate division and unique identification of a historical version, avoids redundant operations of full scanning, only carries out version processing on a change hot area, significantly optimizes storage space utilization, and improves the efficiency and accuracy of version tracing; the application generates an incremental patch through a binary difference algorithm accelerated by a suffix array, combines network topology optimization to build an optimal transmission path, adds error correction code check information to data blocks, reduces data transmission volume, enhances transmission fault tolerance through the error correction code, and ensures data integrity and transmission efficiency when network fluctuation or node failure occurs.

The invention relates to the technical field of server content management, in particular to a server content updating and rollback method based on a static technology. According to the method, the change of the storage hot spots is detected through the entropy difference matrix, the characteristic moments are marked in combination with the timestamps, accurate division and unique identification of historical versions are achieved, redundant operation of full-amount scanning is avoided, versioning processing is only carried out on the change hot spot regions, the storage space utilization rate is remarkably optimized, and the version tracing efficiency and accuracy are improved; according to the method, the incremental patch is generated by adopting the binary differential algorithm accelerated by the suffix array, the optimal transmission path is constructed by combining network topology optimization, and the erasure code check information is added for the data block, so that the data transmission quantity is reduced, and meanwhile, the transmission fault tolerance is enhanced through the erasure code; and the data integrity and the transmission efficiency during network fluctuation or node failure are ensured.

The application discloses a sparse data index construction method and device, electronic equipment and medium. The method comprises the following steps: obtaining a target string and a position array; performing multi-layer scanning processing on the target string to obtain a virtual string; sorting a target type substring corresponding to a binary tuple in the virtual string to obtain a suffix order; scanning a target character at a target position from the target string based on the position array; obtaining an associated character of the target character from the virtual string based on a preset association rule; constructing a triple based on a position index value corresponding to each target character, a position index of a tail character of the associated character and the suffix order; sorting all the triples, and then sequentially outputting the position index value of each triple based on a sorting result to obtain a sparse suffix array of the target string. The application can efficiently implement the construction of the sparse data index, and can be widely applied to the technical field of data processing.

The present invention provides a method and system for generating a sparse suffix array, which relates to the technical field of data structure construction, and includes: dividing a source file into multiple strings with a preset length by a computing terminal; when the preset value is greater than a threshold, the computing terminal sorts each string to obtain a string sorting result, and then maps the corresponding strings in the string sorting result to integers respectively to form an unsigned integer array; when the preset value is not greater than the threshold, the computing terminal obtains its own attribute parameters, and when it determines that it is a little-endian computer according to the attribute parameters, it reversely arranges each string to obtain a reversed string, and converts each reversed string into an integer to form an unsigned integer array; the computing terminal performs integer suffix sorting on the unsigned integer array to obtain a sorted array, and corrects each element in the sorted array according to the preset value to obtain the sparse suffix array of the source file. The beneficial effect is to effectively save the running space.

The application relates to the technical field of computers, and provides a correctness verification method, device and equipment of a large text sequence BWT and a medium. The method comprises the following steps: blocking a character string X and a suffix array thereof, recursively calling block induction sorting to contract the character string X until the characters in the contracted character string X1 are unique; in the backtracking stage of the recursion 0 layer, the order of S* type suffixes of the character string X is calculated according to the suffix array of X1, and the hash value of the descending S* type suffix sequence is calculated; according to the order of the S* type suffixes of the character string X, the L type and S type suffix predecessor character sequence and the hash value thereof of each block are calculated; the L* type suffix order and the hash value of each block of the L type BWT sequence are calculated by using a small top heap; the S* type suffix sequence and the hash value of each block of the S type BWT sequence are calculated by using a large top heap; and the two types of BWT sequences are combined, and the final verification result is output, so that the time and space overheads of BWT correctness verification are reduced.

The application relates to the field of bioinformatics, and relates to a method for improving gene sequence alignment based on BWT transformation by using a multiplication algorithm. First, a suffix array is directly calculated by using the multiplication algorithm through a reference long sequence, and then data structures required by BWT transformation are obtained. The BWT matrix does not need to be stored in space, N*N two-dimensional space is saved, and time is greatly improved. Seed sequences are subjected to pretreatment operation, the seed sequences are sorted, the same suffixes are gathered together, the intermediate result of the previous seed sequence alignment is conveniently saved, and the number of alignment times is reduced. After the sa interval is found, a search is conducted by using a BFS (Breadth-First Search) method, the BFS can guarantee that the optimal sequence score is found as soon as possible, the optimal sa is not missed, then a heap data structure is used for storage, the top of the heap is the highest score each time, the highest score item is obtained each time to conduct the next layer search, and the optimal matching result is obtained until the matching is finished.