Method and device for sorting character strings

Abstract

The invention discloses a character string sorting method and device, and relates to the technical field of big data. A specific embodiment of the method comprises the following steps: generating a polynomial corresponding to each character string to be sorted; polynomials corresponding to the character strings to be sorted are arranged in an ascending order according to a lexicographical order, and character strings are grouped according to a first section of strict ascending order arrangement sequence of the polynomials; the grouping has a sequence; for each group, comparing the sizes of polynomials according to the polynomials corresponding to the character strings in the group so as to sort the character strings in each group; and obtaining a sorting result of the character strings to be sorted according to the sequence of the groups and the sorting result of the character strings in each group. According to the embodiment, the character string sorting efficiency is improved. The end-to-end algorithm processing process is carried out by utilizing concurrent thread processing in a multi-thread environment and utilizing a shared memory to keep cache consistency, and the performance bottleneck caused by the fact that the current suffix sorting algorithm does not carry out parallel computing is overcome.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method and device for sorting character strings. Background Art

[0002] Suffix sorting is an algorithm that sorts the suffix substrings of a string data in lexicographic order. It is an important sorting algorithm that is widely used in fields such as data compression technology, full-text indexing, information retrieval, and search engine technology, and is further extended to the application of various information software used in daily life. Current suffix sorting algorithms are generally constructed by preprocessing the text string and matching the preprocessed string. Currently, the commonly used suffix sorting methods include: using all suffixes in the string as keywords to construct a compressed dictionary tree as a suffix tree, a method for suffix sorting based on the suffix tree, and a method for suffix sorting based on the suffix array.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The suffix sorting method based on the suffix tree has high time and space complexity and the construction process of the suffix tree is quite difficult; the suffix sorting method based on the suffix array is limited by its serialized calculation characteristics, which makes the efficiency of suffix sorting of string data relatively low. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for string sorting, which can use a polynomial representation to avoid treating a string as a continuous character sequence for comparison and sorting, and then use the idea of ​​radix sorting to do suffix sorting, thereby improving the efficiency of string sorting. The present invention uses a polynomial representation method and introduces bit operations to more efficiently complete the comparison operation in the sorting algorithm; at the same time, the present invention uses concurrent thread processing in a multi-threaded environment and uses shared memory to maintain cache consistency technology to perform end-to-end algorithm processing, thereby overcoming the performance bottleneck caused by the current suffix sorting algorithm not performing parallel computing.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for sorting strings is provided, comprising:

[0007] Generate a polynomial corresponding to each string to be sorted;

[0008] Arrange the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and group the strings according to the first section of the strictly ascending sequence of the polynomials; the grouping has a sequence;

[0009] For each group, the polynomials corresponding to the strings in the group are compared to sort the strings in each group;

[0010] According to the order of the groups and the sorting result of the character strings in each group, the sorting result of the character strings to be sorted is obtained.

[0011] Optionally, generating a polynomial corresponding to each string to be sorted includes: expressing each string to be sorted in polynomial form to obtain a first polynomial; using the number of adjacent identical characters in the first polynomial as coefficients, and merging identical characters in the first polynomial to generate a polynomial corresponding to each string to be sorted.

[0012] Optionally, for each group, a size comparison is performed between polynomials according to the polynomials corresponding to the strings in the group to sort the strings in each group, including: starting a process for each group, using the process to adopt a shared memory method, and a size comparison is performed between polynomials according to the polynomials corresponding to the strings in the group to sort the strings in each group.

[0013] Optionally, for each group, the sizes of the polynomials are compared based on the polynomials corresponding to the strings in the group to sort the strings in each group, including: for each group, the polynomial corresponding to each string in the group is encoded according to a set representation method; and the sizes of the polynomials are compared based on the encoding of the polynomial corresponding to each string in the group to sort the strings in each group.

[0014] Optionally, for each group, a size comparison is performed between polynomials according to the polynomials corresponding to the strings in the group to sort the strings in each group, including: for each group, encoding the polynomial corresponding to each string in the group according to a set representation method; for any two strings in the group, based on the encodings of the polynomials corresponding to the two strings, performing an XOR operation and then a specified bit operation to obtain the polynomial containing the lowest-order terms of the different terms in the polynomials corresponding to the two strings, so as to compare the sizes of the polynomials corresponding to the two strings; and sorting the strings in the group according to the size comparison result of any two strings in the group.

[0015] Optionally, the polynomial corresponding to each string in the group is encoded according to a set representation method, including: for each string in the group, obtaining the length of the string according to the number of data items in the polynomial corresponding to the string, and obtaining the highest power of the polynomial corresponding to the string; for each data item, obtaining the encoding of the data item according to the index value and coefficient corresponding to different powers of the variables in the data item, the power not exceeding the highest power; and obtaining the encoding of the polynomial corresponding to the string according to the encoding of each data item in the polynomial corresponding to the string.

[0016] Optionally, the space occupied by the encoding of the polynomial corresponding to the string is 1+n*(m+2) bits, where n is the number of data items in the polynomial corresponding to the string, and m is the highest power.

[0017] According to another aspect of an embodiment of the present invention, a device for sorting character strings is provided, comprising:

[0018] A polynomial generation module, used to generate a polynomial corresponding to each string to be sorted;

[0019] A first sorting module, used to arrange the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and to group the strings according to the first section of the strictly ascending sequence of the polynomials; the groups have an order;

[0020] A second sorting module is used to compare the polynomials corresponding to the strings in each group, so as to sort the strings in each group;

[0021] The sorting and summarizing module is used to obtain the sorting result of the character strings to be sorted according to the order of the groups and the sorting result of the character strings in each group.

[0022] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for string sorting provided in an embodiment of the present invention.

[0023] According to another aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for sorting strings provided in the embodiments of the present invention is implemented.

[0024] An embodiment of the above invention has the following advantages or beneficial effects: by generating a polynomial corresponding to each string to be sorted; arranging the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and grouping the strings according to the first section of the strictly ascending sequence of the polynomials, the groups have an order; for each group, the polynomials are compared according to the polynomials corresponding to the strings in the group, so as to sort the strings in each group; according to the order of the groups and the sorting results of the strings in each group, the technical solution of obtaining the sorting results of the strings to be sorted is realized, and the form of using polynomial representation is realized, avoiding comparing and sorting the strings as continuous character sequences, and then using the idea of ​​radix sorting to do suffix sorting, thereby improving the efficiency of string sorting. The present invention uses the polynomial representation method and introduces bit operations to more efficiently complete the comparison operation in the sorting algorithm; at the same time, the present invention uses concurrent thread processing in a multi-threaded environment and uses shared memory to maintain cache consistency technology to perform end-to-end algorithm processing, thereby overcoming the performance bottleneck caused by the current suffix sorting algorithm not performing parallel computing.

[0025] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0027] Figure 1 is a schematic diagram of main steps of a method for sorting strings according to an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of the implementation flow of string sorting according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of main modules of a device for sorting strings according to an embodiment of the present invention;

[0030] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0031] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0033] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution disclosed in the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0034] Among the two commonly used methods for sorting string data by suffix, the suffix sorting algorithm based on the suffix tree has high time and space complexity and is quite difficult to construct. Specifically, for a string C1C2…C n , there are n(n+1) / 2 suffixes that need to be stored. After the dictionary tree Tries algorithm O(n 2 ) is optimized to a length of O(n); the process of building the suffix tree has a time complexity of O(alphabat·n), and the pattern matching required by quer4 requires a time complexity of O(alphabet·m), where m is the length of the pattern string. The efficiency of the suffix sorting algorithm based on the suffix array is relatively low. The existing suffix sorting method is basically a serial calculation in a single-threaded environment, and it is not reasonably designed to use concurrent acceleration under multi-threading, which shows disadvantages when facing large-scale data computing scenarios and algorithm degradation caused by uneven data distribution in extreme cases.

[0035] For the purpose of optimizing the performance of existing suffix sorting methods, the present invention proposes a suffix sorting method based on shared memory parallel acceleration represented by polynomials. Since any comparison-based sorting algorithm requires nlogn comparisons, that is, it has an upper limit of time complexity of O(nlogn); and radix sorting, as a distributive sorting algorithm, can stably increase the average time complexity to O(alphabat·n) based on this. Therefore, the present invention uses the form of polynomial representation to avoid comparing and sorting the string as a continuous character sequence, and then uses the idea of ​​radix sorting to perform suffix sorting to improve the efficiency of string data sorting. At the same time, the present invention uses concurrent thread processing and cache consistency technology in a multi-threaded environment to perform end-to-end algorithm processing, thereby overcoming the performance bottleneck caused by the lack of parallel computing in the current suffix sorting algorithm.

[0036] Figure 1 FIG. 1 is a schematic diagram of the main steps of the method for sorting strings according to an embodiment of the present invention. Figure 1 As shown, the method for sorting strings in the embodiment of the present invention mainly includes the following steps S101 to S104.

[0037] Step S101: Generate a polynomial corresponding to each string to be sorted. According to an embodiment of the present invention, when generating a polynomial corresponding to each string to be sorted, it can specifically include: expressing each string to be sorted in the form of a polynomial to obtain a first polynomial; using the number of adjacent identical characters in the first polynomial as coefficients, merging the identical characters in the first polynomial to generate a polynomial corresponding to each string to be sorted. By merging adjacent identical characters in the first polynomial, the space occupied by the polynomial can be reduced, which facilitates data storage and comparison of the size of the polynomial.

[0038] Given a string str = C0C1...C n-1 Expressed in polynomial form as P(n)=x 0 ·x 1 ·…·x n-1 In order to distinguish the same characters that may appear in a string, the sequence of adjacent identical strings is multiplied by the position information as a coefficient, and P(n) = C0x 0 C1x 1 ·…·C n-1 x n-1 In the present invention, the string is represented as a polynomial and then further sorted by suffix bucket. Therefore, str is a string sequence, and P(n) is a simple sequence of powers from low to high, representing the position information of the polynomial. For example, for a sequence str using characters a, b, and c, it can be represented as 1, x, x respectively. 2The dot “·” between each item is not a dot product, but is used as a separator without any actual calculation meaning. The polynomial representations corresponding to some specific strings are shown in Table 1 below.

[0039] Table 1

[0040] String Polynomial Representation babaabcbabaa <![CDATA[x·(1·x)·2·(x·x 2 )·x·(1·x)·2]]> abaabcbabaa <![CDATA[(1·x)·2·(x·x 2 )·x·(1·x)·2]]> baabcbabaa <![CDATA[x·2·(x·x 2 )·x·(1·x)·2]]> aabcbabaa <![CDATA[2·(x·x 2 )·x·(1·x)·2]]> abcbabaa <![CDATA[(1·x·x 2 )·x·(1·x)·2]]> bcbabaa <![CDATA[(x·x 2 )·x·(1·x)·2]]> cbabaa <![CDATA[x 2 ·x·(1·x)·2]]> babaa x·(1·x)·2 abaa 1·x·2 baa x·2 aa 2 a 1

[0041] In Table 1 above, the letters in the string are first represented in lexicographic order (a to z), a=x 0 =1, b = x 1 =x,c=x 2 Etc. At the same time, the strictly ascending items will be merged into the brackets, so "b·a·b" is expressed as x·(1·x), and the following "aa" is not strictly ascending, so it is expressed as 1·1 and can be merged into 2. Similarly, "bc" is expressed as (x·x 2 ); the subsequent "babaa" is represented as x·(1·x)·2. At the same time, because the suffix sorting needs to represent all the suffixes of the string, the above Table 1 deletes one character at the head of the string from top to bottom, thereby obtaining the polynomial representation of all the suffixes. After the string is represented by a polynomial, the string can be sorted based on the polynomial.

[0042] Step S102: Arrange the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and group the strings according to the first strictly ascending sequence of the polynomials; the groups have an order. In this step, all generated polynomial representations are arranged in ascending order according to the lexicographic order (az order), and the same terms in the polynomials are merged and the coefficients are added as counts. At the same time, for each group (also known as bucket), the first strictly ascending sequence of the polynomial representation after suffix sorting is taken as the group ID (bucket ID) for bucket allocation. The results obtained after sorting the strings in Table 1 above and allocating buckets are shown in Table 2 below.

[0043] Table 2

[0044] Sort suffix string Corresponding polynomial Allocate buckets a 1 1 aa 2 1 aabcbabaa <![CDATA[2·(x·x 2 )·x·(1·x)·2]]> 1 abaa 1·x·2 1·x abaabcbabaa <![CDATA[(1·x)·2·(x·x 2 )·x·(1·x)·2]]> 1·x abcbabaa <![CDATA[(1·x·x 2 )·x·(1·x)·2]]> <![CDATA[1·x·x 2 ]]> baa x·2 x baabcbabaa <![CDATA[x·2·(x·x 2 )·x·(1·x)·2]]> x babaa x·(1·x)·2 x babaabcbabaa <![CDATA[x·(1·x)·2·(x·x 2 )·x·(1·x)·2]]> x bcbabaa <![CDATA[(x·x 2 )·x·(1·x)·2]]> <![CDATA[x·x 2 ]]> cbabaa <![CDATA[x 2 ·x·(1·x)·2]]> <![CDATA[x 2 ]]>

[0045] In Table 2 above, since the constant terms are all composed of data items represented as 1, the item marked as 2 is also treated as 1 only (similarly for higher-order terms). For all suffixes generated from the original string, they are sorted in ascending lexicographic order. When the corresponding position is empty, it is smaller and has a more forward position. For example, a < aa, and aa < aabcbabaa. Also according to the lexicographic order, aabcbabaa < abaa because the second a < b. Compare the characters at each position of the string in sequence from left to right. After determining the relative positions, look at the polynomial representations corresponding to each suffix in the second column of the table, and take the first strictly ascending sequence as the bucket position for bucketing. For example, for the polynomial 1·x·2 of the string "abaa", only 1·x can be taken as the bucket position because when comparing the string "abaa" with the string "aabcbabaa", the first strictly ascending sequence is the data item 1·x corresponding to the characters "ab".

[0046] Step S103: For each group, perform a size comparison between the polynomials corresponding to the strings in the group to perform string sorting within each group.

[0047] According to an embodiment of the present invention, for each group, perform a size comparison between the polynomials corresponding to the strings in the group to perform string sorting within each group. Specifically, it may include: starting a process for each group, and through the process, in a way of using shared memory, perform a size comparison between the polynomials corresponding to the strings in the group to perform string sorting within each group. Start a new process for each group (or bucket position) to perform sorting preprocessing operations, and adopt the fastest IPC method - shared memory. Thus, the sorting operations on the strings performed by each process can be visible to other processes. After each bucket independently completes concurrent sorting, directly obtain the sorting result from the shared memory. In this way, the end-to-end algorithm processing process can be performed by utilizing concurrent thread processing and cache coherence technology in a multi-threaded environment, overcoming the performance bottleneck caused by the lack of parallel computing in the current suffix sorting algorithm and improving the string sorting efficiency.

[0048] According to another embodiment of the present invention, for each group, the polynomials corresponding to the strings in the group are compared in size to sort the strings in each group, which may include: for each group, the polynomials corresponding to each string in the group are encoded according to a set representation; and the polynomials are compared in size according to the encoding of the polynomials corresponding to each string in the group to sort the strings in each group. Since the size comparison between polynomials cannot be performed directly in a computer, the polynomials corresponding to each string can be encoded according to a set representation to obtain a binary code that can be recognized by the computer, and then the size comparison is performed according to the binary code to sort the strings.

[0049] According to another embodiment of the present invention, for each group, the polynomials corresponding to the strings in the group are compared in size to sort the strings in each group, including: for each group, the polynomial corresponding to each string in the group is encoded according to a set representation; for any two strings in the group, according to the encoding of the polynomial corresponding to the two strings, an XOR operation is performed and then a specified bit operation is performed to obtain the polynomial where the lowest order term of the different terms in the polynomials corresponding to the two strings is located, so as to compare the size of the polynomials corresponding to the two strings; according to the size comparison result of any two strings in the group, the strings in the group are sorted. When the polynomial corresponding to each string is encoded according to the set representation and then the size comparison is performed, specifically, the encoding of the two polynomials can be XORed and then a specified bit operation can be performed to obtain the polynomial where the lowest order term of the different terms in the polynomials corresponding to the two strings is located, so as to compare the size of the polynomials corresponding to the two strings.

[0050] Among them, the designated bit operation is, for example, n&~(n-1), which can be implemented by the atomic instructions provided by the CPU. Atomic operation is an operation that cannot be interrupted. It is either fully executed or not executed at all, and there will be no partial execution. Atomic operations are usually used in multi-threaded programming to ensure the correctness and consistency of shared resources and prevent data disorder errors in a multi-threaded competition environment. Among them, XOR operation: exclusive OR operation, which is 1 only when the two sides of the comparison are different in binary representation, that is, 0XOR1, 1XOR0 are 1, and 0XOR0, 1XOR1 get 0; & operation: bitwise AND operation, which is 1 only when both sides of the comparison are 1 in binary representation, that is, 0&0, 0&1, 1&0 all get 0, only 1&1=1; ~ operation: bitwise inversion operation, which reverses the 0 and 1 in binary representation, such as ~101=010.

[0051] According to an embodiment of the present invention, the polynomial corresponding to each string in the group is encoded according to a set representation method, which may specifically include: for each string in the group, the length of the string is obtained according to the number of data items in the polynomial corresponding to the string, and the highest power of the polynomial corresponding to the string is obtained; for each data item, the encoding of the data item is obtained according to the index value and coefficient corresponding to the different powers of the variable in the data item, and the power does not exceed the highest power; the encoding of the polynomial corresponding to the string is obtained according to the encoding of each data item in the polynomial corresponding to the string. Specifically, the encoding of the polynomial may be obtained by splicing the encoding of each data item in the polynomial in the order of the data items.

[0052] According to an embodiment of the present invention, the space occupied by the encoding of the polynomial corresponding to the string is 1+n*(m+2) bits, where n is the number of data items in the polynomial corresponding to the string, and m is the highest power. Here, the size of each bit is, for example, a byte.

[0053] The polynomial definition representation is as follows: the highest power m and string length n are counted, the first byte is used to record the string length, the first byte of the following n (m+2) length bytes is used to store the coefficient, and the subsequent bytes store the presence or absence of data items in the polynomial. For example, taking the string "abcdefg" as an example, the corresponding polynomial is represented as x 0 x 1 x 2 x 3 x 4 x 5 x 6 , and its corresponding encoding is, for example, as shown in the following Table 3.

[0054] Table 3

[0055] coefficient <![CDATA[x 6 ]]> <![CDATA[x 5 ]]> <![CDATA[x 4 ]]> <![CDATA[x 3 ]]> <![CDATA[x 2 ]]> <![CDATA[x 1 ]]> <![CDATA[x 0 <!-- 6 -->]]> 1 1 1 1 1 1 1 1

[0056] Taking the string "abaabcbabaa" as an example, its corresponding polynomial is: (1·x)·2·(x·x 2 )·x·(1·x)·2, when it is encoded according to the set representation method, the sequence as a whole is first represented in the format shown in the following Table 4.

[0057] Table 4

[0058] n (number of ascending sequences) Sequence 1 Sequence 2 …… sequence n 6 (1·x) 2 …… 2

[0059] In the second step, each sequence is further encoded and represented. For example, sequence 1 will be represented as 1011 (the highest power m=2), as shown in Table 5 below:

[0060] Table 5

[0061] coefficient <![CDATA[x 2 ]]> <![CDATA[x 1 ]]> <![CDATA[x 0 ]]> 1 0 1 1

[0062] Sequence 2 will be represented as 2001, as shown in Table 6 below:

[0063] Table 6

[0064] coefficient <![CDATA[x 2 ]]> <![CDATA[x 1 ]]> <![CDATA[x 0 ]]> 2 0 0 1

[0065] Other sequences are not listed here. Each subsequence has a length of (m+1+1), so 1+n*(m+2) bits of space are needed to complete the encoding representation (the extra length 1 is one bit to store the number of ascending sequences n).

[0066] Finally, the two polynomials are compared to determine the size of the two strings. After determining the index of the first different byte, perform the XOR operation, and then perform the bit operation n&~(n-1) to obtain the index corresponding to the rightmost set byte and compare it with the two polynomials involved in the comparison. The polynomial with the corresponding position byte also set is the smaller string. This operation is equivalent to comparing and querying in which polynomial the lowest-order term of the different terms appears. For example, for the data item X=x·x with the highest power m=2, 2 The comparison with the data item Y=2 is shown in Table 7 below.

[0067] Table 7

[0068] X: 0 1 1 0 Y: 2 0 0 1 X XOR Y: 2 1 1 1 After n&~(n-1): 0 0 0 1

[0069] Because x 0 The item is set and exists in the polynomial Y, so the Y polynomial is smaller than the X polynomial, thereby realizing the bitwise comparison between the polynomials. In addition, the multi-threaded comparison performed in the suffix bucket process is faster than the traditional method of building a suffix tree or suffix array, and has a better performance implementation space.

[0070] Step S104: According to the order of the groups and the order of the strings in each group, the order of the strings to be sorted is obtained. Finally, according to the order of the groups and the order of the strings in each group, the order of all the strings to be sorted can be obtained.

[0071] Figure 2 FIG. 1 is a schematic diagram of a process flow for implementing string sorting according to an embodiment of the present invention. Figure 2As shown, in one embodiment of the present invention, the implementation process of string sorting mainly includes: expressing the string as a polynomial to obtain a first polynomial; merging the same characters in the first polynomial to generate a polynomial of the string; arranging the polynomials in ascending order according to the lexicographic order and allocating buckets; starting a process for each bucket to perform polynomial encoding of each string, performing XOR operations and bit operations on the encodings of different polynomials to obtain size comparison results between the polynomials; accessing the shared memory between each bucket to obtain the string sorting result.

[0072] Figure 3 4 is a schematic diagram of main modules of a device for sorting strings according to an embodiment of the present invention.

[0073] like Figure 3 As shown, the string sorting device 300 of the embodiment of the present invention mainly includes:

[0074] A polynomial generation module 301 is used to generate a polynomial corresponding to each string to be sorted;

[0075] A first sorting module 302 is used to sort the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and to group the strings according to the first strictly ascending sequence of the polynomials; the groups have a sequence;

[0076] The second sorting module 303 is used to compare the polynomials corresponding to the strings in each group, so as to sort the strings in each group;

[0077] The sorting and summarizing module 304 is used to obtain the sorting result of the character strings to be sorted according to the order of the groups and the sorting result of the character strings in each group.

[0078] According to one embodiment of the present invention, the polynomial generation module 301 can also be used to: express each string to be sorted in the form of a polynomial to obtain a first polynomial; use the number of adjacent identical characters in the first polynomial as coefficients, and merge the identical characters in the first polynomial to generate a polynomial corresponding to each string to be sorted.

[0079] According to another embodiment of the present invention, the second sorting module 303 can also be used to: start a process for each group, and use the shared memory method of the process to compare the sizes of polynomials corresponding to the strings in the group to sort the strings in each group.

[0080] According to another embodiment of the present invention, the second sorting module 303 can also be used to: for each group, encode the polynomial corresponding to each string in the group according to a set representation method; compare the sizes of the polynomials according to the encoding of the polynomial corresponding to each string in the group to sort the strings in each group.

[0081] According to another embodiment of the present invention, the second sorting module 303 can also be used for: for each group, encoding the polynomial corresponding to each string in the group according to a set representation method; for any two strings in the group, performing an XOR operation and then a specified bit operation based on the encoding of the polynomials corresponding to the two strings, to obtain the polynomial of the lowest order term of the different terms in the polynomials corresponding to the two strings, so as to compare the size of the polynomials corresponding to the two strings; and sorting the strings in the group based on the size comparison result of any two strings in the group.

[0082] According to another embodiment of the present invention, the second sorting module 303 can also be used for: for each string in the group, obtaining the length of the string according to the number of data items in the polynomial corresponding to the string, and obtaining the highest power of the polynomial corresponding to the string; for each data item, obtaining the encoding of the data item according to the index values ​​and coefficients corresponding to different powers of the variables in the data item, the power not exceeding the highest power; and obtaining the encoding of the polynomial corresponding to the string according to the encoding of each data item in the polynomial corresponding to the string.

[0083] According to another embodiment of the present invention, the space occupied by the encoding of the polynomial corresponding to the string is 1+n*(m+2) bits, wherein n is the number of data items in the polynomial corresponding to the string, and m is the highest power.

[0084] According to the technical solution of the embodiment of the present invention, by generating a polynomial corresponding to each string to be sorted; arranging the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and grouping the strings according to the first section of the strictly ascending sequence of the polynomials, the groups have an order; for each group, the polynomials are compared according to the polynomials corresponding to the strings in the group, so as to sort the strings in each group; according to the order of the groups and the sorting results of the strings in each group, the technical solution of the sorting results of the strings to be sorted is obtained, and the form of using polynomial representation is realized, avoiding comparing and sorting the strings as continuous character sequences, and then using the idea of ​​radix sorting to do suffix sorting, thereby improving the efficiency of string sorting. The present invention uses a polynomial representation method and introduces bit operations to more efficiently complete the comparison operation in the sorting algorithm; at the same time, the present invention uses concurrent thread processing in a multi-threaded environment and uses shared memory to maintain cache consistency technology to perform end-to-end algorithm processing, thereby overcoming the performance bottleneck caused by the current suffix sorting algorithm not performing parallel computing.

[0085] The present invention performs bucketing according to the characteristics of the generating polynomial and ensures that the information between buckets does not interfere with each other. It has better effects for polynomials with low power (more repeated characters in equal-length inputs). Therefore, it has corresponding application prospects in the fields of signal processing, bioinformatics genome calculation, etc.

[0086] Figure 4 An exemplary system architecture 400 is shown to which the method for sorting strings or the apparatus for sorting strings according to the embodiments of the present invention may be applied.

[0087] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0088] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as data processing applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0089] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0090] Server 405 may be a server that provides various services, such as a background management server that provides support for data sorting websites browsed by users using terminal devices 401, 402, and 403 (for example only). The background management server may generate a polynomial corresponding to each string to be sorted for the received string sorting request and other data; arrange the polynomials corresponding to the strings to be sorted in ascending order according to the dictionary order, and group the strings according to the first section of the strictly ascending sequence of the polynomials, and the groups have an order; for each group, compare the polynomials corresponding to the strings in the group to sort the strings in each group; obtain the sorting result of the strings to be sorted according to the order of the groups and the sorting result of the strings in each group, and feed back the processing result (for example, the sorting result - for example only) to the terminal device.

[0091] It should be noted that the string sorting method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the string sorting device is generally disposed in the server 405 .

[0092] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0093] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 5 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0094] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0095] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0096] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0097] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0098] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0099] The units or modules involved in the embodiments of the present invention may be implemented in software or hardware. The units or modules described may also be arranged in a processor, for example, may be described as: a processor includes a polynomial generation module, a first sorting module, a second sorting module, and a second sorting module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves, for example, the second sorting module may also be described as "a module for obtaining the sorting results of the strings to be sorted according to the order of the groups and the sorting results of the strings in each group".

[0100] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: generating a polynomial corresponding to each string to be sorted; arranging the polynomials corresponding to the strings to be sorted in ascending order according to the dictionary order, and grouping the strings according to the first section of the strictly ascending sequence of the polynomials; the groups have an order; for each group, comparing the polynomials corresponding to the strings in the group, so as to sort the strings in each group; obtaining the sorting result of the strings to be sorted according to the order of the groups and the sorting result of the strings in each group.

[0101] According to the technical solution of the embodiment of the present invention, by generating a polynomial corresponding to each string to be sorted; arranging the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and grouping the strings according to the first section of the strictly ascending sequence of the polynomials, the groups have an order; for each group, the polynomials are compared according to the polynomials corresponding to the strings in the group, so as to sort the strings in each group; according to the order of the groups and the sorting results of the strings in each group, the technical solution of the sorting results of the strings to be sorted is obtained, and the form of using polynomial representation is realized, avoiding comparing and sorting the strings as continuous character sequences, and then using the idea of ​​radix sorting to do suffix sorting, thereby improving the efficiency of string sorting. The present invention uses a polynomial representation method and introduces bit operations to more efficiently complete the comparison operation in the sorting algorithm; at the same time, the present invention uses concurrent thread processing in a multi-threaded environment and uses shared memory to maintain cache consistency technology to perform end-to-end algorithm processing, thereby overcoming the performance bottleneck caused by the current suffix sorting algorithm not performing parallel computing.

[0102] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for sorting strings, characterized in that: include: Generate a polynomial corresponding to each string to be sorted; Arrange the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and group the strings according to the first section of the strictly ascending sequence of the polynomials; the grouping has a sequence; For each group, the polynomials corresponding to the strings in the group are compared to sort the strings in each group; According to the order of the groups and the sorting result of the character strings in each group, the sorting result of the character strings to be sorted is obtained.

2. The method according to claim 1, characterized in that Generate the polynomial corresponding to each string to be sorted, including: Represent each string to be sorted in polynomial form to obtain a first polynomial; The number of adjacent identical characters in the first polynomial is used as a coefficient, and identical characters in the first polynomial are merged to generate a polynomial corresponding to each character string to be sorted.

3. The method according to claim 1, characterized in that For each group, the polynomials corresponding to the strings in the group are compared to sort the strings in each group, including: A process is started for each group, and the polynomials are compared with each other according to the polynomials corresponding to the strings in the group in a shared memory manner through the process, so as to sort the strings in each group.

4. The method according to claim 1, characterized in that: For each group, the polynomials corresponding to the strings in the group are compared to sort the strings in each group, including: For each group, the polynomial corresponding to each string in the group is encoded according to a set representation method; The polynomials are compared in size according to the codes of the polynomials corresponding to each character string in the group, so as to sort the characters in each group.

5. The method according to claim 1, characterized in that For each group, the polynomials corresponding to the strings in the group are compared to sort the strings in each group, including: For each group, the polynomial corresponding to each string in the group is encoded according to a set representation method; For any two strings in the group, according to the codes of the polynomials corresponding to the two strings, an XOR operation is performed and then a specified bit operation is performed to obtain the polynomial of the lowest order term of the different terms in the polynomials corresponding to the two strings, so as to compare the sizes of the polynomials corresponding to the two strings; The strings in the group are sorted according to a size comparison result of any two strings in the group.

6. The method according to claim 4 or 5, characterized in that: Encoding the polynomial corresponding to each string in the group according to a set representation method includes: For each string in the group, obtain the length of the string according to the number of data items in the polynomial corresponding to the string, and obtain the highest power of the polynomial corresponding to the string; For each data item, obtaining the encoding of the data item according to index values ​​and coefficients corresponding to different powers of variables in the data item, wherein the powers do not exceed the highest power; The encoding of the polynomial corresponding to the character string is obtained according to the encoding of each data item in the polynomial corresponding to the character string.

7. The method according to claim 6, characterized in that The space occupied by the encoding of the polynomial corresponding to the string is 1+n*(m+2) bits, where n is the number of data items in the polynomial corresponding to the string, and m is the highest power.

8. A device for sorting character strings, characterized in that: include: A polynomial generation module, used to generate a polynomial corresponding to each string to be sorted; A first sorting module, used to arrange the polynomials corresponding to the strings to be sorted in ascending order according to the lexicographic order, and to group the strings according to the first section of the strictly ascending sequence of the polynomials; the groups have an order; A second sorting module is used to compare the polynomials corresponding to the strings in each group, so as to sort the strings in each group; The sorting and summarizing module is used to obtain the sorting result of the character strings to be sorted according to the order of the groups and the sorting result of the character strings in each group.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images