Data encoding method and device, electronic equipment and computer readable storage medium

Abstract

This disclosure discloses a data encoding method and apparatus, an electronic device, and a computer-readable storage medium. The data encoding method relates to the field of encoding technology and includes: determining multiple columns of data corresponding to a target row in a target data table based on a query sorting statement; performing order-preserving encoding on the strings corresponding to each of the multiple columns of data to obtain a first byte stream for each column; for each first byte stream, performing conversion encoding based on the number of padding characters and the character values ​​of non-padding characters to obtain a second byte stream, the conversion encoding being used to compress the padding characters included in the first byte stream; padding the end of the second byte stream with bytes representing a terminator to obtain a third byte stream; and determining the result byte stream of the target row data based on the third byte streams corresponding to each column of data. This method can maintain order even in complex situations involving comparison of padding characters and merging comparisons of multiple columns of data.

Description

Technical Field

[0001] This disclosure relates to the field of coding technology, specifically to a data coding method and apparatus, electronic equipment, and computer-readable storage medium. Background Technology

[0002] As the size of data in databases continues to grow, the importance of sorting and query execution performance becomes increasingly apparent. Traditionally, encoding the data to be compared in the database into a pre-ordered byte stream is a key method for improving query execution performance.

[0003] However, since data comparison in a database involves multiple semantics and multiple columns of data, it is easy to generate complex situations that combine the comparison of supplementary character strings and the merging comparison of multiple columns of data. Traditional encoding methods cannot obtain ordered byte streams when faced with complex situations of comparing supplementary character strings and merging comparisons of multiple columns of data. Summary of the Invention

[0004] In view of this, the present disclosure provides a data encoding method and apparatus, electronic device and computer-readable storage medium to solve the problem that traditional encoding methods cannot obtain a pre-ordered byte stream when faced with complex situations such as comparing supplementary character strings and merging and comparing multiple columns of data.

[0005] Firstly, a data encoding method is provided, comprising: determining multiple columns of data corresponding to target row data in a target data table based on a query sorting statement; performing order-preserving encoding on the strings corresponding to each of the multiple columns of data to obtain a first byte stream corresponding to each of the multiple columns of data; for each first byte stream, performing conversion encoding on the first byte stream based on the number of padding characters and the character values ​​of non-padding characters included in the first byte stream to obtain a second byte stream, wherein the conversion encoding is used to compress the padding characters included in the first byte stream; padding the end of the second byte stream with bytes representing a terminator to obtain a third byte stream; and determining the result byte stream of the target row data based on the third byte streams corresponding to each of the multiple columns of data.

[0006] Secondly, a data encoding apparatus is provided, comprising: a first determining module configured to determine multiple columns of data corresponding to target row data in a target data table based on a query sorting statement; a first encoding module configured to perform order-preserving encoding on the strings corresponding to each of the multiple columns of data to obtain a first byte stream corresponding to each of the multiple columns of data; a second encoding module configured to perform conversion encoding on each first byte stream based on the number of padding characters and the character values ​​of non-padding characters included in the first byte stream to obtain a second byte stream, wherein the conversion encoding is used to compress the padding characters included in the first byte stream; a padding module configured to pad the end of the second byte stream with bytes representing terminators to obtain a third byte stream; and a second determining module configured to determine the result byte stream of the target row data based on the third byte streams corresponding to each of the multiple columns of data.

[0007] Thirdly, an electronic device is provided, comprising: a processor and a memory for storing computer-executable instructions, the processor being configured to execute the computer-executable instructions to implement the methods mentioned in the first or second aspect above.

[0008] Fourthly, a computer-readable storage medium is provided that stores instructions which, when executed, enable the implementation of the methods mentioned in the first or second aspect above.

[0009] Fifthly, a computer program product is provided, including instructions that, when executed, can implement the methods mentioned in the first or second aspect above.

[0010] The data encoding method provided in this disclosure utilizes the number of padding characters and the character values ​​of non-padding characters in the first byte stream to perform conversion encoding. All padding characters in the first byte stream are compressed to regularize the stream, ensuring correct results even when comparing two rows of data with unequal numbers of padding characters. Furthermore, a terminator is used to represent the result of each column of data, guaranteeing the correctness of variable-length encoding for merging multiple columns. This enables order preservation in complex situations involving padding character string comparisons and multi-column data merging comparisons, thereby optimizing the sorting algorithm, improving query execution performance, and ensuring a good user experience. Attached Figure Description

[0011] Figure 1 The diagram shown is a system architecture diagram of an application scenario of the data encoding method provided in an embodiment of this disclosure.

[0012] Figure 2 The diagram shown is a flowchart illustrating a data encoding method provided in an embodiment of this disclosure.

[0013] Figure 3The diagram shows a flowchart of an embodiment of this disclosure, which converts the number of padding characters included in the padding character set and the character value of the first non-padding character after the padding character set into bytes representing the padding character set to obtain a second byte stream.

[0014] Figure 4 The diagram shown is a flowchart of a data encoding method provided in an embodiment of this disclosure.

[0015] Figure 5 The diagram shown is a structural schematic of a data encoding device provided in an embodiment of this disclosure.

[0016] Figure 6 The diagram shown is a structural schematic of a data encoding device provided in another embodiment of this disclosure.

[0017] Figure 7 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0018] The technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments.

[0019] Sorting and query execution performance is closely related to user experience. A major bottleneck in sorting and query execution performance lies in improving the performance of comparing the size of two data points. On the hardware side, thanks to the efforts of generations of chip designers, the performance of computers comparing memory values ​​is already quite excellent, leaving relatively little room for improvement. Therefore, preserving the order of the data and using the result of this preserving encoding for direct memory comparison becomes crucial for improving the performance of comparing the size of two data points. In other words, preserving the order is an important means of optimizing sorting algorithms to improve query execution performance.

[0020] A data value can be represented as a byte array using a certain encoding method, and the magnitude of two values ​​can be inferred from the encoded byte array. This encoding method is called order-preserving encoding. Order-preserving encoding can also be interpreted as encoding the data to be compared into an order-preserving byte stream. Because database comparisons involve multiple semantics (e.g., MySQL string comparison rules, Oracle string comparison rules, etc.) and the problem of merging and comparing multiple columns of data, traditional encoding methods cannot produce an order-preserving byte stream when faced with the complexity of comparing complemented string comparisons and merging and comparing multiple columns of data.

[0021] The above-mentioned padding character string comparison means that for strings str1 and str2, compare the lengths of the two strings. If the lengths of the two strings are equal, directly compare them according to the ordinary string comparison method. If the two strings are not equal, the shorter string needs to be padded with some characters so that the length of the padded string is the same as that of the longer string, and then compare them according to the ordinary string comparison method.

[0022] For example, taking padding with 0x20 as an example, a and b below are the byte streams corresponding to two columns of data located on two rows in a whole column of data in the database.

[0023] a: 0x5232191823

[0024] b: 0x523221

[0025] Since len(a) > len(b), that is, the string length of a is greater than that of b, so b is padded to the length of a, and b’ obtained is: 0x5232212020. According to the ordinary string comparison method, a < b’ is obtained, so a < b.

[0026] The above-mentioned multi-column data merging comparison means that in the database, a row is composed of several columns. When comparing the sizes of two rows of data, multiple columns are usually involved. For example, for select * from t order by c1, c2 (indicating querying in ascending order of c1 first and in ascending order of c2 for the same c1 in table t of the database), c1 and c2 need to be compared simultaneously. Therefore, in the actual process of order-preserving encoding, the encoded values of c1 and c2 need to be merged into a byte stream while ensuring correctness.

[0027] Combined with the following table, an example is given to illustrate the problem that the traditional encoding method cannot obtain an order-preserving byte stream when facing the complex situations of padding character string comparison and multi-column data merging comparison.

[0028] In Table 1, Row refers to the row of the database, c1 refers to the column in the database, and the byte stream is encoded in an order-preserving manner using the traditional encoding method.

[0029] Table 1

[0030]

[0031] The traditional encoding method uses a direct splicing method when performing multi-column data merging comparison. However, the direct splicing method cannot obtain an order-preserving byte stream when encountering the situation of padding character string comparison.

[0032] Specifically, during the database sorting comparison, the byte streams of Row1 and Row2 in column c1 are compared first. According to the string comparison rules, the byte stream 0x 6162 in column c1 of Row2 becomes 0x 61 62 20 after padding. Since 0x 61 62 20 is less than 0x 61 62 63, Row1 is directly determined to be less than Row2. After merging c1 and c2, 0x 61 62 6465 66 becomes 0x 61 62 64 65 66 20 after padding. Since 0x 61 62 64 65 66 20 is greater than 0x 61 62 63 6566 67, Row1 is determined to be greater than Row2.

[0033] Originally, when comparing single-column padding characters, Row1 is less than Row2. However, when comparing padding character strings or merging multiple columns of data, Row1 is greater than Row2. Therefore, the traditional direct concatenation method cannot obtain a byte stream that preserves order.

[0034] To address the aforementioned issues, this disclosure provides a data encoding method. This method utilizes the number of padding characters and the values ​​of non-padding characters in the first byte stream to perform conversion encoding. All padding characters in the first byte stream are compressed to regularize the stream, ensuring correct results even when comparing two rows of data with unequal numbers of padding characters. Furthermore, a terminator is used to represent the result of each column of data, guaranteeing the correctness of variable-length encoding for merging multiple columns. This enables order preservation in complex situations involving padding character string comparisons and multi-column data merging comparisons, thereby optimizing the sorting algorithm, improving query execution performance, and ensuring a good user experience.

[0035] The following is combined Figure 1 The system architecture of the application scenario of the data encoding method is illustrated with an example.

[0036] like Figure 1 As shown, the application scenario of the data encoding method provided in this embodiment involves a user terminal 110 and a database system 120 that is communicatively connected to the user terminal 110. The user terminal 110 deploys an application that uses the database system, and the database system 120 deploys the data encoding method provided in this embodiment.

[0037] In practical use, the user inputs a query sorting statement (such as the aforementioned `select * from t order by c1,c2`) on user terminal 110. Based on the query sorting statement, the database system determines the multiple columns of data corresponding to the target row in the target data table (i.e., c1 and c2 corresponding to the target row). The strings corresponding to c1 and c2 are then encoded in order-preserving manner to obtain the first byte stream for each. For each first byte stream, based on the number of padding characters and the values ​​of non-padding characters, a conversion encoding is performed to obtain the second byte stream. This conversion encoding compresses all padding characters in the first byte stream to regulate its structure. Furthermore, bytes representing terminators are added to the end of the second byte stream to obtain the third byte stream. Finally, based on the third byte streams corresponding to c1 and c2, the result byte stream of the target row data is determined.

[0038] The data encoding method disclosed herein can produce an ordered byte stream even when faced with complex situations such as comparing supplementary character strings and merging and comparing multiple columns of data. This optimizes the sorting algorithm in the database, thereby improving the database query execution performance and thus enhancing the user experience.

[0039] For example, user terminal 110 can be a mobile phone, tablet computer, or desktop computer. Database system 120 includes multiple servers, which can be physical servers or cloud servers. Database system 120 can be a relational database system, such as MySQL.

[0040] The following is combined Figures 2 to 4 The data encoding methods mentioned in the embodiments of this disclosure are described in detail.

[0041] Figure 2 The diagram shown is a flowchart illustrating a data encoding method provided in an embodiment of this disclosure. Figure 2 As shown in the embodiments of this disclosure, the data encoding method includes the following steps.

[0042] Step S210: Based on the query sorting statement, determine the multiple columns of data corresponding to the target row data in the target data table.

[0043] For example, the query sorting statement mentioned above refers to the Structured Query Language (SQL) used in a database to sort data in order to view the data.

[0044] The above-mentioned determination of multiple column data corresponding to the target row data in the target data table based on the query sorting statement can be implemented as determining multiple sorting keys in the query sorting statement, and the column data corresponding to the multiple sorting keys in the target row data constitutes the multiple column data to be encoded.

[0045] Step S220: Respectively perform order-preserving encoding on the strings corresponding to the multiple column data to obtain the first byte streams corresponding to the multiple column data.

[0046] Exemplarily, the specific implementation manner of respectively performing order-preserving encoding on the strings corresponding to the multiple column data to obtain the first byte streams corresponding to the multiple column data is as follows: For each column data in the multiple column data, according to the data type corresponding to the column data, select an order-preserving encoding method matching the data type, and then encode the string corresponding to the column data based on the matching order-preserving encoding method to obtain the first byte stream corresponding to the column data. Among them, the data types include, but are not limited to, int type, ob number type, etc. The order-preserving encoding methods include Unicode, ASCII, GBK, GB2312, UTF-8, UTF-16, etc.

[0047] For example, the byte stream obtained by encoding the string "严" using Unicode is 0xFF FE 25 4E.

[0048] Step S230: For each first byte stream, perform transformation encoding on the first byte stream based on the number of padding characters included in the first byte stream and the character values of the non-padding characters to obtain the second byte stream.

[0049] The purpose of performing transformation encoding on the first byte stream based on the number of padding characters included in the first byte stream and the character values of the non-padding characters is to compress all the padding characters included in the first byte stream to regularize the first byte stream, so that the column data in different target row data of a whole column can be compared in a regular format, avoiding the situation where the order-preserving byte stream cannot preserve order due to the addition of padding characters.

[0050] Performing transformation encoding on the first byte stream based on the number of padding characters included in the first byte stream and the character values of the non-padding characters to obtain the second byte stream, the specific implementation manner includes: If there is a byte representing a padding character at the end of the first byte stream, delete the byte representing the padding character. If there is a byte representing a padding character set at a non-end position of the first byte stream, transform the byte representing the padding character set based on the number of padding characters included in the padding character set and the character value of the first non-padding character after the padding character set to obtain the second byte stream, and the padding character set includes one padding character or consecutive multiple padding characters.

[0051] In the above description, deleting the bytes representing padding characters at the end is to achieve a neat ending. Furthermore, considering that bytes representing a single padding character or multiple consecutive padding characters at non-end positions in the first byte stream can easily cause errors that prevent the ordered byte stream from maintaining its order, the bytes representing the padding character set are transformed by considering the number of padding characters included in the padding character set and the character value of the first non-padding character after the padding character set. This ensures that bytes representing the padding character set at non-end positions are presented in a neat and specific format, thereby achieving order preservation when facing complex situations such as comparing padding character strings and merging multiple columns of data.

[0052] For example, if the first byte stream is 0xFE 56 36 20 35 20 20, it means the padding character byte is 0x20. Deleting the last two padding characters and arranging the end of the first byte stream, we get 0xFE 56 36 20 35. The byte 0x20 between 0x36 and 0x35 in 0xFE 56 36 20 35 is the padding character set. That is, this padding character set includes one padding character byte. Using the encoded value of the first non-padding character byte 0x35 after 0x20 and a 1, we can convert 0x20.

[0053] If the first byte stream is 0x FE 56 36 20 19 20 20 20, remove the last three padding bytes to neatly arrange the end of the first byte stream, resulting in 0x FE 56 36 20 19. The byte 0x 20 between 0x 36 and 0x 19 in 0x FE 56 36 20 19 is the padding character set, which includes one padding character byte. Using the encoded value of the first non-padding character byte 0x 19 after 0x 20 and a 1, 0x 20 is converted.

[0054] Step S240: Fill the end of the second byte stream with bytes representing the terminator to obtain the third byte stream.

[0055] The purpose of padding the end of the second byte stream with bytes representing the terminator is to accurately determine the termination position of each column of data corresponding to the target row data, so as to reduce the probability that the order-preserving encoding caused by merging multiple columns of data cannot be preserved.

[0056] For example, the method for determining the byte representing the terminator includes: determining a preset byte corresponding to the padding character included in the first byte stream; determining two adjacent preset bytes as bytes representing the terminator, wherein the bytes representing the terminator are used to indicate the termination position of the byte stream.

[0057] Step S250: Determine the result byte stream of the target row data based on the third byte stream corresponding to each of the multiple columns of data.

[0058] The purpose of determining the result byte stream of the target row data based on the third byte streams corresponding to each of the multiple columns of data is that, since memory reads in the database are performed using the result byte stream of the target row data, after obtaining the third byte streams corresponding to each of the multiple columns of data, it is necessary to determine the result byte stream of the target row data in order to write it into memory. Further exemplified, this is used to display the sorted query results to the user.

[0059] In some embodiments, determining the result byte stream of the target row data based on the third byte streams corresponding to each of the multiple columns of data can be implemented as follows: determining the sort key of the multiple columns of data in the query sorting statement; concatenating the third byte streams corresponding to each of the multiple columns of data according to the sort key order to obtain the result byte stream.

[0060] The data encoding method provided in this disclosure utilizes the number of padding characters and the character values ​​of non-padding characters in the first byte stream to perform conversion encoding. All padding characters in the first byte stream are compressed to regularize the stream, ensuring correct results even when comparing two rows of data with unequal numbers of padding characters. Furthermore, a terminator is used to represent the result of each column of data, guaranteeing the correctness of variable-length encoding for merging multiple columns. This enables order preservation in complex situations involving padding character string comparisons and multi-column data merging comparisons, thereby optimizing the sorting algorithm, improving query execution performance, and ensuring a good user experience.

[0061] The following is combined Figure 3 This section details how to transform the bytes representing the padding character set, based on the number of padding characters included in the padding character set and the character value of the first non-padding character following the padding character set, to obtain the second byte stream. For example... Figure 3 As shown in this embodiment, the step of converting the bytes representing the padding character set to obtain the second byte stream based on the number of padding characters included in the padding character set and the character value of the first non-padding character after the padding character set includes the following steps.

[0062] Step S310: Determine the preset bytes corresponding to the padding characters included in the padding character set.

[0063] For example, the character set corresponding to the order-preserving encoding method of the first byte stream is obtained, and then the preset bytes corresponding to the padding characters are determined based on this character set. Different character sets have different weights for their string comparison rules (collations). For instance, the weight value of utf8mb4_bin is 1 byte, while utf8mb4_general_ci requires converting multibyte characters to Unicode, then calculating the weight value (2 bytes long) based on the Unicode. The byte length of the weight value is defined as the encoding unit. Different encoding units result in different forms for the preset bytes corresponding to the padding characters.

[0064] For example, the preset bytes for padding characters in different character sets are shown in Table 2 below.

[0065] Table 2

[0066]

[0067]

[0068] Step S320: Based on the preset bytes, generate the bytes corresponding to the padding character identifier.

[0069] The padding character identifier is used to characterize the relationship between the value of the first non-padding character and the values ​​of the padding characters included in the padding character set.

[0070] In one example, if the value of the first non-padding character is greater than the value of any padding character in the padding character set, then the combination of a preset byte and a byte that is greater than the preset byte is determined as the byte corresponding to the padding character identifier. In this case, the byte of the padding character identifier is also called the byte greater than the identifier.

[0071] For example, the default byte for the padding character is 0x20. In 0xFE 56 36 20 35, the byte 0x20 between 0x36 and 0x25 is the padding character set. The byte corresponding to the first non-padding character after the padding character set is 0x35. By comparing 0x35 and 0x20, it is determined that the character value of the first non-padding character is greater than the character value of the padding character. Therefore, 0x2021 is used as the byte for the padding character identifier.

[0072] In another example, if the character value of the first non-padding character is less than the character value of the padding characters included in the padding character set, then the combination of the preset byte and the byte smaller than the preset byte is determined as the byte corresponding to the padding character identifier. In this case, the byte of the padding character identifier is also called the byte smaller than the identifier.

[0073] For example, in 0x FE 56 36 20 19, the byte 0x 20 between 0x 36 and 0x 19 is the padding character set. The byte corresponding to the first non-padding character after the padding character set is 0x 19. By comparing 0x 19 and 0x 20, it is determined that the character value of the first non-padding character is less than the character value of the padding character. Therefore, 0x 20 19 is used as the byte of the padding character identifier.

[0074] For example, the filler character identifiers for different character sets are shown in Table 3 below.

[0075] Table 3

[0076] Encoding unit bytes greater than the identifier bytes smaller than the identifier 1 byte 0x20 21 0x20 19 2 bytes 0x0020 0021 0x0020 0019 4 bytes 0x00000020 00000021 0x00000020 00000019

[0077] Step S330: Based on the size relationship between the character value of the first non-filling character and the character values ​​of the filling characters included in the filling character set, determine the number of bytes corresponding to the number of filling characters included in the filling character set.

[0078] If the value of the first non-padding character is greater than the value of any padding character in the padding character set, then a negative count value corresponding to the number of padding characters in the padding character set is determined. The negative count value is a negative number, and its absolute value equals the number of padding characters in the padding character set. The byte corresponding to the negative count value is determined as the number of padding characters in the padding character set. Specifically, when the padding character identifier's byte value is greater than the identifier's byte value, the more padding characters in the padding character set, the smaller the string will appear in the comparison. Therefore, the byte corresponding to the negative count value is determined as the number of padding characters in the padding character set.

[0079] For example, the default byte for padding characters is 0x20. In 0xFE 56 36 20 35, the byte 0x20 between 0x36 and 0x25 is the padding character set. The padding character set includes a byte for one padding character. Based on the fact that the character value of the first non-padding character is greater than the character value of the padding character, -1 is converted into bytes as the bytes corresponding to the number of padding characters. That is, the bytes corresponding to the number of padding characters are convert_int32_bytes(-1) = 0x7F.

[0080] If the value of the first non-padding character is less than the value of any padding character in the padding character set, then the count value corresponding to the number of padding characters in the padding character set is determined, and the byte corresponding to this count value is set as the number of padding characters in the padding character set. Specifically, when the byte value of the padding character identifier is less than the byte value of the identifier, the more bytes of padding characters in the padding character set, the larger the string will be during comparison. Therefore, the byte corresponding to this count value is set as the number of padding characters in the padding character set.

[0081] For example, in 0x FE 56 36 20 19, the byte 0x 20 between 0x 36 and 0x 19 is the padding character set. The padding character set includes a byte with 1 padding character. Based on the premise that the character value of the first non-padding character is less than the character value of the padding character, 1 is converted into a byte as the byte corresponding to the number of padding characters. That is, the byte corresponding to the number of padding characters is convert_int32_bytes(1) = 0x 80.

[0082] Step S340: Combine the bytes corresponding to the padding character identifier and the bytes corresponding to the number of padding characters included in the padding character set to obtain the combined bytes.

[0083] For example, in 0x FE 56 36 20 35, the bytes for the padding character identifier are 0x 20 21, the bytes corresponding to the number of padding characters are 0x 7F, and the combined bytes are 0x 20 21 7F.

[0084] In 0x FE 56 36 20 19, the bytes for the padding character identifier are 0x 20 19, the bytes corresponding to the number of padding characters are 0x 80, and the combined bytes are 0x 20 19 80.

[0085] Step S350: Replace the bytes representing the padding character set with combined bytes to obtain the second byte stream.

[0086] For example, replacing 0x 20 in 0x FE 56 36 20 35 with 0x 20 21 7F will result in the second byte stream being 0x FE 56 36 20 21 7F 35.

[0087] Replacing 0x 20 in 0x FE 56 36 20 19 with 0x 20 19 80 results in the second byte stream being 0x FE56 36 20 19 80 19.

[0088] In this embodiment, by processing the byte order of the padding characters in the first byte stream, the encoding results are ensured to be aligned when comparing padding characters. This ensures that when comparing two target rows of data, the data before the padding characters are equal, and the correct comparison result can be obtained even if the number of padding characters is different. In other words, bytes representing the padding character set at non-end positions are presented in a regular and specific format, thereby achieving the purpose of preserving order when facing complex situations such as comparing padding character strings and merging and comparing multiple columns of data.

[0089] As described above, when the preset byte for the padding character is 0x20 and the encoding unit is 1 byte, the character groups 0x2020, 0x2019, and 0x2021 are prone to ambiguity. Although these three character groups have special meanings and may be present in the resulting byte stream, the ambiguity of these three character groups can be eliminated in the resulting byte stream obtained using the steps described above.

[0090] Specifically, the encoding method of this embodiment processes all whitespace characters, ensuring that the 0x20 character will not exist in the resulting byte stream. Therefore, in the resulting byte stream, 0x20 will only have the semantic meaning of an escape character, without any ambiguity. In other words, throughout the entire encoding method, the character groups starting with 0x20 only contain three cases: 0x2019, 0x2020, and 0x2021. Furthermore, these three character groups contain only bytes smaller than the identifier byte, a terminator byte, and bytes larger than the identifier byte. Based on this, the ambiguity of these three characters is eliminated, thereby further ensuring the correctness and reliability of the encoding.

[0091] To better illustrate the data encoding methods mentioned above, the following section will combine... Figure 4 Further examples will be provided. For instance... Figure 4 As shown, the three strings are str1 = "abc\t", str2 = "ab c", and str3 = "ab\r". Using utf8mb4_bin, the three strings are encoded in order-preserving manner, resulting in the following first-byte streams: 0x FE 56 36, 0x FE 56 3620 35 20 20, and 0x FE 56 36. The default byte for the padding character is 0x 20.

[0092] For the first byte stream corresponding to str1, 0x FE 56 36, there are no bytes representing padding characters at the end of the first byte stream, so no operation is needed. Also, there are no bytes representing padding character sets at the non-end positions of the first byte stream, so no operation is needed there either. Therefore, the second byte stream corresponding to str1 is the same as the first byte stream, but with 0x20 20 padded at the end, resulting in the third byte stream corresponding to str1 being 0x FE 56 36 20 20.

[0093] For the first byte stream corresponding to str2, 0x 56 36 20 35 20 20, there are bytes representing padding characters at the end of the first byte stream. Therefore, delete the last two padding character bytes, and the end of the first byte stream is 0xFE 56 36 20 35. The byte 0x 20 between 0x 36 and 0x 25 in 0xFE 56 36 20 35 is the padding character set. The byte corresponding to the first non-padding character after the padding character set is 0x 35. By comparing 0x 35 and 0x 20, it is determined that the character value of the first non-padding character is greater than the character value of the padding character. Therefore, the byte 0x 20 21, which is greater than the identifier, is used as the padding character identifier byte. The padding character set includes one padding character byte, which is greater than the identifier byte. Therefore, the number of padding characters corresponds to the number of bytes convert_int32_bytes(-1) = 0x 7F. The combined byte is 0x 20 21 7F. Replacing 0x 20 in 0x FE 56 36 20 35 with 0x 20 21 7F results in the second byte stream being 0x FE 56 36 2021 7F 35. The terminator is 0x 20 20, so the third byte stream corresponding to str2 is 0x FE 56 36 20 21 7F 35 2020.

[0094] For the first byte stream corresponding to str3, 0x 56 36 20 19 20 20 20, there are bytes representing padding characters at the end of the first byte stream. Delete the last three padding character bytes and normalize the end of the first byte stream to get 0xFE 56 36 20 19. The byte 0x 20 between 0x 36 and 0x 19 in 0xFE 56 36 20 19 is the padding character set. The byte corresponding to the first non-padding character after the padding character set is 0x 19. By comparing 0x 19 and 0x 20, it is determined that the character value of the first non-padding character is less than the character value of the padding character. Therefore, the byte 0x 20 19, which is greater than the identifier, is used as the byte of the padding character identifier. The padding character set includes one padding character byte, which is less than the identifier byte. Therefore, the number of padding characters corresponds to the number of bytes in convert_int32_bytes(1) = 0x 80. The combined byte is 0x 20 21 80. Replacing 0x 20 in 0x FE 56 36 20 35 with 0x 20 21 80 results in the second byte stream being 0x FE 56 36 20 19 80 19. The terminator is 0x 20 20, so the third byte stream corresponding to str3 is 0x FE 56 36 20 19 80 19 20 20.

[0095] The three third-byte streams are concatenated according to their respective sort keys to obtain the resulting byte stream 0x FE 5636 20 20 FE 56 36 20 21 7F 35 20 20 FE 56 36 20 19 80 19 20 20.

[0096] Therefore, the embodiments of this disclosure can maintain order when faced with complex situations such as comparing supplementary character strings and merging and comparing multiple columns of data, thereby optimizing the sorting algorithm in the database, improving the database query execution performance, and ensuring a good user experience.

[0097] The above text combined Figures 2 to 4 The present disclosure describes in detail the method embodiments, which are then combined with the following. Figure 5 The apparatus embodiments of this disclosure are described in detail below. Furthermore, it should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0098] Figure 5 The diagram shown is a structural schematic of a data encoding device provided in an embodiment of this disclosure. Figure 5 As shown, the data encoding device 500 provided in this embodiment includes a first determining module 510, a first encoding module 520, a second encoding module 530, a filling module 540, and a second determining module 550.

[0099] In this embodiment, the first determining module 510 is configured to determine multiple columns of data corresponding to the target row data in the target data table based on a query sorting statement. The first encoding module 520 is configured to perform order-preserving encoding on the strings corresponding to each of the multiple columns of data to obtain a first byte stream corresponding to each column of data. The second encoding module 530 is configured to perform conversion encoding on each first byte stream based on the number of padding characters and the character values ​​of non-padding characters, to obtain a second byte stream. The conversion encoding is used to compress the padding characters included in the first byte stream. The padding module 540 is configured to pad the end of the second byte stream with bytes representing a terminator to obtain a third byte stream. The second determining module 550 is configured to determine the result byte stream of the target row data based on the third byte streams corresponding to each of the multiple columns of data.

[0100] Figure 6 The diagram shown is a structural schematic of a data encoding device provided in another embodiment of this disclosure. Figure 6 Extending from the illustrated embodiment Figure 5 The illustrated embodiment will be described in detail below. Figure 6 The illustrated embodiments and Figure 5 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0101] like Figure 6 As shown, the second encoding module 530 provided in this embodiment includes a deletion unit 610 and a conversion unit 620.

[0102] In this embodiment of the disclosure, the deletion unit 610 is configured to delete the byte representing a padding character if there is one at the end of the first byte stream. The conversion unit 620 is configured to convert the byte representing the padding character set based on the number of padding characters included in the padding character set and the character value of the first non-padding character after the padding character set, to obtain a second byte stream, wherein the padding character set includes one padding character or multiple consecutive padding characters.

[0103] In some embodiments, the conversion unit 620 is further configured to: determine a preset byte corresponding to the fill characters included in the fill character set; generate a byte corresponding to the fill character identifier based on the preset byte, the fill character identifier being used to characterize the size relationship between the character value of the first non-fill character and the character values ​​of the fill characters included in the fill character set; determine a byte corresponding to the number of fill characters included in the fill character set based on the size relationship between the character value of the first non-fill character and the character values ​​of the fill characters included in the fill character set; combine the byte corresponding to the fill character identifier and the byte corresponding to the number of fill characters included in the fill character set to obtain a combined byte; and replace the byte characterizing the fill character set with the combined byte to obtain a second byte stream.

[0104] In some embodiments, the conversion unit 620 is further configured to, if the character value of the first non-filler character is greater than the character value of the filler characters included in the filler character set, determine the combination of a preset byte and a byte that is a preset value greater than the preset byte as the byte corresponding to the filler character identifier. If the character value of the first non-filler character is less than the character value of the filler characters included in the filler character set, determine the combination of a preset byte and a byte that is a preset value less than the preset byte as the byte corresponding to the filler character identifier.

[0105] In some embodiments, the conversion unit 620 is further configured to: if the character value of the first non-filler character is greater than the character value of the filler characters included in the filler character set, then determine a negative quantity value corresponding to the number of filler characters included in the filler character set, wherein the negative quantity value is a negative number and the absolute value of the negative quantity value is equal to the number of filler characters included in the filler character set; and determine the byte corresponding to the negative quantity value as the byte corresponding to the number of filler characters included in the filler character set. If the character value of the first non-filler character is less than the character value of the filler characters included in the filler character set, then determine a quantity value corresponding to the number of filler characters included in the filler character set; and determine the byte corresponding to the quantity value as the byte corresponding to the number of filler characters included in the filler character set.

[0106] In some embodiments, the method for determining the byte representing the terminator includes: determining a preset byte corresponding to a padding character included in the first byte stream; determining two adjacent preset bytes as bytes representing the terminator, wherein the bytes representing the terminator are used to indicate the termination position of the byte stream.

[0107] In some embodiments, the second determining module 550 is further configured to determine the sorting keys of the multiple columns of data in the query sorting statement; and to connect the third byte streams corresponding to the multiple columns of data according to the sorting key order to obtain the result byte stream.

[0108] Figure 7 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this disclosure. Figure 7 The illustrated electronic device 700 includes a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the bus 704.

[0109] The memory 701 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 701 may store a program, and when the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the various steps of the data encoding method of the embodiments of this disclosure.

[0110] The processor 702 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, for executing relevant programs to achieve the functions required by the units in the data encoding apparatus of this disclosure embodiment.

[0111] The processor 702 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the data processing method of this disclosure can be completed by the integrated logic circuits in the hardware of the processor 702 or by instructions in software form. The processor 702 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 701. The processor 702 reads the information in the memory 701 and, in conjunction with its hardware, performs the functions required by the units included in the data encoding apparatus of this disclosure embodiment, or executes the data encoding method of this disclosure method embodiment.

[0112] The communication interface 703 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the electronic device 700 and other devices or communication networks. For example, the communication interface 703 can receive promotional requests or promotional data.

[0113] Bus 704 may include a pathway for transmitting information between various components of electronic device 700 (e.g., memory 701, processor 702, communication interface 703).

[0114] It should be noted that, although Figure 7 The illustrated electronic device 700 only shows the memory, processor, and communication interface. However, those skilled in the art should understand that in specific implementations, the electronic device 700 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the electronic device 700 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that the electronic device 700 may only include the devices necessary for implementing the embodiments of this disclosure, and may not necessarily include... Figure 7 All the devices shown.

[0115] In addition to the methods, apparatus, and devices described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the various steps of the data encoding methods provided in the various embodiments of this disclosure.

[0116] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0117] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform various steps of the data encoding methods provided in the various embodiments of this disclosure.

[0118] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0120] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0121] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional units in the various embodiments of this disclosure can be integrated into a similar region segmentation unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0124] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0125] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A data encoding method, comprising: determining, based on a query order statement, a plurality of column data corresponding to target row data in a target data table; respectively performing order-preserving encoding on a plurality of strings respectively corresponding to the plurality of column data to obtain a plurality of first byte streams respectively corresponding to the plurality of column data; for each of the plurality of first byte streams, performing transformation encoding on the first byte stream based on a number of padding characters included in the first byte stream and character values of non-padding characters included in the first byte stream to obtain a second byte stream, the transformation encoding being configured to compress the padding characters included in the first byte stream; padding a byte representing a terminator at an end of the second byte stream to obtain a third byte stream; determining, based on the plurality of third byte streams respectively corresponding to the plurality of column data, a result byte stream of the target row data. the performing transformation encoding on the first byte stream based on the number of padding characters included in the first byte stream and the character values of the non-padding characters included in the first byte stream to obtain the second byte stream comprises: if a byte representing a padding character exists at an end of the first byte stream, deleting the byte representing the padding character; if a byte representing a padding character set exists at a non-end of the first byte stream, transforming the byte representing the padding character set based on a number of padding characters included in the padding character set and a character value of a first non-padding character after the padding character set, the padding character set including one padding character or a plurality of consecutive padding characters. 2.The method of claim 1, the transforming the byte representing the padding character set based on the number of padding characters included in the padding character set and the character value of the first non-padding character after the padding character set to obtain the second byte stream comprises: determining a preset byte corresponding to a padding character included in the padding character set; generating a byte corresponding to a padding character identifier based on the preset byte, the padding character identifier being configured to represent a size relationship between the character value of the first non-padding character and a character value of the padding character included in the padding character set; determining a byte corresponding to the number of padding characters included in the padding character set based on the size relationship between the character value of the first non-padding character and the character value of the padding character included in the padding character set; combining the byte corresponding to the padding character identifier and the byte corresponding to the number of padding characters included in the padding character set to obtain a combined byte; replacing the byte representing the padding character set with the combined byte to obtain the second byte stream. 3.The method of claim 2, the generating the byte corresponding to the padding character identifier based on the preset byte comprises: if the character value of the first non-padding character is greater than the character value of the padding character included in the padding character set, determining, as the byte corresponding to the padding character identifier, a combination of the preset byte and a byte greater than the preset byte by a preset value. If the character value of the first non-padding character is less than the character value of the padding character included in the padding character set, the combination of the preset byte and the byte smaller than the preset byte by the preset value is determined as the byte corresponding to the padding character identifier.

4. The method of claim 2, wherein the determining the byte corresponding to the number of padding characters included in the padding character set based on the size relationship between the character value of the first non-padding character and the character value of the padding character included in the padding character set comprises: if the character value of the first non-padding character is greater than the character value of the padding character included in the padding character set, determining a negative number value corresponding to the number of padding characters included in the padding character set, the negative number value being a negative number, and an absolute value of the negative number value being equal to the number of padding characters included in the padding character set; determining the byte corresponding to the negative number value as the byte corresponding to the number of padding characters included in the padding character set; if the character value of the first non-padding character is less than the character value of the padding character included in the padding character set, determining a number value corresponding to the number of padding characters included in the padding character set; and determining the byte corresponding to the number value as the byte corresponding to the number of padding characters included in the padding character set.

5. The method of any one of claims 1 to 4, wherein the determining the byte representing the end of the byte stream comprises: determining a preset byte corresponding to the padding character included in the first byte stream; and determining two adjacent preset bytes as the byte representing the end of the byte stream.

6. The method of any one of claims 1 to 4, wherein the determining the result byte stream of the target row data based on the third byte streams respectively corresponding to the plurality of columns of data comprises: determining the sorting keys of the plurality of columns of data in the query sorting statement respectively; and connecting the third byte streams respectively corresponding to the plurality of columns of data in the order of the sorting keys to obtain the result byte stream.

7. A data encoding apparatus, comprising: a first determining module configured to determine, based on a query sorting statement, a plurality of columns of data corresponding to target row data in a target data table; a first encoding module configured to perform order-preserving encoding on a string respectively corresponding to each of the plurality of columns of data to obtain a first byte stream respectively corresponding to each of the plurality of columns of data; a second encoding module configured to, for each of the first byte streams, perform transformation encoding on the first byte stream based on a number of padding characters included in the first byte stream and a character value of a non-padding character to obtain a second byte stream, the transformation encoding being used to compress the padding characters included in the first byte stream; a padding module configured to pad a byte representing an end of character at the end of the second byte stream to obtain a third byte stream; and a second determining module configured to determine, based on the third byte streams respectively corresponding to the plurality of columns of data, a result byte stream of the target row data. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The second encoding module is further configured to, if a byte representing a padding character exists at the end of the first byte stream, delete the byte representing the padding character; and if a byte representing a padding character set exists at a non-end position of the first byte stream, convert the byte representing the padding character set based on a number of padding characters included in the padding character set and a character value of a first non-padding character after the padding character set, the padding character set including one padding character or a plurality of padding characters in succession. 8.An electronic device comprising: a processor; and a memory having computer program instructions stored therein, the computer program instructions, when executed by the processor, causing the processor to perform the method of any one of claims 1 to 6. 9.A computer-readable storage medium having computer program instructions stored therein, the computer program instructions, when executed by a processor, causing the processor to perform the method of any one of claims 1 to 6.

Images