Encoding method and apparatus

Abstract

The present disclosure provides an encoding method and device. The method comprises: obtaining to-be-encoded data; storing each basic type field in the to-be-encoded data in a basic type storage area in sequence according to the order of the to-be-encoded data, wherein each basic type field is stored according to the actual byte length of the respective type; for each row in the to-be-encoded data, storing the address of the string data of each string field in the current row in a string address storage area in sequence according to the order of the corresponding string field in the current row, wherein each address of the string data is stored according to the actual byte length of the respective address, and the string address storage area is located after the basic type storage area; and taking the to-be-encoded data after the storage as encoded data.

Description

Technical Field

[0001] This application relates to the field of data processing, and the following description relates to an encoding method and apparatus. Background Technology

[0002] Currently, in in-memory databases or other memory-sensitive scenarios, memory usage, encoding / decoding efficiency, and the ability to efficiently perform projection (reading a subset of columns at a time) are several crucial system metrics. Related technologies such as TiDB and Spark encoding / decoding consume significant amounts of memory, which cannot meet the requirements of in-memory databases. Figure 1 As shown in the Spark encoding and decoding, primitive type fields are all of fixed byte length. For example, primitive type fields occupy 8 bytes. When there is a primitive type bool, since it only occupies 1 byte, it actually uses 8 bytes, which causes memory waste. Moreover, storing primitive types and strings together (such as int / double as primitive fields and string as strings) also makes subsequent decoding more complicated. Summary of the Invention

[0003] The exemplary embodiments disclosed herein may at least solve the above-described problems, or may not solve the above-described problems.

[0004] According to a first aspect of this disclosure, an encoding method is provided, the method comprising: acquiring data to be encoded; storing each basic type field in the data to be encoded consecutively in a basic type storage area according to the order in which the data is to be encoded, wherein each basic type field is stored according to the actual byte length of its respective type; for each row in the data to be encoded, storing the addresses of the string data of each string field in the current row consecutively in a string address storage area according to the order of the corresponding string fields in the current row, wherein the address of each string data is stored according to the actual byte length of its respective address, and the string address storage area is located after the basic type storage area; and using the re-stored data to be encoded as the encoded data.

[0005] Optionally, storing the addresses of the string data for each string field in the current row consecutively in the string address storage area according to the order of the corresponding string fields in the current row includes: storing the string data for each string field in the current row consecutively in the string data storage area according to the order of the current row; and storing the addresses of the string data in the string data storage area consecutively in the string address storage area according to the order of the corresponding string fields in the current row, wherein the string address storage area is located after the basic type storage area and before the string data storage area, and the byte length of the string address storage area is fixed.

[0006] Optionally, the byte length of the string address storage area is determined as follows: the byte length of all areas preceding the string address storage area is determined; for each line of data to be encoded, the byte length of the string address storage area is determined by the byte length of all areas preceding the string address storage area, the number of fields in the current line, and the byte length of the string data of all string fields in the current line.

[0007] Optionally, all areas preceding the string address storage area include: a basic type storage area, an area for identifying whether each field in the current line is NULL, and an area for identifying the total byte length of the encoded current line.

[0008] Optionally, the byte length of the region used to identify whether each field in the current row is NULL is determined as follows: obtain the number of fields in the current row; add 7 to the number of fields in the current row and then divide by 8; use the resulting integer as the byte length of the region used to identify whether each field in the current row is NULL.

[0009] Optionally, all areas preceding the string address storage area may also include: an area for identifying the version number of the encoding method of the encoded data or an area for identifying the version number of the data to be encoded.

[0010] According to a second aspect of this disclosure, an encoding apparatus is provided, comprising: an acquisition unit configured to acquire data to be encoded; a basic type storage unit configured to store each basic type field in the data to be encoded consecutively in a basic type storage area according to the order in which the data is to be encoded, wherein each basic type field is stored according to the actual byte length of its respective type; a string storage unit configured to, for each line in the data to be encoded, store the addresses of the string data of each string field in the current line consecutively in a string address storage area according to the order of the corresponding string fields in the current line, wherein the address of each string data is stored according to the actual byte length of its respective address, and the string address storage area is located after the basic type storage area; and a determination unit configured to use the re-stored data to be encoded as the encoded data.

[0011] Optionally, the string storage unit is further configured to store the string data of each string field in the current row consecutively in the string data storage area according to the order in the current row; and to store the addresses of the string data in the string data storage area consecutively in the string address storage area according to the order of the corresponding string fields in the current row, wherein the string address storage area is located after the basic type storage area and before the string data storage area, and the byte length of the string address storage area is fixed.

[0012] Optionally, the byte length of the string address storage area is determined as follows: the byte length of all areas preceding the string address storage area is determined; for each line of data to be encoded, the byte length of the string address storage area is determined by the byte length of all areas preceding the string address storage area, the number of fields in the current line, and the byte length of the string data of all string fields in the current line.

[0013] Optionally, all areas preceding the string address storage area include: a basic type storage area, an area for identifying whether each field in the current line is NULL, and an area for identifying the total byte length of the encoded current line.

[0014] Optionally, the byte length of the region used to identify whether each field in the current row is NULL is determined as follows: obtain the number of fields in the current row; add 7 to the number of fields in the current row and then divide by 8; use the resulting integer as the byte length of the region used to identify whether each field in the current row is NULL.

[0015] Optionally, all areas preceding the string address storage area may also include: an area for identifying the version number of the encoding method of the encoded data or an area for identifying the version number of the data to be encoded.

[0016] According to a third aspect of this disclosure, a computer-readable storage medium for storing instructions is provided, wherein when the instructions are executed by at least one computing device, they cause at least one computing device to perform the encoding method described above.

[0017] According to a fourth aspect of this disclosure, a system is provided that includes at least one computing device and at least one storage device for storing instructions, wherein the instructions, when executed by at least one computing device, cause at least one computing device to perform the encoding method described above.

[0018] According to the encoding method and apparatus of this exemplary embodiment, basic types and strings are stored separately, that is, basic type storage areas and string address storage areas are set up respectively to store the actual location information of basic type data and string data, which can reduce the complexity of subsequent decoding. In addition, each basic type field is stored according to the actual byte length of its own type and the address of each string data is also stored according to its actual byte length, which reduces memory usage. Attached Figure Description

[0019] These and / or other aspects and advantages of the invention will become clear and more readily understood from the following description of the embodiments, taken in conjunction with the accompanying drawings, wherein:

[0020] Figure 1 This diagram illustrates Spark encoding and decoding in related technologies.

[0021] Figure 2 A flowchart illustrating an encoding method according to an exemplary embodiment of the present disclosure;

[0022] Figure 3 This diagram illustrates an encoding format according to an exemplary embodiment of the present disclosure.

[0023] Figure 4 A block diagram of a data query apparatus according to an exemplary embodiment of the present disclosure is shown. Detailed Implementation

[0024] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the invention as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0025] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0026] The following will be explained by referring to the appendix. Figures 2 to 4 The embodiments of this disclosure are described in order to explain this disclosure.

[0027] Figure 2 A flowchart illustrating an encoding method according to an exemplary embodiment of this disclosure is shown. Figure 2 As shown, the encoding method includes the following steps:

[0028] In step S201, the data to be encoded is obtained. This data to be encoded can be as shown in Table 1 below:

[0029] Table 1 Data to be encoded

[0030] id name1 age amt name2 1111 abc 12 1.3 ming 1112 aee 13 3.5 tian

[0031] It should be noted that Table 1 is only one possible representation of the data to be encoded, and this disclosure is not limited to it.

[0032] In step S202, each basic type field in the data to be encoded is stored consecutively in the basic type storage area according to its order in the data to be encoded. Each basic type field is stored according to its actual byte length. The basic types mentioned above can include, but are not limited to, bool, int8, int16, int32, int64, float, and double. Specifically, bool / int8 occupies 1 byte, int16 occupies 2 bytes, int32 / float occupies 4 bytes, and int64 / double occupies 8 bytes. As shown in Table 1 above, id belongs to the basic type int64, age belongs to the basic type int32, and amt belongs to the basic type float.

[0033] Specifically, the basic type storage area stores the corresponding data in the order of id, age, and amt. The basic type storage area corresponding to the first row stores 1111, 12, and 1.3, where 1111 occupies 8 bytes, 12 occupies 4 bytes, and 1.3 occupies 4 bytes. The basic type storage area corresponding to the second row stores 1112, 13, and 3.5, where 1112 occupies 8 bytes, 13 occupies 4 bytes, and 3.5 occupies 4 bytes.

[0034] In step S203, for each row of the data to be encoded, the addresses of the string data of each string field in the current row are stored consecutively in the string address storage area according to the order of the corresponding string fields in the current row. The address of each string data is stored according to its actual byte length, and the string address storage area is located after the basic type storage area. As shown in Table 1 above, 'name' belongs to the string 'string' and is not a basic type.

[0035] Specifically, taking the first row of data as an example, assuming that the address where the first row of data under name1 is stored is XX, and the address where the first row of data under name2 is stored is YY, then the string address storage area stores XX and YY in the order of name1 and name2, and stores XX with the actual byte length of XX and stores YY with the actual byte length of YY.

[0036] According to an exemplary embodiment of this disclosure, storing the addresses of string data for each string field in the current row consecutively in the string address storage area according to the order of the corresponding string fields in the current row includes: storing the string data for each string field in the current row consecutively in the string data storage area according to the order of the current row; and storing the addresses of the string data in the string data storage area consecutively in the string address storage area according to the order of the corresponding string fields in the current row. The string address storage area is located after the basic type storage area and before the string data storage area, and the byte length of the string address storage area is fixed. According to this embodiment, placing the string data storage area after the string address storage area can reduce the complexity of decoding because the byte length of the string data storage area is not fixed, while the byte length of the string address storage area is fixed.

[0037] Specifically, string data can be stored after the string address storage area mentioned above; that is, the string data storage area is placed after the string address storage area. For example, after the string address storage area, abc and ming can be stored in the order of name1 and name2.

[0038] According to an exemplary embodiment of this disclosure, the byte length of the string address storage area is determined as follows: the byte lengths of all areas preceding the string address storage area are determined; for each line of data to be encoded, the byte length of the string address storage area is determined by the byte lengths of all areas preceding the string address storage area, the number of fields in the current line, and the byte lengths of the string data of all string fields in the current line. According to this embodiment, the byte length of the string address storage area can be determined conveniently and quickly, facilitating subsequent decoding.

[0039] Specifically, to facilitate subsequent decoding, the string address storage area can be set according to certain rules. For example, assuming there are N string fields, the total byte length of all strings is len1, and the total byte length of all areas before the string address storage area is size1, the byte length of the string address storage area can be obtained as follows:

[0040] If size1+(N*1)+len1<=2^8, then the string address storage area occupies 1 byte, that is, the string address storage area has a length of 1 byte.

[0041] If size1+(N*2)+len1<=2^16, then the string address storage area occupies 2 bytes, that is, the string address storage area has a length of 2 bytes.

[0042] If size1+(N*3)+len1<=2^24, then the string address storage area occupies 3 bytes, that is, the string address storage area has a length of 3 bytes.

[0043] If size1+(N*4)+len1<=2^32, then the string address storage area occupies 4 bytes, that is, the string address storage area has a length of 4 bytes.

[0044] Similarly, size1+(N*X)+len1<=2^(X×8), where X is the length of the string address storage area in bytes.

[0045] According to an exemplary embodiment of this disclosure, all areas preceding the string address storage area include: a basic type storage area, an area for identifying whether each field of the current line is NULL, and an area for identifying the total byte length of the encoded current line.

[0046] According to an exemplary embodiment of this disclosure, the byte length of the region used to identify whether each field in the current row is NULL is determined as follows: the number of fields in the current row is obtained; 7 is added to the number of fields in the current row, and then divided by 8; the resulting integer is used as the byte length of the region used to identify whether each field in the current row is NULL. In this embodiment, adjusting the byte length to (number of fields + 7) / 8, which is no longer a multiple of 8, reduces memory usage and overcomes the memory consumption problem in related technologies where the nullbitset field used to identify NULL is a multiple of 8 bytes.

[0047] Specifically, a bitmap can be used to indicate whether a field is NULL. As shown in Table 1, if there are 5 fields, the byte length of the area used to indicate whether each field in the current row is NULL can be calculated by (5+7) / 8, which is 1.5. Taking the integer part, we get 1, which means the byte length of the area used to indicate whether each field in the current row is NULL is 1.

[0048] According to an exemplary embodiment of this disclosure, all areas preceding the string address storage area further include: a region for identifying the version number of the encoding method of the encoded data or a region for identifying the version number of the data to be encoded. In this embodiment, setting these two regions facilitates subsequent decoding because different encoding methods can correspond to different version numbers. For example, after encoding with an encoding method different from that disclosed, the version number used to identify the encoding method of the encoded data can be changed. Conversely, if fields are deleted or added to the data to be encoded, the version number used to identify the data to be encoded is changed. Subsequent decoding can refer to the information in these two regions for rapid decoding.

[0049] In step S204, the data to be encoded after being re-stored is used as the encoded data.

[0050] To facilitate understanding of the above embodiments, the following is in conjunction with... Figure 3 Provide a systematic explanation. Figure 3 This diagram illustrates an encoding format according to an exemplary embodiment of the present disclosure, such as... Figure 3 As shown:

[0051] FVersion represents the data encoding format version number. The area it is located in is the same area used in the above embodiment to identify the version number of the encoding method of the encoded data. It can be set to occupy one byte. If the encoding method is modified, the version number can be incremented by 1. Different version numbers correspond to different encoding and decoding schemes.

[0052] SVersion represents the schema version number. The area it is located in is the area used to identify the version number of the data to be encoded in the above embodiment. It can be set to occupy one byte. If fields are added or deleted in the original data, the version number can be incremented by 1.

[0053] Size represents the total size of the data, and its area is the area used in the above embodiment to identify the total byte length of the current line after encoding, which can be set to occupy four bytes.

[0054] The BitMap indicates which field is NULL. The area it is located in is the area used in the above embodiment to identify whether each field in the current row is NULL. If the number of schema fields is N, then the length of this area is (N+7) / 8.

[0055] The basic type storage area stores fields of types bool, int8, int16, int32, int64, float, and double.

[0056] The string address storage area stores the actual location of the string, and stores it according to the actual byte length of the address information.

[0057] The string data storage area stores the actual content of the string.

[0058] According to an exemplary embodiment of this disclosure, after receiving encoded data, the receiving end needs to decode the encoded data. First, it reads the FVersion field to determine the encoding method used by the encoded data. After determining the encoding method, it decodes the encoded data according to the format corresponding to that encoding method. For example, after determining the format corresponding to the encoding method, such as... Figure 3As shown, at this point, the BitMap field can be read to parse out which fields of the original data before encoding are NULL. Then, the data corresponding to the basic type fields can be parsed out from the basic type storage area. Next, the string address storing the string data can be parsed out from the string address storage area, and the corresponding string data can be retrieved from the corresponding string address. It should be noted that this disclosure does not limit the decoding order.

[0059] Figure 4 A block diagram of a data query apparatus according to an exemplary embodiment of the present disclosure is shown. Figure 4 As shown, the data query device includes: an acquisition unit 40, a basic type storage unit 42, a string storage unit 44, and a determination unit 46.

[0060] The acquisition unit 40 is configured to acquire the data to be encoded; the basic type storage unit 42 is configured to store each basic type field in the data to be encoded consecutively in the basic type storage area according to the order in which the data is to be encoded, wherein each basic type field is stored according to the actual byte length of its respective type; the string storage unit 44 is configured to, for each row in the data to be encoded, store the address of the string data of each string field in the current row consecutively in the string address storage area according to the order of the corresponding string field in the current row, wherein the address of each string data is stored according to the actual byte length of its respective address, and the string address storage area is located after the basic type storage area; the determination unit 46 is configured to use the re-stored data to be encoded as the encoded data.

[0061] According to an exemplary embodiment of this disclosure, the string storage unit 44 is further configured to store the string data of each string field of the current row consecutively in the string data storage area in the order of the current row; and to store the addresses of the string data in the string data storage area consecutively in the string address storage area in the order of the corresponding string fields in the current row, wherein the string address storage area is located after the basic type storage area and before the string data storage area, and the byte length of the string address storage area is fixed.

[0062] According to an exemplary embodiment of this disclosure, the byte length of the string address storage area is determined as follows: the byte length of all areas preceding the string address storage area is determined; for each line of data to be encoded, the byte length of the string address storage area is determined by the byte length of all areas preceding the string address storage area, the number of fields in the current line, and the byte length of the string data of all string fields in the current line.

[0063] According to an exemplary embodiment of this disclosure, all areas preceding the string address storage area include: a basic type storage area, an area for identifying whether each field of the current line is NULL, and an area for identifying the total byte length of the encoded current line.

[0064] According to an exemplary embodiment of this disclosure, the byte length of the region used to identify whether each field in the current row is NULL is determined as follows: the number of fields in the current row is obtained; 7 is added to the number of fields in the current row and then divided by 8; the resulting integer is used as the byte length of the region used to identify whether each field in the current row is NULL.

[0065] According to an exemplary embodiment of this disclosure, all areas preceding the string address storage area further include: an area for identifying the version number of the encoding method of the encoded data or an area for identifying the version number of the data to be encoded.

[0066] The above has been referred to Figures 1 to 4 An encoding method and apparatus according to exemplary embodiments of the present disclosure are described.

[0067] Figure 4 The individual units in the illustrated data query device can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, pure software code, or a module combining software and hardware. Furthermore, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, client, or server).

[0068] In addition, refer to Figure 1 The described encoding method can be implemented by a program (or instructions) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform the encoding method according to the present disclosure.

[0069] The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, and servers. It should be noted that the computer program can also be used to perform additional steps beyond those described above, or to perform more specific processing while performing the above steps. The details of these additional steps and further processing are already described in the reference... Figure 1 The relevant methods were mentioned in the description of the process, so they will not be repeated here to avoid repetition.

[0070] It should be noted that each unit in the data query apparatus according to the exemplary embodiments of this disclosure can rely entirely on the operation of a computer program to realize its corresponding function. That is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., a lib library) to realize its corresponding function.

[0071] on the other hand, Figure 4 The units shown can also be implemented using hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segment used to perform the corresponding operation can be stored in a computer-readable medium such as a storage medium, so that the processor can perform the corresponding operation by reading and running the corresponding program code or code segment.

[0072] For example, exemplary embodiments of the present disclosure can also be implemented as a computing device including a storage component and a processor, wherein the storage component stores a set of computer-executable instructions, and when the set of computer-executable instructions is executed by the processor, an encoding method according to exemplary embodiments of the present disclosure is executed.

[0073] Specifically, the computing device can be deployed on a server or client, or on node devices in a distributed network environment. Furthermore, the computing device can be a PC, tablet, personal digital assistant, smartphone, web application, or other device capable of executing the aforementioned set of instructions.

[0074] Here, the computing device is not necessarily a single computing device, but can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The computing device can also be part of an integrated control system or system manager, or can be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0075] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0076] Some operations described in the encoding method according to exemplary embodiments of this disclosure can be implemented in software, some operations can be implemented in hardware, and some operations can be implemented in a combination of software and hardware.

[0077] The processor can execute instructions or code stored in one of the storage components, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0078] Storage components can be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, storage components can include separate devices, such as external disk drives, storage arrays, or other storage devices that can be used by any database system. Storage components and the processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the storage component.

[0079] In addition, the computing device may include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be interconnected via a bus and / or network.

[0080] The encoding method according to exemplary embodiments of this disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logic device or operate according to non-precise boundaries.

[0081] Therefore, refer to Figure 1 The described encoding method can be implemented by a system comprising at least one computing device and at least one storage device containing at least one storage instruction.

[0082] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for encoding a method according to an exemplary embodiment of the present disclosure, and a storage device stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by at least one computing device, a reference is executed. Figure 1 The encoding method described.

[0083] The foregoing has described various exemplary embodiments of this disclosure. It should be understood that the foregoing description is exemplary only and not exhaustive, and this disclosure is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An encoding method, wherein, The method includes: Obtain the data to be encoded; Each basic type field in the data to be encoded is stored consecutively in the basic type storage area according to the order of the data to be encoded, wherein each basic type field is stored according to the actual byte length of its respective type; For each line of the data to be encoded, the addresses of the string data of each string field in the current line are stored consecutively in the string address storage area according to the order of the corresponding string fields in the current line. The address of each string data is stored according to the actual byte length of its address. The string address storage area is located after the basic type storage area. The data to be encoded is stored again and used as the encoded data. The step of storing the addresses of the string data of each string field in the current row consecutively in the string address storage area according to the order of the corresponding string fields in the current row includes: Store the string data of each string field in the current row consecutively in the string data storage area, in the order they appear in the current row. The string data is stored consecutively in the string address storage area according to the order of the corresponding string fields in the current row. The string address storage area is located after the basic type storage area and before the string data storage area. The byte length of the string address storage area is fixed.

2. The encoding method according to claim 1, wherein, The byte length of the string address storage area is determined in the following way: Determine the byte length of all regions preceding the string address storage region; For each line of the data to be encoded, the byte length of the string address storage area is determined by the byte length of all areas preceding the string address storage area, the number of fields in the current line, and the byte length of the string data of all string fields in the current line.

3. The encoding method according to claim 2, wherein, All areas preceding the string address storage area include: the basic type storage area, an area for identifying whether each field of the current line is NULL, and an area for identifying the total byte length of the encoded current line.

4. The encoding method according to claim 3, wherein, The byte length of the region used to identify whether each field in the current row is NULL is determined in the following way: Get the number of fields in the current row; Add 7 to the number of fields in the current row and then divide by 8; The resulting integer is used as the byte length of the region used to identify whether each field in the current row is NULL.

5. The encoding method according to claim 3, wherein, All areas preceding the string address storage area also include: an area for identifying the version number of the encoding method of the encoded data or an area for identifying the version number of the data to be encoded.

6. An encoding device, wherein, The encoding device includes: The acquisition unit is configured to acquire data to be encoded. The basic type storage unit is configured to store each basic type field in the data to be encoded consecutively in the basic type storage area according to the order of the data to be encoded, wherein each basic type field is stored according to the actual byte length of its respective type; The string storage unit is configured to store the addresses of the string data of each string field in the current row consecutively in the string address storage area for each row of the data to be encoded, according to the order of the corresponding string fields in the current row. The address of each string data is stored according to the actual byte length of its respective address. The string address storage area is located after the basic type storage area. The determining unit is configured to use the re-stored data to be encoded as the encoded data; The string storage unit is further configured to store the string data of each string field of the current row consecutively in the string data storage area according to the order of the current row; and to store the addresses of the string data in the string data storage area consecutively in the string address storage area according to the order of the corresponding string fields in the current row, wherein the string address storage area is located after the basic type storage area and before the string data storage area, and the byte length of the string address storage area is fixed.

7. The encoding device according to claim 6, wherein, The byte length of the string address storage area is determined as follows: the byte length of all areas preceding the string address storage area is determined; for each line of the data to be encoded, the byte length of the string address storage area is determined by the byte length of all areas preceding the string address storage area, the number of fields in the current line, and the byte length of the string data of all string fields in the current line.

8. The encoding device according to claim 7, wherein, All areas preceding the string address storage area include: the basic type storage area, an area for identifying whether each field of the current line is NULL, and an area for identifying the total byte length of the encoded current line.

9. The encoding device according to claim 8, wherein, The byte length of the region used to identify whether each field in the current row is NULL is determined as follows: obtain the number of fields in the current row; add 7 to the number of fields in the current row and then divide by 8; use the resulting integer as the byte length of the region used to identify whether each field in the current row is NULL.

10. The encoding device according to claim 8, wherein, All areas preceding the string address storage area also include: an area for identifying the version number of the encoding method of the encoded data or an area for identifying the version number of the data to be encoded.

11. A computer-readable storage medium for storing instructions, wherein, When the instruction is executed by at least one computing device, it causes the at least one computing device to perform the encoding method as described in any one of claims 1 to 5.

12. A system comprising at least one computing device and at least one storage device for storing instructions, wherein, When the instruction is executed by the at least one computing device, it causes the at least one computing device to perform the encoding method as described in any one of claims 1 to 5.

Images