A data processing method, electronic device and storage medium for a database to be processed

Abstract

The present invention relates to the technical field of data storage, and in particular to a data processing method, electronic device and storage medium for a database to be processed, the method comprising: obtaining the programming language type of the database to be processed, the field name of the database to be processed, the type corresponding to the field name, and the number of records, and calculating the priority of the database to be processed based on the above information obtained, and taking corresponding processing measures for the data in the database to be processed according to the priority; it can be seen that the present invention can reasonably determine whether to perform subsequent encoding and compression processing based on the obtained priority, and is suitable for large-scale data storage through compression processing.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a data processing method, electronic equipment and storage medium for a database to be processed. Background Art

[0002] At present, when processing and storing data, the data source sent will be read first, such as a structured data table. In order to prevent garbled characters from appearing when storing large amounts of data, mainstream storage protocols must also store data types in the form of Tag-Length-Value, that is, TLV form. Tag corresponds to the field and type information of the data, Length is the length of the data content, and Value is the specific data content. Therefore, large-scale data sources will take up more storage space when stored. Due to the inconsistency of the number of bytes in Chinese and English, the different programming languages ​​used in different data tables, as well as the proportion of Chinese and English in the data tables, will affect the size of the number of bytes. It is not comprehensive to only consider the size of the bytes to determine whether to compress and store the data source. Therefore, how to determine the storage priority of the data source to determine whether to compress the data source is an urgent problem to be solved. Summary of the invention

[0003] In view of the above technical problems, the technical solution adopted by the present invention is:

[0004] A method for processing data in a database to be processed, the method comprising the following steps:

[0005] S100, acquiring attribute information of a database to be processed; the attribute information of the database to be processed includes: a programming language type of the database to be processed, a field name of the database to be processed, a type corresponding to the field name, and a number of records.

[0006] S200: Determine the priority F of the database to be processed according to the attribute information of the database to be processed.

[0007] In S200, the steps further include the following steps:

[0008] S201, acquiring the number of Chinese characters and the number of English characters in the field name of the database to be processed from the attribute information of the database to be processed.

[0009] S202: Determine the size of each Chinese character and each English character in the field name of the database to be processed based on the programming language type of the database to be processed in the attribute information of the database to be processed.

[0010] S203: Determine the record size under each field name based on the type corresponding to the field name in the attribute information of the database to be processed.

[0011] S204, determining the priority F of the database to be processed according to the number of Chinese characters and English characters in the field name, the size of each Chinese character and the size of each English character, and the size of the records under each field name.

[0012] Among them, the priority F of the database to be processed meets the following conditions:

[0013] F=W 1 ×F 1 ×d 1 +W 2 ×F 2 ×d 2 +∑ h g=1 W 3g ×F 3g , where h is the number of field names in the database to be processed, W 1 is the weight of Chinese characters in the field name, W 2 is the weight of English characters in the field name, W 3g is the record weight under the gth field name of the database to be processed, F 1 is the number of Chinese characters in the field names of the database to be processed, F 2 F is the number of English characters in the field name of the database to be processed. 3g refers to the record size under the gth field name, d 1 is the size of each Chinese character in the field name of the database to be processed, d 2 It is the size of each English character in the field name of the database to be processed.

[0014] S300: taking corresponding processing measures for the data in the database to be processed according to the priority F of the database to be processed.

[0015] The present invention also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored. The at least one instruction or the at least one program is loaded and executed by a processor to implement the above-mentioned data processing method of the database to be processed.

[0016] The present invention also provides an electronic device, comprising a processor and the above-mentioned non-transitory computer-readable storage medium.

[0017] Compared with the prior art, the present invention has obvious beneficial effects. By means of the above technical solution, the data processing method of a database to be processed provided by the present invention can achieve considerable technical advancement and practicality, and has wide industrial utilization value, and has at least the following beneficial effects:

[0018] The present invention provides a data processing method for a database to be processed. First, the programming language type of the database to be processed, the field name of the database to be processed, the type corresponding to the field name, and the number of records are obtained, and the priority of the database to be processed is calculated based on the above information obtained. According to the priority, corresponding processing measures are taken for the data in the database to be processed. When calculating the priority, it is considered that the number of bytes occupied by Chinese and English in the field is inconsistent, and the number of bytes occupied by the same Chinese or the same English in different programming language types may also be inconsistent. Therefore, different programming language types and the number of Chinese and English are introduced in the calculation to obtain a reasonable storage priority. That is, for multiple databases to be processed of different programming language types, there are correspondingly the same priority judgment criteria. According to the obtained priority, it can be reasonably determined whether to perform subsequent encoding and compression processing. Through compression processing, it is suitable for large-scale data storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a data processing method for a database to be processed provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0023] This embodiment provides a method for processing data in a database to be processed, the method comprising the following steps: Figure 1 As shown:

[0024] S100, acquiring attribute information of a database to be processed; the attribute information of the database to be processed includes: a programming language type of the database to be processed, a field name of the database to be processed, a type corresponding to the field name, and a number of records; for example, the type corresponding to the field name is an integer type or a floating point type.

[0025] Specifically, the database to be processed refers to a received incremental database, wherein the incremental database is a structured database, for example, the incremental database is a table whose records are not null.

[0026] S200: Determine the priority F of the database to be processed according to the attribute information of the database to be processed.

[0027] Specifically, step S200 also includes the following steps:

[0028] S201, obtaining the number of Chinese characters and the number of English characters in the field names of the database to be processed from the attribute information of the database to be processed; which can be understood as: the sum of the number of Chinese characters in each field name in the database to be processed and the sum of the number of English characters in each field name.

[0029] S202, based on the programming language type of the database to be processed in the attribute information of the database to be processed, determine the size of each Chinese character and the size of each English character in the field name of the database to be processed; it can be understood that: based on the programming language type of the database to be processed and each character type in the field name, it can be determined that each Chinese character occupies a byte and each English character occupies a byte.

[0030] S203: Determine the record size under each field name based on the type corresponding to the field name in the attribute information of the database to be processed.

[0031] S204, determining the priority F of the database to be processed according to the number of Chinese characters and English characters in the field name, the size of each Chinese character and the size of each English character, and the size of the records under each field name.

[0032] Preferably, in step S204, the priority F of the database to be processed meets the following conditions:

[0033] F=W 1 ×F 1 ×d 1 +W 2 ×F2 ×d 2 +∑ h g=1 W 3g ×F 3g , where h is the number of field names in the database to be processed, W 1 is the weight of Chinese characters in the field name, W 2 is the weight of English characters in the field name, W 3g is the record weight under the gth field name of the database to be processed, F 1 is the number of Chinese characters in the field names of the database to be processed, F 2 F is the number of English characters in the field name of the database to be processed. 3g refers to the record size under the gth field name, d 1 is the size of each Chinese character in the field name of the database to be processed, d 2 It is the size of each English character in the field name of the database to be processed; it can be understood as: F 1 With F 2 The sum is the total number of characters in the field names of the database to be processed.

[0034] Furthermore, W 1 Meet the following conditions:

[0035] W 1 =(d 1 / D 1 ) / ((d 1 / D 1 )+(d 2 / D 2 )+(∑ h g=1 F 3g / D 3g )), D 1 Refers to the maximum size of Chinese characters in different programming language types, D 2 Refers to the maximum size of English characters in different programming languages. 3g Refers to the record size under the gth field name, D 3g It refers to the maximum value of the record size under the g-th field name under the type corresponding to different field values ​​in the g-th field name.

[0036] For ease of understanding, the following explanation is given: A Chinese character generally occupies 2-4 bytes. 1 The maximum value is determined by comparing the number of bytes of the Chinese character in different programming language types. Similarly, D 2The maximum value is determined after comparing the number of bytes of English characters in different programming language types; the data in different field values ​​in the g-th column may have different types, for example, there are z types. All records under the g-th field name are classified according to any of the same types, and the record size under the g-th field name is obtained. A total of z record sizes are obtained, and the largest record size is taken as D 3g .

[0037] Furthermore, W 2 Meet the following conditions:

[0038] W 2 =(d 2 / D 2 ) / ((d 1 / D 1 )+(d 2 / D 2 )+(∑ h g=1 F 3g / D 3g )).

[0039] Furthermore, W 3g Meet the following conditions:

[0040] W 3g =(d 3g / D 3g ) / ((d 1 / D 1 )+(d 2 / D 2 )+(∑ h g=1 F 3g / D 3g )).

[0041] As mentioned above, considering that the number of bytes occupied by Chinese and English in the field is inconsistent in different programming language types, it is necessary to calculate the number of bytes of Chinese and English separately, and introduce different programming language types in the calculation, so as to obtain a reasonable storage priority, that is, for multiple to-be-processed databases of different programming language types, there are corresponding same priority judgment criteria, and whether to perform subsequent encoding and compression processing is determined according to the obtained priority.

[0042] S300: taking corresponding processing measures for the data in the database to be processed according to the priority F of the database to be processed.

[0043] Specifically, step S300 also includes the following steps:

[0044] S301, when F>λ, obtain the to-be-stored data information set A corresponding to the to-be-processed database = {A1 , A 2 , ..., A i , ..., A m} and the data sending end information corresponding to A, where the i-th data information to be stored A i =(A i1 , A i2 , A i3 ), A i1 Refers to the field and data type to be stored corresponding to the i-th data to be stored. i2 is the data length corresponding to the i-th data to be stored, A i3 is the field value to be stored corresponding to the i-th data to be stored, i=1, 2, ..., m, m is the number of data information to be stored, wherein λ is the preset priority threshold; for example, the data types include numeric type int, character type string, etc.

[0045] In another specific embodiment, when F≤λ, the database to be processed is stored in a preset database.

[0046] Specifically, the data sending end information includes the name of the data sending platform and the corresponding programming language, for example, Java, Python, and different platforms have corresponding different data type representations.

[0047] As mentioned above, since different data sending ends may correspond to different programming languages, it is necessary to set a corresponding preset field mapping relationship set for each data sending end, which can be applicable to data storage of different data sending platforms, and has high flexibility in cross-platform data storage. The binary code number of the field mapping value is smaller than the binary code number corresponding to the corresponding preset field. After conversion according to the preset field mapping relationship set, the number of coded characters can be reduced, thereby reducing the storage space occupied.

[0048] In a specific embodiment, λ is obtained by the following steps:

[0049] S3011, obtaining the original byte sizes of several historical records.

[0050] S3012, obtaining the target byte size corresponding to each historical record after it is stored in TLV format.

[0051] S3013, obtaining the ratio of the original byte size corresponding to each historical record to the target byte size, and fitting a number of ratios corresponding to a number of historical records, and taking the original byte size of the historical record corresponding to the maximum slope in the obtained fitting curve as λ.

[0052] S302, according to the data sending end information corresponding to A, determine the target field mapping relationship set corresponding to A from a plurality of preset field mapping relationship sets; it can be understood that: each preset field mapping relationship set corresponds to a data sending end information.

[0053] Specifically, the preset field mapping relationship set includes several data types, several preset fields, the binary code corresponding to each data type and the field mapping value corresponding to each preset field; for example, the binary code corresponding to int is defined as 0001, and the binary code corresponding to string is defined as 0010. In different preset field mapping relationship sets, the same field can be defined as different field mapping values. Different preset field mapping relationship sets are decoupled, and the corresponding relationships do not affect each other.

[0054] For a better understanding, the following example is used for explanation: when the preset fields are id, name, and number, the corresponding field mapping values ​​are 1, 2, and 3, that is, string id=1, string name=2, int number=3.

[0055] Specifically, the binary code number of the field mapping value corresponding to the preset field is smaller than the binary code number corresponding to the preset field.

[0056] As mentioned above, when the target number of bytes is small, the storage space occupied is small, and there is no need to encode and compress it. It can be directly stored in the preset database, which can reduce the occupation of computing resources. When the target number of bytes is large, the corresponding target field mapping relationship set is found to convert the field to be stored into a field mapping value that saves more storage space, thereby reducing the storage space occupied by the target storage data information set.

[0057] S303: A is mapped based on the target field mapping relationship set. i1 Encode and convert A i1 The encoding result is the same as A i2 The encoding result and A i3 The encoding results are merged to obtain A i The corresponding first encoding result A 0 i .

[0058] In a specific embodiment, step S303 also includes the following steps:

[0059] S3031, searching A from the target field mapping relationship set i1 The corresponding field mapping value.

[0060] S3033, when A is found in the target field mapping relationship set,i1 When mapping values ​​to the corresponding fields, A i1 The corresponding field mapping value is mapped in binary to obtain A i1 The corresponding encoding result.

[0061] S3035, for A i2 and A i3 Perform binary mapping respectively and obtain A i2 The encoding result and A i3 The encoding result.

[0062] S3037, A i1 The corresponding encoding result, A i2 The encoding result and A i3 The encoding results are merged in sequence to obtain A 0 i .

[0063] In order to better understand the above encoding process, the following example is used to illustrate: i1 The field to be stored in is number, the data type is int, i3 When the value is 10, int number=3 is set in the target field mapping relationship set. According to the target field mapping relationship set, the binary code of int is 0001, and number is mapped to 3, so the binary code is 0011. The data length is 2, and the corresponding binary code is 0010. i3 The corresponding binary code is 1010, so A i The corresponding first encoding result is 00010011 0010 1010.

[0064] For data to be stored without Chinese characters, such as string name=2, with the value "bob", the English character bob can be directly converted into binary format according to the ASCII code. The binary code of string is 0010, name is mapped to 2, the binary code is 0010, the data length is 3, the binary bit is 0011, the ASCII code of character b is 98, and the ASCII code of character o is 111. Therefore, the binary code corresponding to the string bob is 011000100110111101100010, so the first encoding result is 00100010 0011 011000100110111101100010. If there are Chinese characters, the UTF-8 encoding in the prior art can be used to encode the Chinese first and then convert it into binary, which will not be repeated here.

[0065] As mentioned above, after replacing the field to be stored with its corresponding field mapping value, the field mapping value is binary mapped so that the number of characters of the obtained binary code is smaller than the number of characters obtained by directly binary encoding the field to be stored. Since the target storage data information set contains a large number of fields to be stored, the above conversion can greatly reduce the storage space occupied.

[0066] In a specific embodiment, step S303 also includes the following steps:

[0067] S3032: When A is not found in the target field mapping relationship set, i1 When the corresponding field mapping value is sent to the target user i1 and fields to add hints.

[0068] S3034, receiving the target user based on A i1 A field with added prompt feedback i1 The corresponding fields to be stored and A i1 The corresponding field mapping value.

[0069] S3036, A i1 The corresponding fields to be stored and A i1 The corresponding field mapping value is added to the target field mapping relationship set.

[0070] As described above, when the required field mapping value is not found, by adding the newly added field to be stored and the corresponding field mapping value to the target field mapping relationship set, the target field mapping relationship set can be expanded to cover more comprehensive fields to be stored, so that the required field mapping value can be obtained when searching again, thereby realizing the conversion of the field to be stored.

[0071] S304, when A 0 i When the corresponding number of characters is greater than the preset character number threshold, A 0 i Compress according to the preset compression rules and obtain A i The corresponding second encoding result.

[0072] In a specific embodiment, a compression protocol such as gzip or snappy is used to compress A. 0 i Compression; those skilled in the art are aware of the specific implementation methods of gzip and snappy for compressing data, and different compression protocols can be selected according to different application scenarios, which will not be repeated here.

[0073] In another specific embodiment, step S304 further includes the following steps:

[0074] S3041, when A 0 i When the corresponding number of characters is greater than the preset character number threshold, 0 i Get the middle character list set B i ={B i1 , B i2 , ..., B ij , ..., B in}, B ij ={B ij1 , B ij2 , ..., B ije , ..., B ijf}, where B ije A 0 i The jth middle character in list B ij The e-th middle character in the string, j=1, 2, ..., n, where n is A 0 i The number of intermediate character lists in A, e=1, 2, ..., f, f is 0 i The jth middle character in list B ij The number of middle characters in any middle character list is the same as the character in A 0 i Consecutive characters in .

[0075] In another specific embodiment, when A 0 i When the corresponding number of characters is not greater than the preset character number threshold, A 0 i The target field mapping relationship set is associated and stored in a preset database.

[0076] S3042, when f≥η, B ij The f intermediate characters in are compressed and converted into B ij1 f, where n is the preset threshold value of the number of identical characters.

[0077] In order to better understand the above process, the following example is used to illustrate: A 0 i When it is 0001001100011110, n=7, which is divided into 000, 1, 00, 11, 000, 1111, 0, such as B i5 ={0, 0, 0}, then B i51 =0, f=3, when η is set to 3, B i5 The corresponding 000 is compressed and converted to 03.

[0078] S3043, Bi B ij Replace with B ij1 f and B i The middle characters in the formula are recombined in the corresponding order to obtain A. i The corresponding second encoding result is A i The corresponding second encoding result adds a compression mark; it can be understood as: i Replace each qualified middle character list in A 0 i Compressed to 031001103140, from 16 characters to 12 characters.

[0079] As mentioned above, considering that when compressing and converting a small number of identical consecutive characters, computing resources will be occupied and the storage space will not be significantly reduced, compression can be omitted. When the number of characters in the first encoding result is large, more storage space will be occupied. By compressing and converting the identical consecutive characters in the first encoding result with a larger number of characters into a second encoding result, more storage space can be released, so that the preset database can store more data.

[0080] S305, associating and storing the plurality of second encoding results corresponding to A and the target field mapping relationship set in a preset database to complete the data storage of the database to be processed.

[0081] In a specific embodiment, after step S305, the following steps are further included:

[0082] S1, when receiving a data query request corresponding to a database to be processed sent by a data query platform, obtain several first encoding results corresponding to A, a second encoding result carrying a compression identifier, and the target field mapping relationship set from a preset database.

[0083] S2, sending the several first encoding results corresponding to A, the second encoding result carrying the compression identifier and the target field mapping relationship set to the data query platform, so that the data query platform decodes the several first encoding results and the second encoding results corresponding to A according to the target field mapping relationship set and the compression identifier and obtains the database to be processed.

[0084] As described above, the data query platform first decompresses the second encoding result according to the compression identifier to obtain the corresponding first encoding result, and then obtains the original data to be stored according to the correspondence between the field to be stored and the field mapping value in the target field mapping relationship set. The decoded data to be stored can also be verified by the data length to ensure the accuracy of the decoded data.

[0085] This embodiment also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored. The at least one instruction or the at least one program is loaded and executed by a processor to implement the above-mentioned data processing method of the database to be processed.

[0086] This embodiment also provides an electronic device, including a processor and the above-mentioned non-transitory computer-readable storage medium.

[0087] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A data processing method for a database to be processed, characterized in that: The method comprises the following steps: S100, acquiring attribute information of a database to be processed; the attribute information of the database to be processed includes: a programming language type of the database to be processed, a field name of the database to be processed, a type corresponding to the field name, and a number of records; S200, determining the priority F of the database to be processed according to the attribute information of the database to be processed; The step S200 also includes the following steps: S201, acquiring the number of Chinese characters and the number of English characters in the field name of the database to be processed from the attribute information of the database to be processed; S202, determining the size of each Chinese character and each English character in the field name of the database to be processed based on the programming language type of the database to be processed in the attribute information of the database to be processed; S203, determining the record size under each field name based on the type corresponding to the field name in the attribute information of the database to be processed; S204, determining the priority F of the database to be processed according to the number of Chinese characters and English characters in the field name, the size of each Chinese character and the size of each English character, and the size of the records under each field name; Among them, the priority F of the database to be processed meets the following conditions: F=W1×F1×d1+W2×F2×d2+∑ h g=1 W 3g ×F 3g , where h is the number of field names in the database to be processed, W1 is the weight of Chinese characters in the field name, W2 is the weight of English characters in the field name, and W 3g is the record weight under the gth field name of the database to be processed, F1 is the number of Chinese characters in the field name of the database to be processed, F2 is the number of English characters in the field name of the database to be processed, and F 3g It refers to the size of the record under the gth field name, d1 is the size of each Chinese character in the field name of the database to be processed, and d2 is the size of each English character in the field name of the database to be processed; S300: taking corresponding processing measures for the data in the database to be processed according to the priority F of the database to be processed.

2. The data processing method of the database to be processed according to claim 1 is characterized in that: W1 meets the following conditions: W1=(d1 / D1) / ((d1 / D1)+(d2 / D2)+(∑ h g=1 F 3g / D 3g ), D1 refers to the maximum size of Chinese characters in different programming language types, D2 refers to the maximum size of English characters in different programming language types, F 3g Refers to the record size under the gth field name, D 3g It refers to the maximum value of the record size under the g-th field name under the type corresponding to different field values ​​in the g-th field name.

3. The data processing method of the database to be processed according to claim 2 is characterized in that: W2 meets the following requirements: W2=(d2 / D2) / ((d1 / D1)+(d2 / D2)+(∑ h g=1 F 3g / D 3g ))。 4. The data processing method of the database to be processed according to claim 2 is characterized in that: W 3g Meet the following conditions: W 3g =(d 3g / D 3g ) / ((d1 / D1)+(d2 / D2)+(∑ h g=1 F 3g / D 3g ))。 5. The data processing method of the database to be processed according to claim 1 is characterized in that: Step S300 also includes the following steps: S301, when F>λ, obtain the to-be-stored data information set A={A1, A2, . . . , A i , ..., A m } and the data sending end information corresponding to A, where the i-th data information to be stored A i =(A i1 , A i2 , A i3 ), A i1 Refers to the field and data type to be stored corresponding to the i-th data to be stored. i2 is the data length corresponding to the i-th data to be stored, A i3 is the field value to be stored corresponding to the i-th data to be stored, i=1, 2, ..., m, m is the number of data information to be stored, where λ is the preset priority threshold; S302, according to the data sending end information corresponding to A, determine the target field mapping relationship set corresponding to A from a plurality of preset field mapping relationship sets; the preset field mapping relationship set includes a plurality of data types, a plurality of preset fields, a binary code corresponding to each data type, and a field mapping value corresponding to each preset field; wherein the binary code number of the field mapping value corresponding to the preset field is less than the binary code number corresponding to the preset field; S303: A is mapped based on the target field mapping relationship set. i1 Encode and convert A i1 The encoding result is the same as A i2 The encoding result and A i3 The encoding results are merged to obtain A i The corresponding first encoding result A 0 i ; S304, when A 0 i When the corresponding number of characters is greater than the preset character number threshold, A 0 i Compress according to the preset compression rules and obtain A i The corresponding second encoding result; S305, associating and storing the plurality of second encoding results corresponding to A and the target field mapping relationship set in a preset database to complete the data storage of the database to be processed.

6. The data processing method of the database to be processed according to claim 5 is characterized in that: Get λ by following the steps below: S3011, obtaining the original byte size of several historical records; S3012, obtaining the target byte size corresponding to each historical record after it is stored in the TLV format; S3013, obtaining the ratio of the original byte size corresponding to each historical record to the target byte size, and fitting a number of ratios corresponding to a number of historical records, and taking the original byte size of the historical record corresponding to the maximum slope in the obtained fitting curve as λ.

7. The method for processing data of a database to be processed according to claim 5, characterized in that: Step S303 includes the following steps: S3031, searching A from the target field mapping relationship set i1 The corresponding field mapping value; S3033, when A is found in the target field mapping relationship set, i1 When mapping values ​​to the corresponding fields, A i1 The corresponding field mapping value is mapped in binary to obtain A i1 The corresponding encoding result; S3035, for A i2 and A i3 Perform binary mapping respectively and obtain A i2 The encoding result and A i3 The encoding result of S3037, A i1 The corresponding encoding result, A i2 The encoding result and A i3 The encoding results are merged in sequence to obtain A 0 i .

8. The method for processing data of a database to be processed according to claim 5, characterized in that: Step S304 includes the following steps: S3041, when A 0 i When the corresponding number of characters is greater than the preset character number threshold, 0 i Get the middle character list set B i ={B i1 , B i2 , ..., B ij , ..., B in }, B ij ={B ij1 , B ij2 , ..., B ije , ..., B ijf }, where B ije A 0 i The jth middle character in list B ij The e-th middle character in the string, j=1, 2, ..., n, where n is A 0 i The number of intermediate character lists in A, e=1, 2, ..., f, f is 0 i The jth middle character in list B ij The number of middle characters in any middle character list is the same as the character in A 0 i Consecutive characters in ; S3042, when f≥η, B ij The f intermediate characters in are compressed and converted into B ij1 f, where n is the preset threshold value of the number of identical characters; S3043, B i B ij Replace with B ij1 f and B i The middle characters in the formula are recombined in the corresponding order to obtain A. i The corresponding second encoding result is A i A compression flag is added to the corresponding second encoding result.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the data processing method of the database to be processed as described in any one of claims 1-8.

10. An electronic device, characterized in that: The method comprises a processor and the non-transitory computer-readable storage medium of claim 9.

Images