Data storage method, device and electronic device based on distributed database

By using key values ​​and physical storage node number in a distributed database, the high load and low efficiency problems caused by data aggregation on one node are solved, and uniform distribution and efficient storage of data are achieved.

CN112818047BActive Publication Date: 2024-11-12CHINA TOWER CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110096014.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-25
Publication Date
2024-11-12
Estimated Expiration
2041-01-25

AI Technical Summary

Technical Problem

In the prior art, data with high access popularity is concentrated in a physical storage node in a distributed database, resulting in problems of high load and low access efficiency.

Method used

By obtaining the key value of the data to be stored and the number of physical storage nodes in the distributed database, random seeds are generated and modulo are determined to determine which physical storage node the data should be stored, thereby evenly distributing the storage data and avoiding the data being concentrated on one node.

Benefits of technology

It achieves the balanced access pressure of cluster nodes, improves access efficiency, ensures that data is evenly distributed on various physical storage nodes, and improves the efficiency of data storage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112818047B_ABST
    Figure CN112818047B_ABST
Patent Text Reader

Abstract

The present invention provides a data storage method, device and electronic device of a distributed database, the method comprising: obtaining a target data file to be stored in the distributed database, the target data file comprising data to be stored and a key value corresponding to the data to be stored; generating a node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database; storing the data to be stored in the physical storage node corresponding to the node number. The embodiment of the present invention can improve node access efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data storage method, device and electronic equipment based on a distributed database. Background Art

[0002] With the expansion of business and the improvement of management refinement, the amount of data has grown rapidly. In terms of data storage management, distributed databases such as HBase have been widely used.

[0003] When writing data, HBase usually uses monotonically increasing or time-series key values ​​to aggregate hotly accessed data in a physical storage node, resulting in a high load on the physical storage node. In this way, problems may arise during data analysis, statistics, and access when accessing a batch of data in parallel, such as data stored in the physical storage node, resulting in low access efficiency. Summary of the invention

[0004] The embodiments of the present invention provide a data storage method, device and electronic device based on a distributed database to solve the problem of low access efficiency caused by aggregating highly accessed data in one physical storage node in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a data storage method based on a distributed database, the method comprising:

[0006] Obtain a target data file to be stored in a distributed database, wherein the target data file includes data to be stored and a key value corresponding to the data to be stored;

[0007] Generate a node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database;

[0008] The data to be stored is stored in the physical storage node corresponding to the node number.

[0009] In a second aspect, an embodiment of the present invention further provides a data storage device based on a distributed database, the device comprising:

[0010] An acquisition module, used for acquiring a target data file to be stored in a distributed database, wherein the target data file includes the data to be stored and a key value corresponding to the data to be stored;

[0011] A generating module, used for generating a node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database;

[0012] The storage module includes storing the data to be stored in the physical storage node corresponding to the node number.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned data storage method based on a distributed database when executed by the processor.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned data storage method based on a distributed database are implemented.

[0015] In an embodiment of the present invention, a target data file to be stored in a distributed database is obtained, the target data file includes the data to be stored and the key value corresponding to the data to be stored; and based on the key value and the number of physical storage nodes of the distributed database, a node number corresponding to the data to be stored is generated; and the data to be stored is stored in the physical storage node corresponding to the node number. In this way, it is possible to avoid allocating the data to be stored in the target data file to one physical storage node, thereby balancing the access pressure of each physical storage node in the cluster, and further improving the node access efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 is a flow chart of a data storage method based on a distributed database provided by an embodiment of the present invention;

[0018] Figure 2 It is a flowchart of a specific embodiment of a data storage method based on a distributed database provided by an embodiment of the present invention;

[0019] Figure 3 It is a schematic diagram of storing multiple data to be stored in the target data file based on HBase;

[0020] Figure 4 is a structural diagram of a data storage device based on a distributed database provided by an embodiment of the present invention;

[0021] Figure 5 is a functional block diagram of a data storage device based on a distributed database provided by an embodiment of the present invention;

[0022] Figure 6It is a structural diagram of an electronic device provided by the implementation of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] First, a data storage method based on a distributed database provided in an embodiment of the present invention is described.

[0025] It should be noted that the data storage method based on a distributed database provided in an embodiment of the present invention can be applied to electronic devices to solve the problem in the prior art that a highly accessed data set in a distributed database such as HBase is stored in one physical storage node, resulting in high load on the physical storage node and low access efficiency, thereby achieving the purpose of improving access efficiency.

[0026] See also Figure 1 , Figure 1 is a flow chart of a data storage method based on a distributed database provided by an embodiment of the present invention, such as Figure 1 As shown, the following steps are included:

[0027] Step 101: Obtain a target data file to be stored in a distributed database, wherein the target data file includes data to be stored and key values ​​corresponding to the data to be stored.

[0028] In this step, the distributed database may be HBase, the target data file may be a data file to be processed, the file format may be any format, the file may include any amount of data to be stored, and the data to be stored may be any data, for example, the data to be stored may be mobile phone Internet log data, or user profile data.

[0029] The target data file may include at least one data to be stored and a key value corresponding to each data to be stored, and its storage form is a key value and data, namely KeyValue. Correspondingly, its file storage format is an HFILE format.

[0030] In a specific implementation, step 101 specifically includes:

[0031] Obtain data files to be stored in a distributed database;

[0032] The data file is subjected to format normalization processing to obtain the target data file in a target storage format, where the target storage format is a storage format corresponding to the distributed database.

[0033] Specifically, obtain the HBase interface file, which is the data file to be stored in HBase. Load the HBase interface file into HDFS, which is the Hadoop distributed file storage system; wherein, the interface file is the data file to be processed, and the following data file is described in detail by taking the file of mobile phone Internet log data as an example.

[0034] The format of the interface file is converted into the HFILE storage format to obtain the target data file after normalization, which is a file in the HFILE storage format. When performing the format conversion, the data of HBase is stored on HDFS, and the files on HDFS are stored in the HFILE format. Therefore, the data usually needs to be stored in the form of files. Generally speaking, for the source data that needs to be stored, structured data files are usually generated first, and then the files are converted to the HFILE format and loaded into HBase. Of course, it is also possible to directly write it in the HFILE format when the source data is obtained, and then load it into HBase. The specific method can be based on application requirements and usage scenarios.

[0035] Step 102: Generate a node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database.

[0036] In this step, a random seed can be generated according to each key value of the data to be stored, and each random seed is modulo the total number of physical storage nodes of HBase as a divisor to obtain a node number, which is the value after modulo the random seed. In this way, the node number can fall within the range of physical storage nodes.

[0037] The random seed can be understood as a random number. The step 102 specifically includes:

[0038] Generate a random number corresponding to the key value;

[0039] The random number is modulo the number of physical storage nodes of the distributed database to obtain the node number corresponding to the data to be stored.

[0040] The random seed may be generated according to the key value according to an existing or new random seed generation method, for example, the square mean method or the linear remainder method may be used. Taking the linear remainder method as an example, the linear remainder method is to set the random seed to X(0) and use an algorithm X(n+1)=(a*X(n)+b)mod c to generate X(n+1).

[0041] Among them, a is the coefficient, which can be confirmed according to the respective business; b is the supplementary random sequence number. If X(n) is already random serialized data, then the value of b can be 0; and c is the modulo data, which is generally calculated using the number of physical storage nodes. Usually, the parameter c is made very large, so that a pseudo-random number between 0 and c-1 can be generated.

[0042] In an optional implementation, the key value may be a mobile phone number, and generating a random number corresponding to the key value includes:

[0043] Determine a target distribution field that can evenly distribute the data to be stored, wherein the amount of the data to be stored associated with each value interval associated with the target distribution field is greater than a first preset value and less than a second preset value, and the first preset value is less than the second preset value;

[0044] Obtaining a target value associated with the target distribution field in the key value;

[0045] A random number corresponding to the key value is generated based on the target value.

[0046] Specifically, we can first determine a target distribution field that can evenly distribute the data to be stored. The target distribution field can be associated with multiple value intervals. Taking the field of the last two digits in a mobile phone number as an example of a target distribution field, it can be associated with an interval between 00 and 10 and an interval between 10 and 20, etc.

[0047] Any data to be stored in the target data file can be associated with a value range. For example, if a certain data to be stored is associated with a target value range, the target value associated with the target distribution field in the key value corresponding to the data to be stored is within the target value range.

[0048] The size of the first preset value and the second preset value can be determined according to the ratio of the amount of data to be stored in the target data file to the number of value intervals. For example, if the amount of data to be stored in the target data file is 100, and the number of value intervals is pre-set to 10, then the ratio is 10, that is, the amount of data to be stored evenly distributed to each value interval is 10, and the amount of data in the statistically balanced distribution fluctuates around 10. For example, the first preset value can be 6, and the second preset value can be 14.

[0049] In actual implementation, generating random data corresponding to the key value may include:

[0050] Determine a target distribution field that can evenly distribute the data to be stored;

[0051] Obtaining a target value associated with the target distribution field in the key value;

[0052] Generate a random number corresponding to the key value based on the target value;

[0053] Among them, when the key values ​​corresponding to the multiple data to be stored in the target data file are divided based on the target distribution field, the number of target values ​​associated with the target distribution field in the key value that are located in each preset interval range is greater than a first preset value and less than a second preset value, and the first preset value is less than the second preset value.

[0054] For example, in order to evenly distribute the data to be stored in the target data file to each physical storage node, a target distribution field that can evenly distribute the data to be stored can be determined, and the target distribution field can be all or part of the fields in the key value. Taking the mobile phone number as an example, the data volume of the data to be stored can be counted according to the fields of the last two digits in the mobile phone number to confirm the data volume in each value range.

[0055] First, roughly estimate the number of records in each value interval, that is, how many records are there in the value interval between 00 and 10, and how many records are there in the value interval between 10 and 20. If the amount of data in each value interval is uneven, you can adjust the preset value interval size or change the values ​​of other fields in the mobile phone number for statistics. Finally, a target distribution field that makes the data volume of the data to be stored evenly distributed will be determined from the mobile phone number as the field for generating random numbers for the key value. After obtaining the number of records in the value interval, create an HBase table according to the number of records in the value interval, and use the number of records in the value interval as the range of the physical storage nodes of the HBase table.

[0056] When the business data is obtained, the values ​​of the last two fields are intercepted from the mobile phone number. A random seed can be generated based on the values ​​of the last two fields. The random seed is modulo the number of physical storage nodes as the divisor to obtain a node number that can ensure balanced data distribution. The node number is used as the row key for writing table data to ensure that the data of the record can fall into the corresponding physical storage node. When all data is written according to this algorithm mechanism, it can be ensured that all data will be evenly stored on each physical storage node; in this way, the data in the target data file can be evenly distributed to each node for storage, balancing the access load of each node, spreading the access pressure of each node, and improving the node access efficiency.

[0057] Optionally, before generating the node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database, the method further includes:

[0058] Counting the amount of data to be stored corresponding to the target distribution field in the target data file;

[0059] Based on the data volume, the number of physical storage nodes of the distributed database is determined.

[0060] In this embodiment, after obtaining the number of records in the value interval, an HBase table is created according to the number of records in the value interval, and the number of records in the value interval is used as the range of the physical storage nodes of the HBase table. In this way, the number of physical storage nodes can be pre-allocated for the storage of each data according to the estimated data size and the node situation of the Hadoop general processing platform for big data. Afterwards, the key value such as the field of the mobile phone number in the target data file is extracted for the calculation basis of the data distribution, and the unique random seed is calculated by the mobile phone number, and then the modulus is taken in combination with the number of physical storage nodes in the streaming computing cluster, and the modulo value is used as the physical node number, so as to ensure that each data in the target data file is evenly distributed to the pre-allocated physical storage nodes.

[0061] Step 103: store the data to be stored in the physical storage node corresponding to the node number.

[0062] In this step, each data to be stored can be stored according to the generated node number, and the data to be stored can be stored in the physical storage node corresponding to the node number. In this way, the data to be stored corresponding to each key value in the target data file can be loaded into the physical storage node corresponding to the node number.

[0063] The following takes the storage of mobile phone Internet log data based on HBase as an example to explain in detail the entire process of the data storage method based on a distributed database provided in an embodiment of the present invention.

[0064] See also Figure 2 , Figure 2 FIG. 1 is a flowchart of a specific embodiment of a data storage method based on a distributed database provided in an embodiment of the present invention. Figure 2 As shown, the data storage method based on the distributed database of this specific embodiment includes the following process:

[0065] Obtaining HBase interface data, which may be mobile phone Internet access log data;

[0066] Data is loaded into HDFS. Specifically, the interface data files can be loaded into HDFS (Hadoop distributed file storage system) through JAVA program;

[0067] HFILE normalization: The application will format the interface data file on HDFS into HFILE, which is convenient for loading the data into HBase later. HFILE is the storage format of KeyValue data in HBase and belongs to the file in HDFS.

[0068] Get the mobile phone number in the interface data as the key value of data distribution;

[0069] Generate a random seed based on the mobile phone number; there are multiple algorithms for generating random seeds, such as the square median method and the linear remainder method; a key value can only generate a unique random seed through a random number algorithm;

[0070] Taking a modulo of the random seed; wherein, in the modulo algorithm, the divisor is defined as the total number of physical storage nodes;

[0071] The value modulo the random seed can be used as the node number of the physical storage node to which the data belongs. The node number is key information to guide the data flow.

[0072] The data is loaded into the corresponding physical storage node according to the node number of the physical storage node.

[0073] See also Figure 3 , Figure 3 This is a schematic diagram of storing multiple data to be stored in the target data file based on HBase, such as Figure 3 As shown, the following steps are included:

[0074] Obtain a target data file, wherein the target data file includes a plurality of data to be stored and a key value corresponding to each data to be stored, including KEY1, KEY2 and KEY3 respectively;

[0075] Get the key values ​​in the target data file, including KEY1, KEY2 and KEY3;

[0076] Get random seeds for KEY1, KEY2 and KEY3 respectively;

[0077] The random seed of each key value is modulo, and the divisor of the modulo is the total number of physical storage nodes, which can be 1024;

[0078] The modulo value is obtained, which is the node number of the physical storage node, which is 2, 132 and N respectively, and the value range of N is 1 to 1024;

[0079] The data to be stored is stored in the physical storage nodes corresponding to the node numbers.

[0080] The following is a description of a data storage device based on a distributed database provided in an embodiment of the present invention.

[0081] See also Figure 4 , Figure 4 1 is a structural diagram of a data storage device based on a distributed database provided by an embodiment of the present invention, which can implement the details of the data storage method based on a distributed database and achieve the same effect. Figure 4 As shown, the data storage device 400 based on the distributed database includes:

[0082] An acquisition module 401 is used to acquire a target data file to be stored in a distributed database, wherein the target data file includes data to be stored and a key value corresponding to the data to be stored;

[0083] A generating module 402, configured to generate a node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database;

[0084] The storage module 403 includes storing the data to be stored in the physical storage node corresponding to the node number.

[0085] Optionally, the generating module 402 includes:

[0086] A generating unit, used to generate a random number corresponding to the key value;

[0087] The modulo unit is used to take the number of physical storage nodes of the distributed database as a divisor, perform modulo operation on the random number, and obtain a node number corresponding to the data to be stored.

[0088] Optionally, the generating unit is specifically used to determine a target distribution field that can evenly distribute the data to be stored, the amount of data to be stored associated with each value interval associated with the target distribution field is greater than a first preset value and less than a second preset value, and the first preset value is less than the second preset value; obtain a target value associated with the target distribution field in the key value; and generate a random number corresponding to the key value based on the target value.

[0089] Optionally, the device further comprises:

[0090] A statistics module, used for counting the amount of data to be stored corresponding to the target distribution field in the target data file;

[0091] A determination module is used to determine the number of physical storage nodes of the distributed database based on the data volume.

[0092] Optionally, the acquisition module is specifically used to acquire a data file to be stored in a distributed database; perform format normalization processing on the data file to obtain the target data file in a target storage format, and the target storage format is a storage format corresponding to the distributed database.

[0093] The above-mentioned data storage device 400 based on a distributed database can implement each process implemented by the electronic device in the above-mentioned data storage method embodiment based on a distributed database, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0094] See also Figure 5 , Figure 5 is a functional block diagram of a data storage device based on a distributed database provided by an embodiment of the present invention, such as Figure 5 As shown, the data storage device based on the distributed database includes an acquisition layer, a processing layer and a functional layer. The specific description of each layer is as follows:

[0095] Acquisition layer: responsible for loading interface data into HDFS and performing HFILE normalization;

[0096] Processing layer: responsible for the definition of HBase table data blocks, the loading instructions of interface data, that is, which HBase data block the data falls into, and the interface loading HBase work;

[0097] At the same time, the number of physical storage nodes is pre-allocated for the storage of each data block based on the estimated data size and Hadoop node conditions. Key values ​​such as mobile phone numbers are extracted for calculation of data distribution. A unique random seed is calculated using the mobile phone number, and the random seed is modulo the number of physical storage nodes in the streaming computing cluster. The modulo value is used as the node number of the physical storage node.

[0098] Functional layer: responsible for external services such as HBase data processing and query.

[0099] See also Figure 6 , Figure 6 is a structural diagram of an electronic device provided by the present invention, such as Figure 6 The electronic device shown includes: a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor. The components in the electronic device are coupled together via a bus interface 603. When the computer program is executed by the processor 601, the following steps are implemented:

[0100] Obtain a target data file to be stored in a distributed database, wherein the target data file includes data to be stored and a key value corresponding to the data to be stored;

[0101] Generate a node number corresponding to the data to be stored based on the key value and the number of physical storage nodes of the distributed database;

[0102] The data to be stored is stored in the physical storage node corresponding to the node number.

[0103] Optionally, the processor 601 is specifically configured to:

[0104] Generate a random number corresponding to the key value;

[0105] The random number is modulo the number of physical storage nodes of the distributed database to obtain the node number corresponding to the data to be stored.

[0106] Optionally, the processor 601 is specifically configured to:

[0107] Determine a target distribution field that can evenly distribute the data to be stored, wherein the amount of the data to be stored associated with each value interval associated with the target distribution field is greater than a first preset value and less than a second preset value, and the first preset value is less than the second preset value;

[0108] Obtaining a target value associated with the target distribution field in the key value;

[0109] A random number corresponding to the key value is generated based on the target value.

[0110] Optionally, the processor 601 is further configured to:

[0111] Counting the amount of data to be stored corresponding to the target distribution field in the target data file;

[0112] Based on the data volume, the number of physical storage nodes of the distributed database is determined.

[0113] Optionally, the processor 601 is specifically configured to:

[0114] Obtain data files to be stored in a distributed database;

[0115] The data file is subjected to format normalization processing to obtain the target data file in a target storage format, where the target storage format is a storage format corresponding to the distributed database.

[0116] Preferably, an embodiment of the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the data storage method based on a distributed database of any of the above-mentioned method embodiments are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0117] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned data storage method based on a distributed database is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0118] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0120] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0122] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0124] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data storage method based on a distributed database, characterized in that: The method comprises: Obtain a target data file to be stored in a distributed database, wherein the target data file includes data to be stored and a key value corresponding to the data to be stored; Generate a random number corresponding to the key value; Taking the number of physical storage nodes of the distributed database as a divisor, performing modulo operation on the random number to obtain a node number corresponding to the data to be stored; Storing the data to be stored in the physical storage node corresponding to the node number; Generating a random number corresponding to the key value includes: Determine a target distribution field that can evenly distribute the data to be stored, wherein the amount of data to be stored associated with each value interval associated with the target distribution field is greater than a first preset value and less than a second preset value, and the first preset value is less than the second preset value; the size of the first preset value and the second preset value is determined according to the ratio of the amount of data to be stored and the number of value intervals in the target data file, and the target distribution field is a partial field in the key value; Obtaining a target value associated with the target distribution field in the key value; Generate a random number corresponding to the key value based on the target value; The determining of a target distribution field that can evenly distribute the data to be stored includes: Estimating the number of records in each value interval of the distribution field in the key value; If the data volumes in each value interval are uneven, adjusting the preset value interval size to determine a target distribution field from the key value that makes the data volume distribution of the data to be stored even; After obtaining the number of records in the value range that makes the data volume of the data to be stored distributed evenly, the method further includes: The distributed database is created according to the number of records in the value interval, and the number of records in the value interval is used as the range of physical storage nodes of the distributed database.

2. The method according to claim 1, characterized in that The method further comprises: Counting the amount of data to be stored corresponding to the target distribution field in the target data file; Based on the data volume, the number of physical storage nodes of the distributed database is determined.

3. The method according to claim 1, characterized in that The step of obtaining a target data file to be stored in a distributed database includes: Obtain data files to be stored in a distributed database; The data file is subjected to format normalization processing to obtain the target data file in a target storage format, where the target storage format is a storage format corresponding to the distributed database.

4. A data storage device based on a distributed database, characterized in that: The device comprises: An acquisition module, used for acquiring a target data file to be stored in a distributed database, wherein the target data file includes the data to be stored and a key value corresponding to the data to be stored; The generating module comprises: a generating unit, configured to generate a random number corresponding to the key value; A modulo unit, used to take the number of physical storage nodes of the distributed database as a divisor, perform modulo operation on the random number, and obtain a node number corresponding to the data to be stored; A storage module, comprising storing the data to be stored in a physical storage node corresponding to the node number; The generating unit is specifically used to determine a target distribution field that can evenly distribute the data to be stored, the amount of data to be stored associated with each value interval associated with the target distribution field is greater than a first preset value and less than a second preset value, and the first preset value is less than the second preset value; the size of the first preset value and the second preset value is determined according to the ratio of the amount of data to be stored and the number of value intervals in the target data file, and the target distribution field is a partial field in the key value; obtain a target value associated with the target distribution field in the key value; and generate a random number corresponding to the key value based on the target value; The generating unit is further used for: Estimating the number of records in each value interval of the distribution field in the key value; If the data volumes in each value interval are uneven, adjusting the preset value interval size to determine a target distribution field from the key value that makes the data volume distribution of the data to be stored even; After obtaining the number of records in the value interval that evenly distributes the amount of data to be stored, the distributed database is created according to the number of records in the value interval, and the number of records in the value interval is used as the range of physical storage nodes of the distributed database.

5. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data storage method based on a distributed database as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data storage method based on a distributed database as claimed in any one of claims 1 to 3 are implemented.