Data storage method and device, storage medium, electronic equipment and product

Abstract

The invention provides a data storage method and device, a storage medium, electronic equipment and a product. The method comprises the following steps: acquiring target data to be stored in a database cluster, and splitting the target data into a plurality of data fragments; wherein the database cluster is composed of a plurality of physical devices, and each physical device is used for providing a storage space; if the data read-write characteristic corresponding to the target data is read, allocating a storage space for each data fragment according to the data read-write characteristic, and if the data read-write characteristic corresponding to the target data is not read, allocating a storage space for each data fragment according to the load conditions of the plurality of physical devices; and storing each data fragment into the storage space allocated for the data fragment. According to the scheme, the storage space distributed for the data fragmentation is based on the data read-write characteristics, and the service storage requirements of users for different data are fully met.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a data storage method, device, storage medium, electronic equipment and product. BACKGROUND

[0002] In the data storage scenario, there is a real problem of adapting device resource utilization to storage demand. Currently, to realize physical isolation of different user data, a storage mode of binding physical devices with users is generally adopted, but this mode exposes double drawbacks in actual application:

[0003] On the one hand, the data volume of most users is small, and a single physical device only stores a small amount of data, resulting in long-term idling of the remaining space of the device and low hardware resource utilization; on the other hand, the fixed storage architecture is difficult to respond to the diversified business storage demand of different data of users. SUMMARY

[0004] Therefore, one or more embodiments of the present specification provide technical solutions as follows:

[0005] According to a first aspect of one or more embodiments of the present specification, a data storage method is provided, comprising:

[0006] obtaining target data to be stored to a database cluster, and splitting the target data into a plurality of data shards; wherein the database cluster is composed of a plurality of physical devices, and each physical device is configured to provide a storage space;

[0007] if the data read-write characteristics corresponding to the target data are read, then assigning a storage space to each data shard according to the data read-write characteristics, and if the data read-write characteristics corresponding to the target data are not read, then assigning a storage space to each data shard according to the load condition of the plurality of physical devices;

[0008] storing each data shard into the storage space assigned to the data shard.

[0009] According to a second aspect of one or more embodiments of the present specification, a data storage device is provided, comprising:

[0010] an obtaining unit configured to obtain target data to be stored to a database cluster, and split the target data into a plurality of data shards; wherein the database cluster is composed of a plurality of physical devices, and each physical device is configured to provide a storage space;

[0011] an allocation unit, configured to allocate storage space for each data shard according to a data read-write characteristic corresponding to the target data if the data read-write characteristic is read, or according to load conditions of the plurality of physical devices if the data read-write characteristic corresponding to the target data is not read;

[0012] a storage unit, configured to store each data shard into the storage space allocated for the data shard.

[0013] According to a third aspect of one or more embodiments of the present specification, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the steps of the above data storage method by running the executable instructions.

[0014] According to a fourth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, having stored thereon computer instructions which, when executed by a processor, implement the steps of the above data storage method.

[0015] According to a fifth aspect of one or more embodiments of the present specification, a computer program product is provided, comprising computer program / instructions which, when executed by a processor, implement the steps of the above data storage method.

[0016] As can be seen from the above embodiments, the present specification obtains target data to be stored in a database cluster, and splits the target data into a plurality of data shards; wherein the database cluster is composed of a plurality of physical devices, and each physical device is configured to provide a storage space; if the target data is provided with a corresponding data read-write characteristic, the storage space for each data shard is allocated according to the data read-write characteristic, and if the target data is not provided with a corresponding data read-write characteristic, the storage space for each data shard is allocated according to load conditions of the plurality of physical devices; and each data shard is stored into the storage space allocated for the data shard. The storage space allocated for the data shard is based on the data read-write characteristic, which fully meets the user's business storage requirements for different data.

[0017] In this method, the database cluster is composed of a plurality of physical devices, the target data is split into independent data shards, and each data shard is stored into the storage space of each physical device. Under this mechanism, the storage space provided by each physical device is a component of the cluster, so that a single physical device can simultaneously carry data shards of different users, significantly improving the resource utilization rate of the physical device, and the storage space allocated for the data shard is based on the data read-write characteristic, which fully meets the user's business storage requirements for different data. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1Figure 1 is a flow diagram of a data storage method according to an example embodiment;

[0019] Figure 2 Figure 2 is an architecture diagram of a database cluster according to an example embodiment;

[0020] Figure 3 Figure 3 is a storage strategy diagram for high-frequency read-write characteristic data according to an example embodiment;

[0021] Figure 4 Figure 4 is a storage strategy diagram for high-frequency read-write characteristic data according to an example embodiment;

[0022] Figure 5 Figure 5 is a storage strategy diagram for load balancing characteristic data according to an example embodiment;

[0023] Figure 6 Figure 6 is a storage strategy diagram for multiple data according to an example embodiment;

[0024] Figure 7 Figure 7 is a structure diagram of a device according to an example embodiment;

[0025] Figure 8 Figure 8 is a block diagram of a data storage device according to an example embodiment. DETAILED DESCRIPTION

[0026] For the purpose of making the purpose, technical scheme and advantages of the present specification clearer, the technical scheme of the present specification will be described clearly and completely below in combination with the specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present specification, not all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present specification.

[0027] In the current storage architecture, dynamic adaptation of device resource utilization and storage demand is still an industry technical problem. On the one hand, the traditional user exclusive mode forces the physical device to be bound with the user, resulting in low storage utilization of a single device and long-term idling of a large amount of hardware resources. When the user scale exceeds the total amount of devices (such as 3 devices need to carry dozens of users), the exclusive mode directly leads to the fact that the storage demand of new users cannot be met. This contradiction between "resource waste" and "insufficient capacity" makes it difficult for the traditional scheme to balance data isolation and resource utilization in a large-scale user scenario.

[0028] On the other hand, different business scenarios present significant differentiated characteristics in data storage requirements: high-frequency interaction type business data, users expect to shorten the communication time through centralized storage; high throughput and load balancing business data need to be evenly distributed to each physical device. However, the existing storage strategy adopts a fixed storage mode, which cannot adapt to the "centralized storage requirements" of high-frequency interaction scenarios, and it is also difficult to meet the "load balancing requirements" of high-throughput scenarios, leading to an increasingly prominent contradiction between the storage requirements of diversified business scenarios and the fixed storage mode.

[0029] Based on this, the present specification provides a data storage method, by splitting the target data into multiple data shards and storing them in a database cluster composed of multiple physical devices, and assigning storage space to the data shards according to the data read-write characteristics of the target data when the target data is provided with corresponding data read-write characteristics, and according to the load of the physical devices when the corresponding data read-write characteristics are provided, thereby realizing dynamic adaptation to different data read-write characteristics and intelligent scheduling of physical device load, effectively improving storage resource utilization and meeting the diversified storage requirements of a large number of users.

[0030] The technical solutions provided by the embodiments of the present specification will be described in detail below with reference to the accompanying drawings.

[0031] Figure 1 Fig. 1 is a flowchart of a data storage method provided by an exemplary embodiment, which includes:

[0032] S100: Obtain target data to be stored in a database cluster, and split the target data into multiple data shards; wherein the database cluster is composed of multiple physical devices, and each physical device is used to provide storage space respectively;

[0033] In the present specification, the execution subject for performing a data storage method can be a designated device such as a server, and for the sake of convenience, the following will take the server as an example to describe a data storage method provided by the present specification.

[0034] The server can obtain target data to be stored in a database cluster, which in the present specification can be graph data, of course, it can also be other data such as relational data, time series data or unstructured data, etc., which is not specifically limited in the present specification.

[0035] The database cluster can be composed of multiple physical devices, which can include storage devices such as a disk array, a solid state drive (SSD), a network attached storage (NAS), a storage area network (SAN) node, and a storage server, for providing storage space respectively.

[0036] For ease of understanding, the present specification provides a schematic diagram of an architecture of a database cluster, as shown in Figure 2

[0037] Figure 2 A schematic diagram of an architecture of a database cluster provided by an exemplary embodiment, the database cluster containing multiple logically interconnected physical devices, the database cluster can be composed of multiple physical devices under the same environment (such as the same machine room) through physical connection (such as optical fiber, twisted pair) or network connection (such as local area network), or multiple physical devices under different environments connected through a wide area network or a dedicated communication link.

[0038] For example, the database cluster can be a standardized database cluster constructed in advance based on multiple physical devices, or a dynamically constructed database cluster by deploying a dedicated software program in each device to automatically establish an interconnection communication link between devices when the user detects that the number of physical devices is insufficient.

[0039] After obtaining the target data, the server can split the target data into multiple data shards.

[0040] In the present specification, the user can upload target data to the server through a corresponding interactive interface or data management tool, and the user can set the data read-write characteristics of the target data and the number of data shards required to split at the same time of uploading the target data.

[0041] For example, the user can upload target data to the server through a client, and the client can show the user options for different data write characteristics and a setting control for the number of data shards for the user to select the data read-write characteristics and the number of data shards, and the setting control can be displayed in various forms, such as an input box, a drop-down list, etc.

[0042] The user's settings for data read-write characteristics and the number of shards can be submitted to the server in the form of a setting instruction, and then the server can determine the target shard number set by the user for the target data according to the shard setting instruction submitted by the user, and split the target data into multiple data shards according to the target shard number.

[0043] ​Of course, the user can also not set the number of shards, in which case the server can not shard the target data, or split the target data into multiple data shards according to a default number of shards.

[0044] S102: If the data read-write characteristic corresponding to the target data is read, the storage space is allocated for each data shard according to the data read-write characteristic, and if the data read-write characteristic corresponding to the target data is not read, the storage space is allocated for each data shard according to the load of the plurality of physical devices.

[0045] S104: Store each data shard in the storage space allocated for the data shard.

[0046] If the server reads the data read-write characteristic corresponding to the target data, the server can allocate storage space for each data shard according to the data read-write characteristic.

[0047] In this specification, the data read-write characteristic of the target data can include a high-frequency read-write characteristic and a load balancing characteristic, wherein:

[0048] The high-frequency read-write characteristic is used to characterize the attribute that the data needs to be frequently read and written. For example, in the execution process of online business, data shards often need to be read and written frequently, so data shards need to be stored in at least several devices to shorten the communication time.

[0049] In other words, the more centralized the storage of data shards (i.e. the fewer the number of storage devices occupied by the shards of the same target data), the shorter the communication path of the data in the transmission process, and the interaction process will also be reduced accordingly, thereby significantly reducing the communication time.

[0050] Specifically, there are the following typical scenarios in the data reading process, and the centralized storage of data shards can effectively shorten the communication time:

[0051] Serial forwarding scenario: when the data reading request is sent to one of the physical devices first, and then forwarded to other physical devices that store data shards one by one, centralized storage can greatly reduce the number of request forwarding times, directly reducing the communication time in the data read-write process.

[0052] Concurrent request scenario: when the data reading request needs to be sent to all physical devices that store data shards at the same time, centralized storage can significantly reduce the total number of requests. Since the shards of the same target data are stored in fewer physical devices, there is no need to initiate requests to a large number of devices, thereby simplifying the scale of communication interaction and further shortening the overall communication time.

[0053] Serial and parallel combination scenario: when a data read request is sent to a certain physical device first, and the data read request is sent to the physical devices storing the data shards by the certain physical device, the centralized storage can reduce the time-consuming of serial transmission by reducing the initial forwarding link, and can reduce the number of target devices of concurrent requests due to the centralized storage of data shards in a small number of devices, so as to simultaneously simplify the serial forwarding level and the scale of concurrent interaction in the serial and parallel combination process, and improve the overall communication efficiency.

[0054] Load balancing characteristic is used to characterize the attribute that data needs to be evenly distributed among multiple devices to share the access pressure. For example, in the case of business such as high-concurrency order processing and traffic distribution, the data throughput is large during the execution of the business, and the data shards need to be stored in each device to realize load balancing.

[0055] Specifically, if the data read-write characteristic is a high-frequency read-write characteristic, then:

[0056] In the case that there is a physical device in each physical device whose free storage space is not less than the target data, the storage space provided by the physical device is allocated to each data shard. In order to facilitate understanding, the present specification provides a storage strategy diagram for high-frequency read-write characteristic data, as shown in Figure 3 .

[0057] Figure 3 is a storage strategy diagram for high-frequency read-write characteristic data provided by an example embodiment.

[0058] Among them, assuming that the data size of the target data is x, there is a physical device A in the database cluster, the remaining storage space of which is y, and x≤y, then the storage space provided by the physical device is allocated to each data shard, so as to store each data shard of the target data in the physical device A.

[0059] It should be noted that when there are multiple physical devices whose remaining storage space is greater than the target data, the physical device with the largest remaining storage space (i.e. the least load) can be selected to allocate storage space for each data shard.

[0060] In the case that the free storage space in each physical device is less than the target data, according to the communication distance between each physical device and the size of the target data, a plurality of target devices are determined in each physical device, and the storage space provided by the target devices is allocated to each data shard, wherein the communication distance between the plurality of target devices is less than the communication distance between each target device and other physical devices.

[0061] Among them, the above communication distance can include the spatial distance between two physical devices.

[0062] Specifically, if multiple physical devices are deployed across regions, since the communication distance between physical devices in the same city is less than the communication distance between physical devices in different cities, the physical devices in the same region that can carry all target data are preferentially selected as target devices.

[0063] For example, if an organization has data centers in cities A, B and C, each of which has multiple physical devices, when a user in city A needs to store target data with high read-write frequency, even though the physical devices in cities B and C can also store the target data, the physical devices in city A are still preferentially selected to store the target data, thereby avoiding the delay caused by cross-city data transmission.

[0064] If multiple physical devices are deployed in different machine rooms in the same city, since the communication distance between physical devices in the same machine room is less than the communication distance between physical devices in different machine rooms, the physical devices in the same machine room that can carry all target data are preferentially selected as target devices.

[0065] For example, a city has two machine rooms A and B, each of which has multiple physical devices, when target data with high read-write frequency mainly flows in machine room A, the target data is preferentially stored in the physical devices in machine room A, thereby avoiding cross-machine room transmission and reducing communication time consumption.

[0066] Of course, the above communication distance can also include the network topology distance (such as network level, routing hop count, etc.) between physical devices, in which case the physical devices with the closest network topology distance and capable of carrying all target data are preferentially selected as target devices.

[0067] For example, a certain network architecture is divided into three levels of core layer, aggregation layer and access layer, each level has multiple physical devices deployed, and the access layer devices are directly connected to user terminals. When target data with high read-write frequency needs to be stored, if the remaining storage space of the access layer physical devices is sufficient to accommodate the data, even though the core layer and the aggregation layer have more idle devices, since the network topology distance between the access layer devices and the user terminals is the closest, the physical devices in the access layer are preferentially selected as target devices, thereby ensuring that the target devices are located in the physical layer.

[0068] For example, a local area network is provided with physical device C, physical device D and physical device E. Physical device C and physical device D can communicate through one routing hop, physical device C and physical device E need to pass through three routing hops, and physical device D and physical device E need to pass through two routing hops. If any one of C, D and E cannot store all target data, but any two physical devices can accommodate all target data, since the routing hops between physical device C and physical device D are the least, the data fragments of the target data are stored in physical device C and physical device D, thereby reducing the time consumption in the data transmission process.

[0069] It should be noted that in actual application, the server does not need to calculate the communication distance between physical devices one by one to screen target devices, but can perform nearby allocation based on a preset allocation mechanism, that is, preferentially allocating target data to physical devices meeting the "physical location or network topology nearby" condition and having data carrying capacity.

[0070] For ease of understanding, the present specification provides a storage strategy diagram for high-frequency read-write characteristic data, as shown in Figure 4 .

[0071] Figure 4 is a storage strategy diagram for high-frequency read-write characteristic data provided by an example embodiment.

[0072] Among them, assuming that the data size of the target data is x, and the free storage space in each physical device in the database cluster is less than the target data, the server can select the closest target devices in each physical device, and the sum of the free storage space of these target devices is not less than x.

[0073] If the data read-write characteristic is a load balancing characteristic, the server can allocate storage space for each data fragment in a balanced manner. For ease of understanding, the present specification provides a storage strategy diagram for load balancing characteristic data, as shown in Figure 5 .

[0074] Figure 5 is a storage strategy diagram for load balancing characteristic data provided by an example embodiment.

[0075] Among them, assuming that physical device A, physical device B and physical device C all have free storage space, the target data fragments are evenly divided into three parts and stored in physical device A, physical device B and physical device C respectively.

[0076] It should be noted that the above is only an example of data read-write characteristics of target data with high-frequency read-write characteristics and load balancing characteristics. In actual applications, other data read-write characteristics and corresponding sharding storage strategies can also be involved, which are not limited in the present specification.

[0077] Further, if the server does not read the data read-write characteristics corresponding to the target data, such as the target data not setting corresponding data read-write characteristics or the corresponding data read-write characteristics being set as random, the server allocates storage space for each data shard according to the load conditions of the plurality of physical devices.

[0078] For example, the server can determine a plurality of physical devices with a load less than a preset load from the plurality of physical devices, and randomly allocate each data shard to these physical devices for storage; for another example, the server can determine a plurality of physical devices with a free storage space greater than the size of a single data shard from the plurality of physical devices, and randomly allocate each data shard to these physical devices for storage. Of course, the server can also determine a physical device with the least load from the plurality of physical devices, and allocate each data shard to the physical device for storage.

[0079] Further, in the same physical device, the storage space can include a plurality of isolated storage spaces divided for different users, and each user only has access to its corresponding isolated storage space; wherein the storage space corresponding to each data shard obtained by splitting the target data is the isolated storage space corresponding to the user to which the target data belongs.

[0080] In addition, the isolated storage space in each physical device includes a plurality of sub-storage spaces, each sub-storage space is used to store a data shard. In the process of allocating storage space for data shards, the server can select a sub-storage space and determine the corresponding address information, and then construct the mapping relationship between any data shard and the selected sub-storage space according to the determined address information.

[0081] In the present specification, the above address information can include the address range or space identifier of the sub-storage space in the corresponding physical device, wherein the address range can be represented by the following methods:

[0082] Continuous space: defined by start address and end address, or determined by start address and address offset (i.e. space size);

[0083] Special case: for continuous and fixed-structure sub-storage spaces, the address range can also be simplified as any characteristic address (such as start address, middle address or end address) in the range, and the server can automatically derive the complete range based on a preset rule.

[0084] The space identifier can provide a unique identifier for the non-continuous or dynamically allocated sub-storage space, for quick positioning and mapping.

[0085] In this way, the user data stored in the same physical device can be isolated, and even if the data of multiple users is stored in the same physical device, through the permission control mechanism of the isolated storage space, each user can only operate the independent space allocated to it, which logically blocks the access path of cross-user data, ensuring that different user data stored in the same physical device is effectively guaranteed in terms of security, privacy and integrity.

[0086] In addition, the server can construct a mapping relationship between any data shard and the selected sub-storage space according to the determined address information; wherein the mapping relationship is used for read-write operation on any data shard in the database cluster. When the user needs to read the target data associated with the ID, the server can read the pre-constructed mapping table, which records the target data under the user ID, the data shards corresponding to the target data, and the mapping relationship between any data shard and the selected sub-storage space. In this way, the server can read the data shards of the target data from each sub-storage space of the database cluster based on the mapping table, and restore each data shard to the target data and return it to the user.

[0087] For ease of understanding, this specification also provides a data storage strategy diagram for multiple users, as shown in Figure 6

[0088] Figure 6 is a data storage strategy diagram for multiple users provided by an exemplary embodiment.

[0089] Among them, the data read-write characteristic of data X is load balancing characteristic, so its data shards are evenly distributed to each physical device for storage, the data read-write characteristic of data Y is not set, so its data shards are randomly distributed to each physical device for storage, and the data read-write characteristic of data Z is high-frequency read-write characteristic, so its data shards are stored in physical device C.

[0090] In this specification, the data read-write characteristic of the target data can be a static data read-write characteristic, or a dynamic data read-write characteristic, wherein the dynamic data read-write characteristic can be set by the user, or can be dynamically changed based on a preset rule.

[0091] ​Specifically, in the actual operation of the business, the server can monitor the data read-write situation of each isolated storage space, and then correct the data read-write characteristics of the target data based on the monitored data read-write situation of each isolated storage space, and then re-allocate the isolated storage space for each data shard according to the corrected data read-write characteristics.

[0092] In practical applications, the above data read-write situation can be the Queries Per Second (QPS) of the data, of course, it can also be the read-write times of the data within a set time.

[0093] When the data read-write characteristics of the target data are not set, the server can determine the corresponding data read-write characteristics based on the actual data read-write situation thereof, and re-allocate the storage space for each data shard based on the data read-write characteristics, if the storage space allocated to each data shard before and the re-allocated storage space are different, then each data shard is migrated to the re-allocated storage space, and the storage space that has changed does not need to be migrated.

[0094] When the target data is set with corresponding data read-write characteristics, if the actual data read-write characteristics of the target data are determined to be inconsistent with the set data read-write characteristics based on the corresponding data read-write situation, then the storage space for each data shard is re-allocated based on the re-determined data read-write characteristics, if the storage space allocated to each data shard before and the re-allocated storage space are different, then each data shard is migrated to the re-allocated storage space, and the storage space that has changed does not need to be migrated.

[0095] In this specification, for each isolated storage space, the server can determine the data read-write characteristics corresponding to the data shards stored in the isolated storage space according to the data read-write situation of the isolated storage space; according to the number of data shards under each data read-write characteristic, the data read-write characteristic with the largest number of corresponding data shards is determined as the corrected data read-write characteristic.

[0096] It should be noted that in this specification, a physical device can be provided with multiple isolated storage spaces, or only one isolated storage space, when only one isolated storage space is provided in a physical device, the size of the isolated storage space is equal to the total size of the storage space provided by the device.

[0097] From the above, it can be seen that the scheme realizes efficient utilization of storage resources and precise adaptation to diversified needs through data fragmentation and dynamic scheduling mechanism. Specifically, the target data is split into multiple data fragments and stored in a database cluster constructed by multiple physical devices, breaking the traditional "single exclusive" mode and laying the foundation for resource sharing and load balancing.

[0098] In the storage space allocation link, differentiated strategies are implemented according to data characteristics: if the target data has set specific read-write characteristics, the storage locations are allocated to each fragment accordingly; if the data has not set characteristics, the fragments are automatically allocated based on the real-time load status of each physical device in the cluster, avoiding excessive pressure on a single point.

[0099] Through this "characteristics-driven + load-aware" dual scheduling mechanism, the real access mode of data can be dynamically adapted during operation, and the resource use efficiency in the cluster can be intelligently balanced. Ultimately, the scheme not only significantly improves the storage space utilization of physical devices, but also effectively meets the diversified data storage demands in large-scale user environments by precisely matching the storage needs of different business scenarios.

[0100] Figure 7 is a schematic structural diagram of a device provided by an exemplary embodiment. Please refer to Figure 7 At the hardware level, the device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for functions. One or more embodiments of the present specification can be implemented in a software manner, such as reading a corresponding computer program from a non-volatile memory into a memory by a processor and then running. Of course, in addition to the software implementation, one or more embodiments of the present specification do not exclude other implementation manners, such as logic devices or a combination of software and hardware, etc. That is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0101] Please refer to Figure 8 , the data storage device can be applied to the device as shown in Figure 7 , to implement the technical solutions of the present specification. Wherein, the data storage device can include:

[0102] The acquisition unit 800 is configured to acquire target data to be stored in a database cluster, and split the target data into multiple data fragments; wherein the database cluster is composed of multiple physical devices, and each physical device is configured to provide a storage space;

[0103] The allocation unit 802 is configured to, if the target data is found to be provided with the data read-write characteristic corresponding to the target data, allocate storage spaces for each data shard according to the data read-write characteristic; or if the target data is found not to be provided with the data read-write characteristic corresponding to the target data, allocate storage spaces for each data shard according to the load conditions of the plurality of physical devices.

[0104] The storage unit 804 is configured to store each data shard into the storage space allocated for the data shard.

[0105] Optionally, the storage spaces in the same physical device include a plurality of isolated storage spaces divided for different users, and each user has access rights only to the isolated storage space corresponding to the user; and the storage spaces corresponding to the data shards obtained by splitting the target data are the isolated storage spaces corresponding to the user to which the target data belongs.

[0106] Optionally, the isolated storage spaces include a plurality of sub-storage spaces, and each sub-storage space is used to store one data shard.

[0107] The allocation unit 802 is specifically configured to select a sub-storage space and determine corresponding address information, wherein the address information includes an address range or a space identifier of the sub-storage space in the corresponding physical device; and construct a mapping relationship between the any data shard and the selected sub-storage space according to the determined address information; and the mapping relationship is used for read-write operations on the any data shard in the database cluster.

[0108] Optionally, the allocation unit 802 is specifically configured to, if the data read-write characteristic is a high-frequency read-write characteristic, allocate storage spaces provided by a physical device to the data shards in a case where the physical device has a free storage space no less than the target data; or in a case where the free storage spaces in the physical devices are all less than the target data, determine a plurality of target devices in the physical devices according to communication distances between the physical devices and a size of the target data, and allocate storage spaces provided by the target devices to the data shards, wherein the communication distances between the target devices are less than the communication distances between the target devices and other physical devices.

[0109] Optionally, the allocation unit 802 is specifically configured to, if the data read-write characteristic is a load balancing characteristic, allocate storage spaces for each data shard in a manner of balancing the data shards to each physical device.

[0110] Optionally, before allocating storage spaces to each data shard according to the data read-write characteristic, the obtaining unit 800 is further configured to determine a target number of shards set by a user for the target data according to a shard setting instruction submitted by the user; and split the target data into the plurality of data shards according to the target number of shards.

[0111] Optionally, the storage unit 804 is further configured to monitor data read-write situations corresponding to each storage space; and correct the data read-write characteristic corresponding to the target data based on the monitored data read-write situations corresponding to each storage space.

[0112] According to the corrected data read-write characteristic, the storage spaces are re-allocated to the data shards.

[0113] Optionally, the storage unit 804 is specifically configured to, for each storage space, determine a data read-write characteristic corresponding to a data shard stored in the storage space according to a data read-write situation corresponding to the storage space; and determine a data read-write characteristic corresponding to the most data shards as the corrected data read-write characteristic according to the number of data shards under each data read-write characteristic.

[0114] Based on the same idea as the above method, the specification also provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0115] Based on the same idea as the above method, the specification also provides a computer program product, which includes computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps of the method according to any one of the above embodiments.

Claims

1.A data storage method, comprising: obtaining target data to be stored in a database cluster, and splitting the target data into a plurality of data shards, wherein the database cluster is composed of a plurality of physical devices, each of which is configured to provide a storage space; if a data read-write characteristic corresponding to the target data is read, assigning a storage space to each data shard according to the data read-write characteristic, or if the data read-write characteristic corresponding to the target data is not read, assigning a storage space to each data shard according to a load condition of the plurality of physical devices; storing each data shard in the storage space assigned to the data shard. 2.The method of claim 1, wherein the storage space in the same physical device comprises a plurality of isolated storage spaces divided for different users, and each user only has access to its corresponding isolated storage space; wherein, The storage space corresponding to each data shard obtained by splitting the target data is an isolated storage space corresponding to a user to which the target data belongs. 3.The method of claim 2, wherein the isolated storage space comprises a plurality of sub-storage spaces, each of which is configured to store a data shard. The method of assigning a storage space to any data shard comprises: selecting a sub-storage space and determining corresponding address information, wherein the address information comprises an address range or a space identifier of the sub-storage space in a corresponding physical device; constructing a mapping relationship between the any data shard and the selected sub-storage space according to the determined address information, wherein the mapping relationship is used for read-write operation on the any data shard in the database cluster. 4.The method of claim 1, wherein assigning a storage space to each data shard according to the data read-write characteristic comprises: if the data read-write characteristic is a high-frequency read-write characteristic, then: in a case where there is a physical device in the plurality of physical devices that has a free storage space not less than the target data, assigning a storage space provided by the physical device to the data shards; or in a case where the free storage space in each of the plurality of physical devices is less than the target data, determining a plurality of target devices in the plurality of physical devices according to a communication distance between the physical devices and a size of the target data, and assigning a storage space provided by the target devices to the data shards, wherein the communication distance between the target devices is less than a communication distance between each target device and other physical devices. 5.The method of claim 1, wherein assigning a storage space to each data shard according to the data read-write characteristic comprises: if the data read-write characteristic is a load balancing characteristic, then assigning a storage space to each data shard in a manner of balancing the data shards to each physical device. 6.The method of claim 1, wherein before assigning a storage space to each data shard according to the data read-write characteristic, the method further comprises: determining a target number of shards set by a user for the target data according to a shard setting instruction submitted by the user; and splitting the target data into a plurality of data shards, comprising: splitting the target data into the plurality of data shards according to the target number of shards. 7.The method of claim 1, further comprising: monitoring data read-write conditions of each storage space. correct the data read-write characteristic corresponding to the target data based on the monitored data read-write situation corresponding to each storage space; reallocate storage spaces for the data shards according to the corrected data read-write characteristic. 8.The method of claim 7, wherein the correcting the data read-write characteristic corresponding to the target data based on the monitored data read-write situation corresponding to each storage space comprises: determining, for each storage space, the data read-write characteristic corresponding to the data shard stored in the storage space according to the data read-write situation corresponding to the storage space; and determining the data read-write characteristic corresponding to the most data shards as the corrected data read-write characteristic according to the number of data shards corresponding to each data read-write characteristic. 9.A data storage apparatus comprising: an obtaining unit configured to obtain target data to be stored in a database cluster and split the target data into a plurality of data shards, wherein the database cluster is composed of a plurality of physical devices, and each physical device is configured to provide a storage space; an allocating unit configured to allocate storage spaces for the data shards according to a data read-write characteristic corresponding to the target data if the data read-write characteristic is read, or according to the load conditions of the plurality of physical devices if the data read-write characteristic is not read; and a storing unit configured to store each data shard in the storage space allocated for the data shard. a processor; a memory for storing processor-executable instructions, wherein the processor implements the steps of the method of any one of claims 1-8 by running the executable instructions. 11.A computer-readable storage medium having computer instructions stored thereon, wherein the instructions are executed by a processor to implement the steps of the method of any one of claims 1-8. 12.A computer program product comprising computer program / instructions, wherein the computer program / instructions are executed by a processor to implement the steps of the method of any one of claims 1-8. ​ 10. An electronic device comprising: ​ ​ ​ ​

Images