A method of data storage, related apparatus and device

Abstract

Embodiments of the present application disclose a data storage method, related device and equipment, which are used for avoiding resource waste of data storage. The method comprises the following steps: obtaining to-be-transmitted data, performing fragmentation processing on the to-be-transmitted data according to a target data type to obtain K pieces of fragmented data and an actual storage amount corresponding to each piece of fragmented data, allocating a preset storage amount to each piece of fragmented data according to the actual storage amount corresponding to each piece of fragmented data, determining a storage address of each piece of fragmented data if a data tag of each piece of fragmented data exists in a historical storage record, the historical storage record comprising a mapping relationship between data tags and storage addresses within a historical time, sending a resource creation request to a server according to the storage address of each piece of fragmented data, so that the server creates storage resources matching the preset storage amount for each piece of fragmented data, and sending the K pieces of fragmented data to the server, so that the server stores the K pieces of fragmented data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data storage method, related apparatus and equipment. Background Technology

[0002] With the rapid development of computer and internet technologies, the internet is growing faster and faster, generating ever-increasing amounts of data every day. To prevent network crashes, distributed storage cluster technology can be used to process this data. A distributed storage cluster involves building a network topology with multiple storage servers to form a cluster. Within this cluster, relevant algorithms provide data reading, writing, and disk persistence functions. Distributed storage clusters utilize multiple storage servers to share the storage load.

[0003] Existing distributed storage cluster technology processes data by pre-allocating a fixed amount of storable disk space and then using the allocated disk space on each machine in the storage server cluster via the network to store the received data according to the fixed allocated disk space. However, since the size of the received data is uncontrollable, unreasonable disk space allocation can easily occur, resulting in wasted disk space resources or data failure to be stored successfully. Summary of the Invention

[0004] This application provides a data storage method, related apparatus, and equipment, which are used to divide large amounts of data to be transmitted into K smaller data segments for distributed transmission by performing data segmentation processing, thereby reducing the data storage load. Furthermore, by allocating a preset storage amount to each data segment, the server can create storage resources matching the preset storage amount for each data segment to store the K data segments, thus avoiding resource waste.

[0005] One embodiment of this application provides a data storage method, including:

[0006] Obtain the data to be transmitted, wherein the data to be transmitted carries a target data type, which is either a time type or a file type;

[0007] Based on the target data type, the data to be transmitted is fragmented to obtain K fragments and the actual storage amount corresponding to each fragment. Each fragment corresponds to a data tag, and K is an integer greater than 1.

[0008] Based on the actual storage amount corresponding to each data shard, a preset storage amount is allocated to each data shard, wherein the preset storage amount is greater than or equal to the corresponding actual storage amount;

[0009] If the historical storage record contains data tags for each data segment, then the storage address of each data segment is determined. The historical storage record includes the mapping relationship between data tags and storage addresses within a historical period.

[0010] Based on the storage address of each data shard, a resource creation request is sent to the server so that the server can create storage resources for each data shard that match the preset storage amount. The resource creation request carries the preset storage amount of the data shard.

[0011] Send K data fragments to the server so that the server can store K data fragments.

[0012] Another aspect of this application provides a data storage apparatus, comprising:

[0013] The acquisition unit is used to acquire data to be transmitted, wherein the data to be transmitted carries a target data type, which is either a time type or a file type;

[0014] The processing unit is used to perform fragmentation processing on the data to be transmitted according to the target data type, to obtain K fragments and the actual storage amount corresponding to each fragment, wherein each fragment corresponds to a data tag, and K is an integer greater than 1;

[0015] The processing unit is also used to allocate a preset storage amount to each data segment based on the actual storage amount corresponding to each data segment, wherein the preset storage amount is greater than or equal to the corresponding actual storage amount;

[0016] The determining unit is used to determine the storage address of each data segment if a data tag for each data segment exists in the historical storage record, wherein the historical storage record includes the mapping relationship between data tags and storage addresses within a historical period;

[0017] The processing unit is also configured to send a resource creation request to the server based on the storage address of each data segment, so that the server creates storage resources matching the preset storage amount for each data segment, wherein the resource creation request carries the preset storage amount of the data segment;

[0018] The processing unit is also used to send K data fragments to the server so that the server can store the K data fragments.

[0019] In one possible design, in another implementation of the embodiments of this application,

[0020] The processing unit is also used to send an address allocation request to the server if there is no data tag for the fragmented data in the historical storage record, so that the server allocates the corresponding storage address for the fragmented data according to the address allocation request.

[0021] The receiving unit is used to receive the address allocation response sent by the server, wherein the address allocation response carries the storage address allocated for the fragmented data.

[0022] In one possible design, in another implementation of the embodiments of this application, the processing unit may specifically be used for:

[0023] Based on the data tag of each data fragment, determine the transmission identifier of each data fragment, where the transmission identifier is used to indicate the transmission order of the data fragments;

[0024] The transmission identifiers of each data fragment are arranged to obtain a data transmission sequence, wherein the data transmission sequence includes K transmission identifiers arranged in descending or ascending order;

[0025] K data fragments are sent to the server sequentially according to the data transmission sequence.

[0026] In one possible design, in another implementation of the embodiments of this application,

[0027] The receiving unit is also used to receive K segments of storage progress sent sequentially by the server, wherein one data segment corresponds to one segment of storage progress.

[0028] The processing unit is also used to send K segments of storage progress to the client sequentially according to the data transmission sequence, so that the client can display the K segments of storage progress.

[0029] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0030] Receive data upload requests sent by clients, where the data upload requests carry the target data type;

[0031] Respond to data upload requests and retrieve the data to be processed;

[0032] Based on the target data type, the data to be processed is transformed to obtain the data to be transmitted.

[0033] In one possible design, in another implementation of the embodiments of this application, the acquisition unit may specifically be used for:

[0034] If the target data type is time, the data to be processed will be converted into time-data-value format data to be transmitted.

[0035] If the target data type is a file type, the data to be processed will be converted into data to be transmitted in filename-file object format.

[0036] In one possible design, in another implementation of the embodiments of this application,

[0037] The receiving unit is also used to receive a first read request sent by the client, wherein the first read request carries a data read range;

[0038] The determining unit is also used to respond to the first read request and determine T data tags according to the data read range, where T is an integer greater than or equal to 1 and less than or equal to K;

[0039] The determining unit is also used to determine the storage address corresponding to each data tag among T data tags based on historical storage records;

[0040] The processing unit is also used to send a second read request to the server according to the storage address corresponding to each data tag, so that the server responds to the second read request and obtains T fragments of data;

[0041] The receiving unit is also used to receive T data fragments sent by the server.

[0042] Another aspect of this application provides a computer device, including: a memory, a transceiver, a processor, and a bus system;

[0043] The memory is used to store programs;

[0044] The processor implements the methods described above when executing a program in memory;

[0045] Bus systems are used to connect memory and processor to enable communication between them.

[0046] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0047] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a network device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the network device to perform the methods provided in the above aspects.

[0048] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0049] By acquiring the data to be transmitted, and based on the target data type carried by the data, the data is fragmented to obtain K fragments and the actual storage capacity corresponding to each fragment. A preset storage capacity is allocated to each fragment based on its actual storage capacity. When the data tag for each fragment exists in the historical storage record, the storage address for each fragment is determined. Based on the storage address of each fragment, a resource creation request is sent to the server, causing the server to create storage resources matching the preset storage capacity for each fragment. Then, the K fragments are sent to the server for storage. Through this method, large amounts of data to be transmitted can be divided into K smaller fragments for distributed transmission, reducing data storage load. Furthermore, by allocating a preset storage capacity to each fragment, the server can create storage resources matching the preset capacity to store the K fragments, avoiding resource waste. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the architecture of a data object control system in an embodiment of this application;

[0051] Figure 2 This is a flowchart of one embodiment of the data storage method in this application;

[0052] Figure 3 This is a flowchart of another embodiment of the data storage method in this application;

[0053] Figure 4 This is a flowchart of another embodiment of the data storage method in this application;

[0054] Figure 5 This is a flowchart of another embodiment of the data storage method in this application;

[0055] Figure 6 This is a flowchart of another embodiment of the data storage method in this application;

[0056] Figure 7 This is a flowchart of another embodiment of the data storage method in this application;

[0057] Figure 8 This is a flowchart of another embodiment of the data storage method in this application;

[0058] Figure 9 This is a flowchart illustrating the data storage method in an embodiment of this application.

[0059] Figure 10(a) is a schematic diagram of a data upload structure of a data storage method in an embodiment of this application;

[0060] Figure 10(b) is a schematic diagram of a data acquisition structure of a data storage method in an embodiment of this application;

[0061] Figure 10(c) is a schematic diagram of a message structure of a data storage method in an embodiment of this application;

[0062] Figure 11 This is a message interaction diagram illustrating a data storage method in an embodiment of this application;

[0063] Figure 12 This is a schematic diagram of one embodiment of the data storage device in this application;

[0064] Figure 13 This is a schematic diagram of one embodiment of the computer device described in this application. Detailed Implementation

[0065] This application provides a data storage method, related apparatus, and equipment, which are used to divide large amounts of data to be transmitted into K smaller data segments for distributed transmission by performing data segmentation processing, thereby reducing the data storage load. Furthermore, by allocating a preset storage amount to each data segment, the server can create storage resources matching the preset storage amount for each data segment to store the K data segments, thus avoiding resource waste.

[0066] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] It should be understood that the entity recognition method provided in this application can be applied to scenarios where data storage is accomplished through data fragmentation. For example, it can be used to fragment daily weather change data for storage. Another example is to fragment monthly stock price fluctuation data for storage. Yet another example is to fragment quarterly product sales reports for storage. In these scenarios, existing solutions for data storage involve pre-allocating a fixed amount of storable disk space and then using the allocated disk space on each machine in a storage server cluster via a network to store the received data according to the fixed allocated disk space. However, since the size of the received data is uncontrollable, unreasonable disk space allocation can easily occur, leading to wasted disk space resources or data storage failure.

[0068] To address the aforementioned problems, this application proposes a data storage method, which is applied to... Figure 1 For the data object system shown, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of the architecture of a data object control system in an embodiment of this application, such as... Figure 1 As shown, by acquiring the data to be transmitted from the client, and based on the target data type carried by the data, the data is fragmented to obtain K fragments and the actual storage amount corresponding to each fragment. Based on the actual storage amount of each fragment, a preset storage amount is allocated to each fragment. When the data tag for each fragment exists in the historical storage record, the storage address of each fragment is determined. Based on the storage address of each fragment, a resource creation request is sent to the server, causing the server to create storage resources matching the preset storage amount for each fragment. Then, the K fragments are sent to the server for storage. Through this method, by fragmenting the data to be transmitted, large amounts of data are divided into K smaller fragments for distributed transmission, reducing the data storage load. Furthermore, by allocating a preset storage amount to each fragment, the server creates storage resources matching the preset storage amount to store the K fragments, avoiding resource waste.

[0069] Understandable, Figure 1 The diagram only shows one type of client terminal device. In real-world scenarios, many more types of terminal devices can participate in the data processing, such as personal computers (PCs). The specific number and types depend on the actual scenario and are not limited here. Additionally, Figure 1The image shows a server, but in real-world scenarios, multiple servers can be involved, especially in scenarios involving multi-model training and interaction. The number of servers depends on the specific scenario and is not limited here.

[0070] It should be noted that in this embodiment, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and the terminal device and the server can be connected to form a blockchain network; this application does not impose any restrictions.

[0071] To address the aforementioned issues, this application proposes a data storage method, which is generally executed by a server or terminal device. Accordingly, the device used for data storage is generally located in the server or terminal device.

[0072] It is understood that, as disclosed in this application, the data storage method, related equipment, and apparatus can comprise a blockchain, with multiple servers / terminal devices forming nodes on the blockchain. In practical applications, data sharing between nodes is required within the blockchain, and each node can store time data and file data, etc.

[0073] The data storage method in this application will be described below; please refer to [link / reference]. Figure 2 One embodiment of the data storage method in this application includes:

[0074] In step S101, data to be transmitted is obtained, wherein the data to be transmitted carries a target data type, which is a time type or a file type;

[0075] In this embodiment, when a user wants to upload data to the server for storage, the user can generate a data upload request through the client application interface. This allows the data to be uploaded to the server's Actor model for processing, so that the large amount of data to be transmitted can be divided into K smaller data fragments for distributed transmission, thereby reducing the data storage load to a certain extent.

[0076] Among them, the Actor model is the most basic unit of computation. The Actor model can receive a message and perform computation based on it. The Actor model is stateful and can be self-controlled or have its state controlled by the message sender.

[0077] Specifically, such as Figure 9 As shown, users can upload data through the client's application interface and generate a data upload request according to the message structure shown in Figure 10(c). For example, if a user wants to upload a week's worth of weather change data, a data upload request can be generated containing the event name "Weather Change Data", the current progress is empty, and the data to be transmitted is a week's worth of weather change data.

[0078] The message structure can be transmitted between the client and the server using sockets, such as... Figure 11 As shown, users can generate data upload requests through the client application interface and follow the steps shown. Figure 11 The StartSocket shown sends a data upload request to the server.

[0079] Among them, Socket is a communication mechanism provided by TCP between two computers. The client program creates a socket and attempts to connect to the server's socket. When the connection is established, the server creates a Socket object, which allows the client and server to communicate by writing to and reading from the Socket object.

[0080] Furthermore, data upload requests can be uploaded to a network layer, i.e., a network protocol, such as... Figure 9 The top-level Actor model shown enables the top-level Actor model to respond to the data upload request and obtain the data to be transmitted, which is of time type or file type, carried in the data upload request.

[0081] In step S102, the data to be transmitted is fragmented according to the target data type to obtain K fragments and the actual storage amount corresponding to each fragment. Each fragment corresponds to a data tag, and K is an integer greater than 1.

[0082] In this embodiment, after acquiring the data to be transmitted, the server can use the Actor model to pass the acquired data to the Akka Cluster Sharding module for sharding processing, thereby obtaining K shard data and the actual storage amount corresponding to each shard data, and assigning a data tag to each shard data. This can divide the large amount of data to be transmitted into K smaller shard data for distributed transmission, thereby reducing the data storage load to a certain extent.

[0083] Sharding is a mechanism for distributing data across nodes. In other words, sharding aims to distribute data as evenly as possible across different nodes to achieve a balanced distribution.

[0084] Among them, data sharding is a mechanism based on data distribution to nodes. The acquired data to be transmitted is divided into K "uniform" smaller data sets according to a certain "uniform" division method.

[0085] The actual storage capacity is the amount of storage space resources actually required to store each data shard. The actual storage capacity can be approximated by the actual data size of each data shard, that is, the actual storage capacity can be slightly greater than or equal to the actual data size of each data shard, so that the server can create storage space resources that match the actual storage capacity to store the data shards.

[0086] Sharding is a sharding addressing mapping framework implemented for the Actor model. It is not a traditional sharding of data, but can be understood as sharding of computation.

[0087] The principle behind Actor Sharding's fragmentation of data to be transmitted is roughly as follows:

[0088] 1. Treat each data shard as an entity and create an Actor, such as... Figure 9 As shown in Actor1 to Actorn, each Actor has an Actor address as a unique identifier, and all messages of each data segment are processed by the corresponding Actor.

[0089] 2. If the Actor does not work for a period of time, it will time out and automatically destroy itself (kill self);

[0090] 3. When a new node, i.e. a new shard of data, is added to a storage service cluster, the new Actor will be automatically created on the new node. Alternatively, the old Actor will be load-balanced and migrated to the new node. Similarly, when a node fails, the failed actor will be migrated to the new node.

[0091] For ease of understanding, combined with Figure 9Step S102 will be explained below. In this embodiment, the ActorSharding module is used to shard the data to be transmitted. Taking the weather change data of a week as an example, when the weather change data of a week is obtained, the server can use the Actor model to pass the obtained weather change data of a week to the Actor Sharding module for sharding. The weather change data of a week can be divided into shards corresponding to Monday to Sunday according to the unit dimension of days, such as weather data of Monday, weather data of Tuesday, etc., and each shard is assigned a data label, such as Monday weather or Sunday weather. At the same time, the Actor Sharding module can calculate the actual storage amount corresponding to each shard based on the data volume of each shard. For example, if the weather data of Monday is 48k, the actual storage amount of the weather data of Monday is calculated to be 48k.

[0092] In step S103, a preset storage amount is allocated to each data segment based on the actual storage amount corresponding to each data segment, wherein the preset storage amount is greater than or equal to the corresponding actual storage amount.

[0093] In this embodiment, after obtaining the actual storage amount corresponding to each data shard, the preset storage amount corresponding to each data shard can be calculated by the Actor Sharding module based on the actual storage amount corresponding to each data shard.

[0094] The preset storage amount corresponding to each data shard is the amount of storage space resources that the server can allocate to store the data shard. In order for the data shard to be successfully stored on the server, the preset storage amount is usually greater than the corresponding actual storage amount, or it can be equal to the actual storage amount.

[0095] For ease of understanding, combined with Figure 9Step S103 will be explained below. In this embodiment, we take Monday's weather data as the data to be sharded, and the actual storage amount corresponding to each data shard is 48k. When the actual storage amount of Monday's weather data is 48k, the server can estimate the storage space resources required for the actual storage amount of Monday's weather data of 48k through the ActorSharding module. For example, if the preset storage amount is greater than the upper limit of the corresponding actual storage amount by +10k, then the ActorSharding module can calculate the preset storage amount corresponding to each data shard based on the actual storage amount of Monday's weather data of 48k. For example, if the actual storage amount of Monday's weather data is 48k, the preset storage amount of Monday's weather data is calculated to be 58k. This can ensure that Monday's weather data is successfully stored on the server and can also avoid the mismatch of the storage space resources allocated by the server, thereby avoiding resource waste to a certain extent.

[0096] In step S104, if the historical storage record contains a data tag for each data segment, then the storage address of each data segment is determined. The historical storage record includes the mapping relationship between data tags and storage addresses within a historical time period.

[0097] In this embodiment, after obtaining each data shard and its data tag, the Actor Sharding module can search the historical storage records for a data tag that matches the data tag of the data shard within a specific historical period. If such a tag exists, it means that the Actor Sharding module has found data characteristics in the historical data that match the data shard. Based on the mapping relationship between data tags and storage addresses in the historical period, a storage address can be provided for the data shard. This allows the Actor Sharding module to allocate each data shard to the corresponding Actor based on its storage address, and then transmit each data shard to the server through the Actor. The server then creates storage resources matching the preset storage amount for each data shard to store K data shards, thus avoiding resource waste.

[0098] The storage address is the Actor address, which is used to accurately allocate fragmented data to the corresponding Actor node for data transmission or processing.

[0099] For ease of understanding, this application embodiment uses Monday's meteorological data as an example, with the data tag "Monday Weather" for the Monday meteorological data. When Monday's meteorological data and its data tag "Monday Weather" are obtained, the server can use the Actor Sharding module to search the historical storage records for a historical data tag that matches the data tag of Monday's meteorological data. If a historical data tag matching Monday's meteorological data is found, the storage address that maps to the historical data tag "Monday Weather" can be used as the storage address for Monday's meteorological data. This allows the Monday meteorological data to be allocated to the corresponding Actor, such as Actor1, based on its storage address. Each data segment is then transmitted to the server via Actor1, enabling the server to create storage resources matching the preset storage volume for each data segment to store Monday's meteorological data, thereby avoiding resource waste to some extent.

[0100] In step S105, a resource creation request is sent to the server according to the storage address of each data segment, so that the server creates storage resources matching the preset storage amount for each data segment, wherein the resource creation request carries the preset storage amount of the data segment;

[0101] In this embodiment, after obtaining the storage address of each data segment, a resource creation request carrying a preset storage amount of the data segment can be sent to the server, so that the server can create storage resources matching the preset storage amount for each data segment. This allows the server to allocate storage resources that are suitable for each data segment to accurately store each data segment, thereby avoiding resource waste to a certain extent.

[0102] Specifically, such as Figure 9As shown, when the storage address of each data shard is obtained, a resource creation request carrying the preset storage amount of the data shard can be sent to the underlying data server through the internal gateway. When the data server receives the resource creation request, it can check the currently available storage resources and create storage resources matching the preset storage amount. At the same time, it generates the corresponding Actor for the data shard. This Actor is used to receive the data shard and transfer the data shard to the storage resources created by the server for storage. When the server has created the storage resources matching the preset storage amount and generated the corresponding Actor, it can be understood that the server is ready to receive data shards. Therefore, the ready state can be notified to the Actor Sharding module in the form of a message structure, informing the Actor Sharding module that it can send the data shards to the server for storage. It can be understood that if the data server receives the resource creation request and finds that the currently available storage resources are insufficient, it can notify the developer to expand the resources and notify the Actor Sharding module of the waiting state in the form of a message structure, informing the Actor Sharding module that the current resources are insufficient and the data shards need to wait.

[0103] For ease of understanding, see Figure 10(c) and Figure 11 As shown, this embodiment of the application uses Monday's meteorological data as the fragmented data, with a preset storage size of 58k for the fragmented data. The storage address for Monday's meteorological data is Actor1. When the storage address for Monday's meteorological data is obtained as Actor1, as shown in Figure 10(c), the resource creation request carrying the preset storage size of the fragmented data can be presented in the form of a message structure, and as... Figure 11 As shown, the StartSocket socket is used to send the resource creation request to the server. When the server receives the resource creation request carrying the preset storage amount of fragmented data, it can create a 58k storage resource matching the preset storage amount, and at the same time generate Actor1 corresponding to the storage address of the fragmented data. Then, the ready state can be presented in the form of a message structure as shown in Figure 10(c), and using the following... Figure 11 The MiddleSocket socket shown notifies the ActorSharding module of its ready status, informing it that the ActorSharding module can send the sharded data to the server for storage.

[0104] The StartSocket is a message structure initiated by the client, which can be used to send upload or retrieval requests, and also to establish a long-lived connection between the client and the data server. Additionally, the data server returns a result message structure from the StartSocket, informing the client whether transmission is ready or not.

[0105] The MiddleSocket socket is a message structure that the data server can return to the client based on the progress of loading or saving data. This message structure can contain information such as data fragments, current progress, and total progress.

[0106] In step S106, K data fragments are sent to the server so that the server can store the K data fragments.

[0107] In this embodiment, after receiving the ready status notification from the data server, the ActorSharding module can distribute K data shards to the corresponding Actors through the storage addresses of each data shard. This allows the Actors to transfer the data shards to the storage resources created by the server for storage, ensuring accurate and smooth storage of each data shard and thus avoiding resource waste to a certain extent.

[0108] For ease of understanding, see Figure 10(c) and Figure 11 As shown, this embodiment uses Monday's meteorological data as the sharded data, with a preset storage size of 58k. The storage address for Monday's meteorological data is Actor1. When the ready status notification is received from the data server, the Actor Sharding module can use Monday's meteorological data and its storage address (Actor1) to send the data to Actor1. Then, Actor1 can transfer the Monday's meteorological data to the 58k storage resource created by the server. When the data server completes the storage of K shards, a message structure as shown in Figure 10(c) can be used. The data storage completion request can be presented as a message structure, and... Figure 11 As shown, the finishSocket socket is used to send a data storage completion request to the client.

[0109] The finishSocket is a message structure that the data server sends to the client after processing all the data. This message structure can contain the current progress and the total progress. After receiving the message, the client will disconnect from the data server.

[0110] It is understandable that the data to be transmitted can be time-based data or file-based data. However, some special file data, such as important encrypted file data, will only be stored on one server, while time-series data can be stored on different data servers according to the time characteristics of the data.

[0111] It's worth noting that when the storage server cluster is idle, it can perform a full node backup or a backup of the data stored on the nodes according to the backup strategy of the storage server cluster. Different data servers can back up data through actor communication and return the backup status to both communicating parties. If any data server fails or malfunctions, causing the stored data to become unusable, the backed-up data can be obtained from other data servers that have backed up data with the failed data server through actor communication and can continue to be used to maintain the normal use of the data.

[0112] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the data storage method provided in this application, such as... Figure 3 As shown, the method also includes:

[0113] In step S301, if there is no data tag for the fragmented data in the historical storage record, an address allocation request is sent to the server so that the server allocates the corresponding storage address for the fragmented data according to the address allocation request.

[0114] In step S302, an address allocation response sent by the server is received, wherein the address allocation response carries the storage address allocated for the fragmented data.

[0115] In this embodiment, after obtaining each data shard and its data tag, the Actor Sharding module searches the historical storage records for a data tag within a historical period that matches the data tag of each data shard. If no such tag exists, it means the Actor Sharding module did not find any data characteristics matching the data shard in the historical data. This data shard is new data that has not been processed before. In this case, an address allocation request is sent to the server so that the server can allocate a corresponding storage address for the data shard. Then, the module receives an address allocation response from the server carrying the allocated storage address for the data shard. This allows the Actor Sharding module to allocate each data shard to the corresponding Actor based on its storage address and transmit each data shard to the server. This enables the server to create storage resources matching the preset storage amount for each data shard to store K data shards, thus avoiding resource waste.

[0116] For ease of understanding, see Figure 10(c) and Figure 11 As shown, this embodiment of the application uses Sunday's weather data as the data segment and Sunday's weather data data tag as Sunday Weather as an example. When Sunday's weather data and Monday's weather data are obtained with the data tag "Sunday Weather", the server can use the Actor Sharding module to search the historical storage records to see if there is a data tag in the server that matches the data tag of Sunday's weather data within a historical time period. When no data tag for Sunday Weather is found in the historical time period, the address allocation request can be sent in the form of a message structure as shown in Figure 10(c). Figure 11 The StartSocket socket shown is sent to the server so that the server allocates the corresponding storage address, such as Actor7, for the fragmented data according to the address allocation request. Then, the address allocation response sent by the server, carrying the storage address Actor7 allocated for the fragmented data, can be received in the form of a message structure as shown in Figure 10(c). The address allocation response is then transmitted through a method such as... Figure 11 The MiddleSocket socket shown is sent to the Actor Sharding module so that the Actor Sharding module can allocate the Sunday weather data to the corresponding Actor7 according to the storage address Actor7 of the Sunday weather data, and then transmit the Sunday weather data to the server through Actor7. This allows the server to create storage resources that match the preset storage amount for the Sunday weather data to store the Sunday weather data, thus avoiding resource waste.

[0117] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the data storage method provided in this application, such as... Figure 4 As shown, K data fragments are sent to the server, including:

[0118] In step S401, a transmission identifier for each data fragment is determined based on the data tag of each data fragment, wherein the transmission identifier is used to indicate the transmission order of the data fragments.

[0119] In step S402, the transmission identifiers of each data fragment are arranged to obtain a data transmission sequence, wherein the data transmission sequence includes K transmission identifiers arranged in descending or ascending order.

[0120] In step S403, K data fragments are sent to the server sequentially according to the data transmission sequence.

[0121] In this embodiment, after obtaining the data tag of each data segment, the transmission identifier carried in each data tag can be obtained as the transmission identifier of each data segment. Then, the obtained transmission identifiers can be arranged to obtain a data transmission sequence including K transmission identifiers in descending or ascending order. Then, according to the data transmission sequence, the K data segments can be sent to the Actors corresponding to the storage addresses in sequence through the storage addresses corresponding to the K data segments. This allows the Actors to transmit the K data segments to the storage resources created by the server for storage, ensuring accurate and smooth storage of the K data segments, thereby avoiding resource waste to a certain extent.

[0122] The transmission identifier (identity, ID) is used to indicate the transmission order of the fragmented data. It can be represented as an integer (int) string or a string, etc.

[0123] Specifically, once the data tags for each data segment are obtained, the transmission identifier can be determined by checking if each data segment tag carries a transmission identifier. If so, the transmission identifier for each data segment can be identified. For example, the data tag for Monday's weather data is "Monday Weather," carrying the transmission identifier "01"; the data tag for Tuesday's weather data is "Tuesday Weather," carrying the transmission identifier "02"; and the data tag for Wednesday's weather data is "Wednesday Weather," carrying the transmission identifier "03." Similarly, the data tag for Sunday's weather data is "Sunday Weather," carrying the transmission identifier "07." Therefore, the transmission identifiers from Monday's weather data to Sunday's weather data can be determined. Then, the transmission identifiers can be... Based on actual application requirements, the transmission identifiers can be arranged in ascending or descending order. For example, arranging the transmission identifiers from Monday's weather data to Sunday's weather data in ascending order yields a data transmission sequence of "01; 02; 03; 04; 05; 06; 07". Then, according to the seven descending transmission identifiers in the data transmission sequence, Monday's weather data can be sent to the storage address Actor1 corresponding to transmission identifier "01". This allows Actor1 to transfer Monday's weather data to the storage resources created by the server for storage. When the server finishes storing Monday's weather data, it can send data to Actor1. The Sharding module sends a notification that Monday's meteorological data storage is complete. Then, the Actor Sharding module takes Tuesday's meteorological data (which is in a waiting state) and, according to the seven descending order of the data transmission sequence, sends Tuesday's meteorological data to the storage address Actor2 corresponding to the transmission identifier "02". This allows Actor2 to transfer Tuesday's meteorological data to the storage resources created by the server for storage. Similarly, the remaining five data shards are sequentially transferred to the storage resources created by the server through the storage addresses corresponding to their transmission identifiers. This ensures accurate and smooth storage of all seven data shards, thereby avoiding resource waste to some extent.

[0124] Furthermore, since each data fragment does not contain a transmission identifier in its data tag, the data fragments can be randomly sent to the server for storage. For example, when the Actor Sharding module receives a ready status notification from the server, it can send the data fragments to the corresponding servers for storage in the order in which the ready status notifications were received. In practical applications, the method of sending K data fragments to the server can be set according to the actual application requirements and is not limited to any specific method.

[0125] Optionally, in the above Figure 2Based on the corresponding embodiments, in another optional embodiment of the data storage method provided in this application, such as... Figure 5 As shown, the method also includes:

[0126] In step S501, the server sequentially sends K segments of storage progress, where one data fragment corresponds to one segment of storage progress.

[0127] In step S502, K segments of storage progress are sent to the client sequentially according to the data transmission sequence so that the client can display the K segments of storage progress.

[0128] In this embodiment, after the server sequentially completes the storage of K categories of data, it can receive K segments of storage progress sent sequentially by the server. Each segment of storage progress is used to indicate the current storage progress of each category of data. Then, the Actor Sharding module can send the K segments of storage progress to the client in sequence according to the data transmission sequence, so that the client can display the K segments of storage progress. The storage status of each category of data can be intuitively displayed on the client's application interface through the K segments of storage progress, so that the user can perceive in real time and accurately which category of data has been stored and perceive the current storage progress of that category of data.

[0129] Specifically, when the server stores K categories of data sequentially, it can send the current storage progress and total storage progress of each category of data to the client via the MiddleSocket socket in the form of a message structure as shown in Figure 10(c) upon completion of storage. This will display the current storage progress and total storage progress of the category of data on the client's display interface. Similarly, the current storage progress and total storage progress of the other K-1 categories of data can be sent to the client sequentially to display the current storage progress and total storage progress of the K categories of data on the client's display interface.

[0130] For ease of understanding, see Figure 10(c) and Figure 11As shown in the example, this application uses K data segments representing meteorological data from Monday to Sunday. When storing Monday's meteorological data, the current storage progress of Monday's data is set to "14.3%" or "1 / 7", and the total storage progress is also set to "14.3%" or "1 / 7". This is converted into a message structure as shown in Figure 10(c). This structure yields the event name "Monday's Meteorological Data", the current progress as "14.3%", the total progress as "14.3%", and the data to be transmitted as empty. Furthermore, using a MiddleSocket socket, the current storage progress of Monday's meteorological data ("14.3%) and the total storage progress ("14.3%) can be displayed. The current storage progress of Monday's weather data is "14.3%" and the total storage progress is "14.3%" when sent to the client's display interface. Similarly, the current storage progress and total storage progress of the other 6 weather data can be sent to the client in sequence, so that the current storage progress of Monday's weather data is "14.3%" and the total storage progress is "14.3%" when sent to the client's display interface, the current storage progress of Tuesday's weather data is "14.3%" and the total storage progress is "28.6%", and so on, until the current storage progress of Sunday's weather data is "14.3%" and the total storage progress is "100%".

[0131] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the data storage method provided in this application, such as... Figure 6 As shown, the data to be transmitted is obtained, including:

[0132] In step S601, a data upload request sent by the client is received, wherein the data upload request carries the target data type;

[0133] In step S602, respond to the data upload request and obtain the data to be processed;

[0134] In step S603, the data to be processed is converted according to the target data type to obtain the data to be transmitted.

[0135] In this embodiment, when a user wants to upload data to the server for storage, the user can generate a data upload request through the client's application interface. When the server's Actor model receives the data upload request sent by the client, it can first determine whether the data to be processed carried in the data upload request is data supported by the Actor model. If so, the Actor model responds to the data upload request and obtains the data to be processed. Then, according to the target data type, the data to be processed can be converted into data to be transmitted, so that the large amount of data to be transmitted can be divided into K smaller data fragments for distributed transmission, thereby reducing the data storage load to a certain extent.

[0136] Specifically, such as Figure 9 As shown, users can upload data through the client's application interface and generate a data upload request according to the message structure shown in Figure 10(c). Before transmitting the data upload request through the network layer, the network layer can authenticate the data upload request. Authentication determines whether the user or the data to be processed carried in the data upload request has the permission to access the server. Specifically, it can be through authentication technologies such as passwords or credentials, or other authentication technologies. No specific restrictions are made here.

[0137] Furthermore, after authenticating the data upload request at the network layer, the data upload request can be uploaded to, for example, […]. Figure 9 In the top-level Actor model shown, the top-level Actor model can respond to the data upload request and obtain the data to be processed in the data upload request, which is of time type or file type. Then, the data to be processed is converted into data to be transmitted according to the data upload structure shown in Figure 10(a).

[0138] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the data storage method provided in this application, such as... Figure 7 As shown, based on the target data type, the data to be processed is transformed to obtain the data to be transmitted, including:

[0139] In step S701, if the target data type is time type, the data to be processed is converted into data to be transmitted in time-data value format;

[0140] In step S702, if the target data type is a file type, the data to be processed is converted into data to be transmitted in file name-file object format.

[0141] In this embodiment, after obtaining the data to be processed, the type of the data upload structure can be determined according to the target data type of the data to be processed to perform data transformation on the data to be processed. This enables the Actor model to better and faster identify or respond to the data to be transmitted, and facilitates the Actor Sharding module to process the data to be transmitted better and faster.

[0142] Specifically, as shown in Figure 10(a), when the data to be processed is obtained, if the target data type of the data to be processed is time type, then according to the time data structure shown in Figure 10(a), the data to be processed is converted into time data sequence-data value sequence data to be transmitted in the time data sequence format; if the target data type of the data to be processed is file type, then according to the file data structure shown in Figure 10(a), the data to be processed is converted into file-file object data to be transmitted in the file name-file object format.

[0143] For ease of understanding, as shown in Figure 10(a), this embodiment of the application takes the meteorological data from Monday to Sunday as an example to illustrate the data to be processed. When the meteorological data from Monday to Sunday is obtained, the meteorological data from Monday to Sunday can be transmitted as a time data sequence-data value sequence according to the time data structure shown in Figure 10(a), such as "Monday" - "the highest temperature is 32 degrees, the lowest temperature is 21 degrees, and the average temperature is 27 degrees".

[0144] Optionally, in the above Figure 2 Based on the corresponding embodiments, in another optional embodiment of the data storage method provided in this application, such as... Figure 8 As shown, the method also includes:

[0145] In step S801, a first read request sent by the client is received, wherein the first read request carries the data read range;

[0146] In step S802, in response to the first read request, T data tags are determined according to the data read range, where T is an integer greater than or equal to 1 and less than or equal to K;

[0147] In step S803, the storage address corresponding to each data tag in the T data tags is determined based on the historical storage records;

[0148] In step S804, a second read request is sent to the server according to the storage address corresponding to each data tag, so that the server responds to the second read request and obtains T fragments of data;

[0149] In step S805, T data fragments are received from the server.

[0150] In this embodiment, when a user wants to retrieve stored data from the server, the user can generate a first read request by performing a data read operation through the client's application interface. When the server's Actor model receives the first read request sent by the client carrying the data read range, it can first determine whether the user has permission to access the server. If so, the Actor model responds to the first read request and determines T data tags based on the data read range. Then, based on the mapping relationship between data tags and storage addresses in historical storage records, it determines the storage address corresponding to each of the T data tags. Subsequently, based on the storage address corresponding to each data tag, it sends a second read request to the corresponding server, so that the server responds to the second read request, retrieves T fragments of data, receives the T fragments of data sent by the server through the corresponding Actor, and then sends the T fragments of data to the client.

[0151] Specifically, such as Figure 9 As shown, users can perform data acquisition operations through the client application interface and generate a first read request according to the message structure shown in Figure 10(c). Before transmitting the first read request through the network layer, the network layer can authenticate the data upload request. After the network layer authenticates the first read request, the first read request can be uploaded to, for example, Figure 9 In the top-level Actor model shown, the top-level Actor model can respond to the first read request and obtain the data read range carried in the first request. Then, the data read range is transmitted to the Actor Sharding module according to the data acquisition structure shown in Figure 10(b). For example, if the data read range is weather data from Monday to Sunday, it is converted into a data read range in the format of data start time-data end time according to the time data structure shown in Figure 10(b). If the data read range is a sales report of a product, it is converted into a data read range in the format of file name-file unique identifier-file type according to the file data structure shown in Figure 10(b).

[0152] Furthermore, the Actor Sharding module can perform data sharding on the data reading range, and then assign T data tags based on the sharding results. This allows the system to find data tags that match the T data tags in the server's historical storage records. Based on the mapping relationship between data tags and storage addresses, the system determines the storage address corresponding to each of the T data tags.

[0153] Furthermore, a second read request can be sent to the corresponding server based on the storage address corresponding to each data tag, so that the server responds to the second read request and can obtain T fragments of data from the storage resources respectively, and receive the T fragments of data sent by the server through the Actor corresponding to the storage address of each data tag, and then send the T fragments of data to the client.

[0154] For ease of understanding, as shown in Figures 10(b) and 10(c), this embodiment of the application uses T fragments of meteorological data from Monday to Sunday as an example. When the first read request sent by the client is received, the top-level Actor model can respond to the first read request and obtain the data read range carried in the first request. Then, the data read range is converted into the format "data start time: Monday - data end time: Sunday" according to the time data structure shown in Figure 10(b), and transmitted to the Actor Sharding module. The Actor Sharding module can then perform fragment processing on "data start time: Monday - data end time: Sunday" to obtain 7 data tags, such as Monday weather or Sunday weather. Then, the data tags that match these 7 data tags can be found in the historical storage records of the server. Then, according to the mapping relationship between data tags and storage addresses, the storage address corresponding to each of the 7 data tags is determined. For example, the storage address corresponding to the data tag "Monday weather" is Actor1.

[0155] Furthermore, based on the storage address corresponding to each data tag, for example, according to the storage address corresponding to Monday's weather as Actor1, a second read request in the form of a message structure as shown in Figure 10(c) can be sent to the server so that the server responds to the second read request and can obtain 7 data fragments from the storage resources respectively. For example, by receiving the Monday weather data sent by the server through Actor1 corresponding to the storage address of Monday's weather, the Monday weather data can be sent to the client. Similarly, the other 6 weather data can be sent to the client.

[0156] Furthermore, when the server responds to the second read request, it can sequentially retrieve T data fragments from the storage resource according to the order of the transmission identifiers carried by the T data tags. The current read progress and total read progress of the retrieved data fragments are then sequentially transmitted and loaded to the client using a message structure as shown in Figure 10(c), so that the client's display interface shows the retrieved data fragments, their current read progress, and the total read progress. Alternatively, the data fragments can be retrieved from the storage resource according to the order in which the server responds to the second read request. In practical applications, the method of retrieving data fragments can be set according to actual application requirements and is not limited to any specific method.

[0157] The data storage apparatus in this application is described in detail below. Please refer to [link / reference]. Figure 12 , Figure 12 This is a schematic diagram of one embodiment of a data storage device in this application. The data storage device 20 includes:

[0158] The acquisition unit 201 is used to acquire data to be transmitted, wherein the data to be transmitted carries a target data type, which is a time type or a file type;

[0159] The processing unit 202 is used to perform fragmentation processing on the data to be transmitted according to the target data type, to obtain K fragmented data and the actual storage amount corresponding to each fragmented data, wherein each fragmented data corresponds to a data tag, and K is an integer greater than 1;

[0160] The processing unit 202 is also configured to allocate a preset storage amount to each data segment based on the actual storage amount corresponding to each data segment, wherein the preset storage amount is greater than or equal to the corresponding actual storage amount;

[0161] The determining unit 203 is used to determine the storage address of each data segment if a data tag for each data segment exists in the historical storage record, wherein the historical storage record includes the mapping relationship between data tags and storage addresses within a historical time period;

[0162] The processing unit 202 is further configured to send a resource creation request to the server according to the storage address of each data segment, so that the server creates storage resources matching the preset storage amount for each data segment, wherein the resource creation request carries the preset storage amount of the data segment;

[0163] The processing unit 202 is also used to send K fragments of data to the server so that the server can store the K fragments of data.

[0164] Optionally, in the above Figure 12 Based on the corresponding embodiments, in another embodiment of the data storage device provided in this application,

[0165] The processing unit 202 is also configured to send an address allocation request to the server if there is no data tag for the fragmented data in the historical storage record, so that the server allocates a corresponding storage address for the fragmented data according to the address allocation request;

[0166] The receiving unit 204 is used to receive the address allocation response sent by the server, wherein the address allocation response carries the storage address allocated for the fragmented data.

[0167] Optionally, in the above Figure 12 Based on the corresponding embodiments, in another embodiment of the data storage device provided in this application, the processing unit 202 may specifically be used for:

[0168] Based on the data tag of each data fragment, determine the transmission identifier of each data fragment, where the transmission identifier is used to indicate the transmission order of the data fragments;

[0169] The transmission identifiers of each data fragment are arranged to obtain a data transmission sequence, wherein the data transmission sequence includes K transmission identifiers arranged in descending or ascending order;

[0170] K data fragments are sent to the server sequentially according to the data transmission sequence.

[0171] Optionally, in the above Figure 12 Based on the corresponding embodiments, in another embodiment of the data storage device provided in this application,

[0172] The receiving unit 204 is also used to receive K segments of storage progress sent sequentially by the server, wherein one segment of data corresponds to one segment of storage progress;

[0173] The processing unit 202 is also configured to send K segments of storage progress to the client sequentially according to the data transmission sequence, so that the client can display the K segments of storage progress.

[0174] Optionally, in the above Figure 12 Based on the corresponding embodiments, in another embodiment of the data storage device provided in this application, the acquisition unit 201 may specifically be used for:

[0175] Receive data upload requests sent by clients, where the data upload requests carry the target data type;

[0176] Respond to data upload requests and retrieve the data to be processed;

[0177] Based on the target data type, the data to be processed is transformed to obtain the data to be transmitted.

[0178] Optionally, in the above Figure 12Based on the corresponding embodiments, in another embodiment of the data storage device provided in this application, the acquisition unit 201 may specifically be used for:

[0179] If the target data type is time, the data to be processed will be converted into time-data-value format data to be transmitted.

[0180] If the target data type is a file type, the data to be processed will be converted into data to be transmitted in filename-file object format.

[0181] Optionally, in the above Figure 12 Based on the corresponding embodiments, in another embodiment of the data storage device provided in this application, the receiving unit is further configured to receive a first read request sent by the client, wherein the first read request carries a data read range;

[0182] The determining unit 203 is also used to respond to the first read request and determine T data tags according to the data read range, where T is an integer greater than or equal to 1 and less than or equal to K;

[0183] The determining unit 203 is also used to determine the storage address corresponding to each data tag among the T data tags based on the historical storage records;

[0184] The processing unit 202 is also configured to send a second read request to the server according to the storage address corresponding to each data tag, so that the server responds to the second read request and obtains T fragments of data;

[0185] The receiving unit 204 is also used to receive T fragments of data sent by the server.

[0186] This application also provides a schematic diagram of another computer device, such as... Figure 13 As shown, Figure 13 This is a schematic diagram of a computer device structure provided in an embodiment of this application. The computer device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 331 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the computer device 300. Furthermore, the CPU 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the computer device 300.

[0187] Computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 333, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0188] The aforementioned computer device 300 is also used to perform, for example Figures 2 to 8 The steps in the corresponding embodiments.

[0189] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform actions such as... Figures 2 to 8 The steps in the method described in the illustrated embodiment.

[0190] Another aspect of this application provides a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform actions such as Figures 2 to 8 The steps in the method described in the illustrated embodiment.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0193] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0194] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0195] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for data storage, characterized in that, The method is applied to a system architecture including a client, a top-level Actor model, an Actor sharding module, at least one Actor, and at least one data server. The client interacts with the top-level Actor model through a network layer, the top-level Actor model interacts directly with the Actor sharding module, the Actor sharding module interacts with the at least one Actor through a network layer, and there is a one-to-one correspondence between the at least one Actor and the at least one data server. Different data servers interact with each other through their corresponding Actors. The top-level Actor model receives data upload requests sent by the client, wherein the data upload request carries a target data type; the target data type is either a time type or a file type. Respond to the data upload request and obtain the data to be processed; If the target data type is the time type, then according to the time data structure, the data to be processed is converted into data to be transmitted as a [time data sequence - data value sequence] according to the [time-data value] format; If the target data type is the file type, then according to the file data structure, the data to be processed is converted into data to be transmitted in the format of [file name - file object]. The top-level Actor model transmits the data to be transmitted to the Actor sharding module. Based on the target data type, the data to be transmitted is sharded to obtain K shards and the actual storage amount corresponding to each shard. Each shard corresponds to a data tag and an Actor. All messages of each shard are processed by the corresponding Actor. K is an integer greater than 1. The Actor sharding module allocates a preset storage amount to each shard of data based on the actual storage amount corresponding to each shard, wherein the preset storage amount is greater than or equal to the corresponding actual storage amount; If the Actor sharding module finds that the data tag for each shard exists in the historical storage record, then the storage address for each shard is determined, wherein the storage address is the Actor address, and the historical storage record includes the mapping relationship between the data tag and the storage address within a historical time period; The Actor sharding module sends a resource creation request to the server based on the storage address of each shard of data. This enables the server to create storage resources matching the preset storage amount for each shard of data and generate a corresponding Actor for each shard of data. The resource creation request carries the preset storage amount of the shard data. If a generated Actor does not work for a period of time, it indicates that the Actor has timed out and is automatically destroyed. Each data shard is sent to the corresponding Actor through the Actor sharding module, and the corresponding Actor sends the K data shards to the server so that the server can store the K data shards; if the target data type is time type, the K data shards are divided according to the time dimension and stored on different servers according to the time characteristics of the target data. When a new node is added to the storage server cluster, the new Actor is automatically created on the new node, or the old Actor is migrated to the new node through load balancing; when a node fails, the failed Actor is migrated to a new node. When the storage server cluster is idle, the data stored on the nodes is backed up according to the backup strategy of the storage server cluster; the backup strategy includes full node backup, or [number of nodes / 2+1] backup; different data servers back up data through Actor communication, so that if any data server fails, the backed-up data can be obtained from other data servers that have backed up data with the failed data server through Actor communication and can continue to be used.

2. The method according to claim 1, characterized in that, The method further includes: If there is no data tag for the fragmented data in the historical storage record, an address allocation request is sent to the server so that the server allocates a corresponding storage address for the fragmented data according to the address allocation request. Receive an address allocation response sent by the server, wherein the address allocation response carries the storage address allocated for the fragmented data.

3. The method according to claim 1, characterized in that, Sending the K data fragments to the server includes: Based on the data tag of each data segment, a transmission identifier for each data segment is determined, wherein the transmission identifier is used to indicate the transmission order of the data segments; The transmission identifiers of each data fragment are arranged to obtain a data transmission sequence, wherein the data transmission sequence includes K transmission identifiers arranged in descending or ascending order; The K data fragments are sent to the server sequentially according to the data transmission sequence.

4. The method according to claim 3, characterized in that, The method further includes: Receive K segments of storage progress sent sequentially by the server, wherein one data shard corresponds to one segment of storage progress; The K segments of storage progress are sent to the client sequentially according to the data transmission sequence, so that the client can display the K segments of storage progress.

5. The method according to claim 1, characterized in that, After sending the K data fragments to the server, the method further includes: Receive a first read request sent by the client, wherein the first read request carries a data read range; In response to the first read request, T data tags are determined according to the data read range, wherein T is an integer greater than or equal to 1 and less than or equal to K; Based on the historical storage records, determine the storage address corresponding to each of the T data tags; Based on the storage address corresponding to each data tag, a second read request is sent to the server so that the server responds to the second read request and obtains T data fragments; Receive the T data fragments sent by the server.

6. A data storage device, characterized in that, The device is applied to a system architecture including a client, a top-level Actor model, an Actor sharding module, at least one Actor, and at least one data server. The client interacts with the top-level Actor model through a network layer, the top-level Actor model interacts directly with the Actor sharding module, the Actor sharding module interacts with the at least one Actor through a network layer, and there is a one-to-one correspondence between the at least one Actor and the at least one data server. Different data servers interact with each other through their corresponding Actors. The device includes: The acquisition unit is configured to receive a data upload request sent by a client through a top-level Actor model, wherein the data upload request carries a target data type; the target data type is a time type or a file type; respond to the data upload request and acquire the data to be processed; if the target data type is the time type, then according to the time data structure, the data to be processed is converted into data to be transmitted in the format of [time data sequence - data value sequence]; if the target data type is the file type, then according to the file data structure, the data to be processed is converted into data to be transmitted in the format of [file name - file object]; The processing unit is used to pass the data to be transmitted to the Actor sharding module through the top-level Actor model, and to shard the data to be transmitted according to the target data type to obtain K shard data and the actual storage amount corresponding to each shard data. Each shard data corresponds to a data tag and an Actor. All messages of each shard data are processed by the corresponding Actor. K is an integer greater than 1. The processing unit is further configured to allocate a preset storage amount to each data segment based on the actual storage amount corresponding to each data segment through the Actor sharding module, wherein the preset storage amount is greater than or equal to the corresponding actual storage amount; The determining unit is configured to determine the storage address of each data segment if the data tag of each data segment is found in the historical storage record through the Actor sharding module. The storage address is the Actor address, wherein the historical storage record includes the mapping relationship between data tags and storage addresses within a historical time period. The processing unit is further configured to send a resource creation request to the server according to the storage address of each data shard through the Actor sharding module, so that the server creates storage resources matching the preset storage amount for each data shard, and generates a corresponding Actor for each data shard; wherein the resource creation request carries the preset storage amount of the data shard; when the generated Actor does not work for a period of time, it indicates that the Actor has timed out and is automatically destroyed; The processing unit is further configured to send each data shard to the corresponding Actor through the Actor sharding module, and send the K data shards to the server through the corresponding Actor, so that the server stores the K data shards; if the target data type is time type, the K data shards are divided according to the time dimension and stored on different servers according to the time characteristics of the target data. The device is also used for: When a new node is added to the storage server cluster, the new Actor is automatically created on the new node, or the old Actor is migrated to the new node through load balancing; when a node fails, the failed Actor is migrated to a new node. When the storage server cluster is idle, the data stored on the nodes is backed up according to the backup strategy of the storage server cluster; the backup strategy includes full node backup, or [number of nodes / 2+1] backup; different data servers back up data through Actor communication, so that if any data server fails, the backed-up data can be obtained from other data servers that have backed up data with the failed data server through Actor communication and can continue to be used.

7. A computer device, characterized in that, include: Memory, transceiver, processor, and bus system; The memory is used to store programs; When the processor is used to execute a program in the memory, it implements the method as described in any one of claims 1 to 5; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

8. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 5.

Images