Data writing method for multiple availability zones, server, system, medium, and product

Abstract

Embodiments of the present disclosure provide a data writing method for multiple availability zones, a server, a system, a medium, and a product. In a data storage method, when different service nodes are distributed in different availability zones, after data to be written respectively corresponding to at least one data packet is obtained, service nodes respectively corresponding to the at least one data packet can be determined on the basis of the management allocation relationship between data packets and service nodes. Availability zones respectively corresponding to the at least one data packet can be determined on the basis of the availability zones to which the service nodes respectively corresponding to the at least one data packet belong. Then, the data to be written respectively corresponding to the at least one data packet may be respectively written into corresponding availability zones. Further, for any data packet, the service node and data corresponding to the data packet may be distributed in a same availability zone. When reading data of a data packet managed by the service node, the service node does not need to perform a data reading operation across availability zones, thereby reducing traffic bandwidth overhead.

Description

[0001]This disclosure claims priority to Chinese Patent Application No. 202410805705.4, filed on June 20, 2024, entitled "Data Writing Method, Server, System, Media, and Product for Multiple Availability Zones," the entire contents of which are incorporated herein by reference. Technical Field This disclosure relates to the field of computer technology, and more particularly to a data writing method, server, system, media, and product for multiple availability zones. Background Art In distributed systems, for the needs of management performance and hotspot dispersion, user data is often divided into multiple data groups (partitions) according to multiple dimensions such as user or file directory. Multiple data groups can be managed by multiple foreground service processes in the distributed system. Each foreground service process is responsible for managing one or more data groups. To improve data storage reliability, some distributed storage systems support data storage in multiple Availability Zones (AZs), where each AZ is an independently managed physical data center. Each AZ has one or more foreground service processes. When a user sends a request to a foreground service process to read data, the foreground service process determines the data node corresponding to the data, reads the data block from that node, and returns it to the user. In multi-AZ storage systems, the foreground service processes of the distributed system are often distributed across different AZs. When data in multiple data groups is distributed across data nodes in multiple AZs, the foreground service process of any data group may access data nodes in other AZs when reading data. This cross-AZ data reading method incurs significant bandwidth overhead. Therefore, a new solution is needed. The present invention discloses a multi-Availability Zone data writing method, server, system, medium, and product to reduce the probability of cross-Availability Zone data reading in distributed systems and reduce bandwidth overhead.This disclosure provides a multi-availability zone data writing method, comprising: responding to a data write request and obtaining data to be written corresponding to at least one data group; determining the service node corresponding to each of the at least one data group based on the management allocation relationship between the data group and the service node; determining the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs; and writing the data to be written corresponding to each of the at least one data group into the availability zone corresponding to each of the at least one data group, so that the service node corresponding to any data group can read the data already written corresponding to the data group in its own availability zone. Optionally, writing the data to be written corresponding to each of the at least one data group into the availability zone corresponding to each of the at least one data group includes: splitting the data to be written corresponding to each of the at least one data group to obtain data blocks corresponding to each of the at least one data group; and writing the data blocks corresponding to each of the at least one data group into the availability zone corresponding to each of the at least one data group. Optionally, the method further includes: aggregating the data blocks corresponding to each of the at least one data group to obtain multiple data blocks; encoding the multiple data blocks to obtain multiple checksums; and storing the multiple checksums in a target availability zone other than the availability zone corresponding to each of the at least one data group. Optionally, writing the data blocks corresponding to each of the at least one data group into the availability zone corresponding to each of the at least one data group includes: for any first data group among the at least one data group, distributing the data blocks corresponding to the first data group into multiple data nodes in the availability zone corresponding to the first data group. Optionally, the method further includes: encoding the data blocks corresponding to the first data group to obtain at least one checksum corresponding to the first data group; and storing the at least one checksum in a target data node in the availability zone where the first data group is located. Optionally, the target data node is another data node in the availability zone where the first data group is located, excluding the data node used to store the data blocks corresponding to the first data group. Optionally, determining the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs includes: for any second data group among the at least one data group, querying metadata information in the distributed system based on the identifier of the service node corresponding to the second data group to obtain the availability zone to which the service node corresponding to the second data group belongs; and using the availability zone to which the service node corresponding to the second data group belongs as the availability zone corresponding to the second data group.This disclosure also provides a distributed system, including: a data writing component and multiple availability zones, each availability zone including a service node and multiple data nodes; wherein, the service node in the multiple availability zones is used to manage data read and write operations of at least one data group; the data writing component is used to: respond to a data write request and obtain the data to be written corresponding to each of the at least one data group; divide the data to be written corresponding to each of the at least one data group to obtain data blocks corresponding to each of the at least one data group; determine the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs; and write the data blocks corresponding to each of the at least one data group into the data nodes in the availability zone corresponding to each of the at least one data group, so that the service node corresponding to any data group can read the written data corresponding to the data group in its own availability zone. This disclosure also provides a server, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: perform the steps in the method provided in this disclosure. This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps of the method provided in this disclosure. This disclosure also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps of the method provided in this disclosure. In the multi-availability zone data writing method provided in this disclosure, when different service nodes are distributed in different availability zones, after obtaining the data to be written corresponding to at least one data group, the service node corresponding to each of the at least one data group can be determined according to the management and allocation relationship between the data group and the service node. Based on the availability zone to which the service node corresponding to each of the at least one data group belongs, the availability zone corresponding to each of the at least one data group can be determined. Then, the data to be written corresponding to each of the at least one data group can be written to the corresponding availability zone respectively. Furthermore, for any data group, the service node and data corresponding to the data group can be distributed in the same availability zone. When a service node reads the data of the data group it manages, it does not need to perform cross-availability zone data reading operations, reducing traffic bandwidth overhead and thus saving data reading costs. The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure.In the accompanying drawings: Figure 1 is a schematic diagram of the data blocks corresponding to the cross-availability zone storage data group; Figure 2 is a schematic diagram of the structure of a distributed system provided by an exemplary embodiment of this disclosure; Figure 3 is a flowchart of a data writing method for multiple availability zones provided by an exemplary embodiment of this disclosure; Figure 4 is a schematic diagram of the structure of a server provided by an exemplary embodiment of this disclosure. Detailed Description: To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. The terminology used in the embodiments of this disclosure is for the purpose of describing specific embodiments only, and is not intended to limit this disclosure. The singular forms "a," "the," and "the" used in the embodiments of this disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise; "multiple" generally includes at least two, but does not exclude the inclusion of at least one. It should be understood that the term "and / or" used in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. It should also be noted that the terms "including," "comprise," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system that includes a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such a product or system. Without further restrictions, an element defined by the phrase "including a..." does not exclude the existence of other identical elements in the product or system that includes said element. In distributed systems, for the sake of management performance and hotspot dispersion, user data is often divided into multiple data groups (Partitions) according to multiple dimensions such as user or file directory. These multiple data groups can be managed by multiple foreground service processes in the distributed system. Each foreground service process is responsible for managing one or more data groups. To improve the reliability of data storage, some distributed storage systems support data storage in multiple Availability Zones (AZs), where each AZ is an independently managed physical data center.In a multi-AZ storage system, each Availability Zone (AZ) has one or more foreground service processes. When a user sends a request to a foreground service process to read data, the foreground service process can determine the data node corresponding to the data, read the data block from that data node, and return it to the user. In a multi-AZ storage system, the foreground service processes of the distributed system are often distributed across different AZs. When data from multiple data groups is distributed across data nodes in multiple AZs, the foreground service process of any data group may access data nodes in other AZs when reading data. Taking a 3AZ system as an example, the foreground service process in any AZ has a 2 / 3 probability of reading data nodes in other AZs when performing data reading. This cross-AZ data reading method leads to significant bandwidth overhead. As shown in Figure 1, multiple data blocks A1-A5 of data A are stored in AZ1, data blocks A6-A10 are stored in AZ2, and check blocks X1-X7 are stored in AZ3. When a foreground service process in AZ1 reads a data block corresponding to data A, there is a probability that it will read a data block from AZ2 and a parity block from AZ3, which can easily lead to cross-availability zone traffic overhead. To address the above technical problem, this disclosure provides a solution, the core of which is: aggregating data write operations from multiple data groups located in different AZs, and writing the data corresponding to different data groups to the AZ where the foreground service process corresponding to the data group is located, so that the data node where the data in the data group is actually stored is located in the same AZ as its corresponding foreground service process, thereby reducing the probability of the foreground service process performing data read operations across AZs and reducing cross-AZ traffic bandwidth in a multi-AZ distributed storage system. The technical solutions provided by various embodiments of this disclosure are described in detail below with reference to the accompanying drawings. Figure 2 is a schematic diagram of the structure of a distributed system provided by an exemplary embodiment of this disclosure. The distributed system 100 may include a storage cluster deployed across multiple availability zones, which may include two or more availability zones. As shown in Figure 2, the storage cluster may be deployed across a first availability zone 10, a second availability zone 20, and a third availability zone 30. Each availability zone may include one service node and multiple data nodes. As shown in Figure 2, the first availability zone 10 may include a first service node 101, a first data node 102, and a second data node 103, and the second availability zone 20 may include a second service node 201, a third data node 202, and a fourth data node 203.It should be understood that the number of availability zones and the number of service nodes and data nodes in each availability zone shown in Figure 2 are for illustrative purposes only and do not constitute a limitation on the number of availability zones, service nodes, and data nodes. In the distributed system 100, any service node can be implemented as a server or container instance running a foreground service process. This foreground service process is a system component that directly interacts with the user or executes user-initiated tasks and is the user-facing interface of the distributed system 100. The foreground service process can receive input from the user and is responsible for handling user requests, executing data processing logic, computing tasks, transaction processing, and other operations. In some scenarios, users can write data through the foreground service process, and this data can be persistently written to data nodes in the availability zone. In this implementation, the target data to be written is divided into at least one data group to facilitate the management of the target data. In some optional embodiments, the target data can be divided into at least one data group based on the user, file directory, or other logical dimensions to which the data belongs. Examples will be given below. In some scenarios, data can be grouped based on the first letter of the username of the user to which the data belongs. For example, in large-scale email service systems, user mailbox data can be segmented according to the first letter of the username. For instance, all mailboxes whose usernames begin with "AM" are stored in one group (or partition), while those beginning with "NZ" are stored in another. When accessing a specific user's emails, the email service system can quickly locate the corresponding data group based on the first letter of the username, thus reducing query time. In some scenarios, data can be grouped based on geographic location. For example, in video streaming services, video content can be grouped according to the user's geographic location. For instance, live stream data uploaded by broadcasters in region R1 is divided into one data group, and live stream data uploaded by broadcasters in region R2 is divided into another data group. Furthermore, when different users access live stream data, they can access different data groups based on their region, thereby reducing network latency. In some scenarios, data can be grouped based on file type. For example, in cloud storage services, data can be grouped and stored according to file type (such as images, documents, videos, etc.). Furthermore, this not only facilitates the implementation of different storage optimization strategies (such as compression and transcoding) based on file type, but also enables quick location of the correct data group when a user requests a specific type of file, improving retrieval efficiency. In the distributed system 100, service nodes in multiple availability zones can be used to provide management services related to data read and write operations for at least one data group.Within an availability zone, a service node can manage data read and write operations for one or more data groups. For example, the target data may be divided into data group P1, data group P2, and data group P3. OIn this embodiment, data packet P1 can be managed by service node A, and data packets P2 and P3 can be managed by service node B. Service node A can run in the first availability zone 10, and service node B can run in the second availability zone 20. In this embodiment, when a service node is assigned to manage the read and write operations of a certain data packet, the relationship between the service node and the data packet can be described as a management allocation relationship. For example, service node B has a management allocation relationship with data packets P2 and P3, and data packet A has a management allocation relationship with data packet P1. As shown in Figure 2, the distributed system 100 also includes a data writing component 40. The data writing component 40 can communicate and interact with service nodes and data nodes in multiple availability zones to read data packets from service nodes and write the data in the data packets to the corresponding data nodes according to a set strategy. In this embodiment, the set strategy can be: a strategy of selecting data nodes for data packets based on the availability zone to which the service node corresponding to the data packet belongs. This will be explained in detail below. In the distributed system 100, the data writing component 40 is mainly used to: respond to data writing requests, obtain the data to be written corresponding to at least one data group, and determine the service node corresponding to each of the at least one data group based on the management and allocation relationship between the data group and the service node. The data writing request can be issued by any service node, or by a client, or by a timed event; this embodiment does not impose any restrictions. The management and allocation relationship between the data group and the service node refers to the relationship formed when a service node is assigned to manage the data read and write operations corresponding to a data group; details can be found in the foregoing description. This management and allocation relationship between the data group and the service node can be established by the load balancing component or service scheduling component in the distributed system 100, or by the user through a client; this embodiment does not impose any restrictions. In this embodiment, after determining the service node corresponding to each of the at least one data group, the data writing component 40 can determine the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs. Taking any data packet as an example, the data writing component 40 can determine the availability zone corresponding to the data packet based on the availability zone to which the service node corresponding to the data packet belongs. After determining the availability zone corresponding to each of the at least one data packet, the data to be written corresponding to each of the at least one data packet can be written to the respective availability zone of the at least one data packet. Therefore, the data corresponding to any data packet and the service node corresponding to the data packet can be distributed in the same availability zone.For any service node, when reading data corresponding to a data group it manages, the data read operation can be performed within its own availability zone without crossing availability zones, reducing the traffic overhead of data reading. In some optional embodiments, the data in the data group can be written to the availability zone in the form of data blocks. Specifically, the data writing component 40 can divide the data to be written corresponding to each of the at least one data group to obtain data blocks corresponding to each of the at least one data group, and write the data blocks corresponding to each of the at least one data group to the availability zone corresponding to each of the at least one data group. The size of the data block can be fixed. For example, continuing with data groups P1, P2, and P3, data group P1 can be managed by service node A, and data groups P2 and P3 can be managed by service node B. After dividing data group P1 into multiple data blocks, since service node A runs in the first availability zone 10, the data writing component 40 can store the multiple data blocks of data group P1 on the data nodes in the first availability zone 10. Accordingly, after dividing data packet P2 into multiple data blocks, since service node B runs in the second availability zone 20, the data writing component 40 can store the multiple data blocks of data packet P2 on data nodes in the second availability zone 20. Based on this implementation, the data to be written corresponding to any data packet can be stored in the corresponding availability zone in a block-based manner, so that different data blocks of the data packet can be processed in parallel, accelerating the data transmission and storage efficiency. In some optional embodiments, in the availability zone corresponding to any data packet, the data blocks corresponding to the data packet can be distributed and stored in multiple data nodes. Taking the first data packet in the at least one data packet as an example, the data blocks corresponding to the first data packet can be distributed and written to multiple data nodes in the availability zone corresponding to the first data packet. Here, the first data packet is any data packet in the at least one data packet. The term "first" is used here to describe the data packet only for easy distinction and does not impose any restrictions on the order or number of data packets. For example, when storing multiple data blocks of data group P1 in the first availability zone 10, data block A1 in data group P1 can be stored in data node C1 in the first availability zone, data block A2 can be stored in data node C2 in the first availability zone, and so on.Based on this implementation, data blocks in the first data group can be distributed across multiple data nodes using a decentralized storage model. Even if a single data node is compromised, the entire dataset cannot be easily corrupted, thus improving overall data security. Furthermore, with multiple data blocks distributed across multiple data nodes, write operations on multiple data blocks can be executed in parallel, significantly improving data processing speed and throughput. This is particularly beneficial in high-performance computing applications such as big data analytics and streaming media services, where high read / write throughput ensures service quality. In this embodiment, with different service nodes distributed across different availability zones, after obtaining the data to be written for each of at least one data group, the service node corresponding to each of the at least one data group can be determined based on the management and allocation relationship between the data group and the service node. Based on the availability zone to which the service node corresponding to each of the at least one data group belongs, the availability zone corresponding to each of the at least one data group can be determined. Then, the data to be written for each of the at least one data group can be written to its respective availability zone. Furthermore, for any data packet, the service node and data corresponding to the data packet can be distributed in the same availability zone. When the service node reads the data of the data packet it manages, it does not need to perform cross-availability zone data read operations, reducing traffic bandwidth overhead and thus saving data read costs. In some exemplary embodiments, to improve data reliability and fault tolerance between availability zones, the data writing component 40 can generate verification data for the data to be written corresponding to each of the at least one data packet, and store the verification data and the data to be written corresponding to each of the at least one data packet in different availability zones, so that data recovery can be performed based on other availability zones when some availability zones fail. The following will provide an exemplary description. Optionally, the data writing component 40 can aggregate multiple data blocks corresponding to each of the at least one data packet to obtain multiple data blocks, and encode the multiple data blocks to obtain multiple check codes. Optionally, when encoding the aggregated multiple data blocks, an erasure coding (EC) based encoding method can be used.Some optional erasure coding methods may include: Reed-Solomon Codes (RS codes), Reed-Solomon Over Large Scale (RS (255, 253)), Cauchy matrix coding, Low-Density Parity-Check Codes (LDPC), Polar Codes, etc., which are included but not limited to in this embodiment. Erasure coding-based methods can add extra redundant information to the original data, enabling the original data to be recovered using the remaining data and redundant information even if some data is lost or damaged. This greatly improves the fault tolerance of the distributed system and ensures the integrity and reliability of the data. After obtaining multiple check codes, the data writing component 40 can store these check codes in a target availability zone other than the availability zone corresponding to each of the at least one data group. Continuing with data group P1, Taking data packets P2 and P3 as examples, multiple data blocks of data packet P1 are stored in multiple data nodes in the first availability zone 10. Multiple data blocks of data packet P2 are stored in multiple data nodes in the second availability zone 20. Multiple data blocks of data packet P3 are stored in multiple data nodes in the second availability zone 20. The data blocks of data packets P1, P2, and P3 are aggregated to obtain multiple aggregated data blocks, denoted here as K data blocks. The data writing component 40 can generate M check blocks from the aggregated K data blocks, and then store the M check blocks in the third availability zone 30 as shown in Figure 2. Here, a check block refers to a storage block used to store check codes. After generating M check codes from K data blocks, the M check codes are stored in the form of blocks to obtain M check blocks. Therefore, when the first availability zone 10 fails, at least K blocks can be read from the data blocks stored in the second availability zone 20 and the check blocks stored in the third availability zone 30. It can recover data blocks stored in the first availability zone 10 based on K blocks. In the event of a failure in the second availability zone 20, at least K blocks can be read from the data blocks stored in the first availability zone 10 and the check blocks stored in the third availability zone 30, and the data blocks stored in the second availability zone 20 can be recovered based on the K blocks.Specifically, when the number of check blocks is greater than or equal to the number of aggregated data blocks (i.e., MNK), if both the first availability zone 10 and the second availability zone 20 fail, the data blocks stored in the first availability zone 10 and the second availability zone 20 can be recovered based on at least K check blocks stored in the third availability zone 30. The operation of generating M check blocks and storing M check blocks for K data blocks can be performed synchronously with the operation of storing K data blocks, or the storage operation of K data blocks can be performed first, followed by the generation and storage operation of M check blocks. This embodiment does not impose any limitations. Besides setting fault tolerance mechanisms between availability zones, in some exemplary embodiments, fault tolerance mechanisms can be set between different data nodes within an availability zone. After writing the data blocks of any data group into the availability zone, the data writing component 40 can generate verification data for multiple data blocks corresponding to the data group, and store the verification data and the data blocks corresponding to the data group in different data nodes in the availability zone. This allows for the recovery of the original data of the data group based on the remaining data blocks and the verification data when some data nodes in the availability zone fail. The following description will continue using the first data group as an example. Optionally, for the first data group, the data writing component 40 encodes the data blocks corresponding to the first data group to obtain at least one verification code corresponding to the first data group, and stores the at least one verification code in the target data node of the availability zone where the first data group is located. Optionally, when encoding the data blocks corresponding to any data group, an erasure coding-based encoding method can be used. In some optional embodiments, the target data node used to store the verification code can be a data node in the availability zone where the first data group is located, other than the data node used to store the data blocks corresponding to the first data group. Based on this implementation, data blocks and check blocks can be stored in isolation. When a physical failure occurs on the data node containing some data blocks, the check blocks on other data nodes remain secure, thus facilitating rapid data recovery and improving overall service stability. For example, continuing with data group P1, the k data blocks of data group P1 are stored in data nodes D1, D2, ..., Dk in the first availability zone 10. The data writing component 40 can encode the k data blocks in data group P1 to generate m check blocks. Then, the m check blocks can be stored in the target data nodes Dk+1, Dk+2......Dk+m in the first availability zone 10.It is worth noting that the operation of generating and storing m check blocks for k data blocks of any data group can be performed synchronously with the operation of storing the k data blocks corresponding to the data group. Alternatively, the storage operation of the k data blocks corresponding to the data group can be performed first, followed by the generation and storage of the m check blocks corresponding to the k data blocks. This embodiment does not impose any restrictions. Based on this implementation, within each availability zone, when the data node containing some data blocks of a data group fails, the original k data blocks can be recovered based on the remaining data blocks and at least k blocks from the check blocks, thereby achieving fault tolerance within the availability zone. In some exemplary embodiments, the distributed system 100 may also include a metadata node 50 as shown in Figure 2. The metadata node 50 is used to store the identifier of the data group and the correspondence between the service nodes. Accordingly, taking the second data group in the at least one data group as an example, when the data writing component 40 determines the availability zone corresponding to each of the second data groups based on the availability zone to which the service node corresponding to each of the second data groups belongs, it can query the metadata information in the metadata node 50 in the distributed system 100 according to the identifier of the second data group to obtain the availability zone to which the service node corresponding to the second data group belongs, and can use the availability zone to which the service node corresponding to the second data group belongs as the availability zone corresponding to the second data group. Thus, the data writing component 40 can accurately obtain the correspondence between the data group and the availability zone, and accurately write the data in the data group to the availability zone to which the service node corresponding to the data group belongs, which will not be elaborated further. In addition to the distributed system provided in the foregoing embodiments, the present disclosure also provides a data writing method for multiple availability zones, which will be described by way of example with reference to the accompanying drawings below. Figure 3 is a schematic flowchart of a data writing method for multiple availability zones provided in an exemplary embodiment of the present disclosure. The method may include the steps shown in Figure 3: Step 301: Respond to the data writing request and obtain the data to be written corresponding to each of the at least one data group. Step 302: Determine the service node corresponding to each of the at least one data group based on the management and allocation relationship between the data group and the service node. Step 303: Determine the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each data group belongs. Step 304: Write the data to be written corresponding to each of the at least one data group into the availability zone corresponding to each of the at least one data group, so that the service node corresponding to any data group can read the data corresponding to that data group in its respective availability zone. The execution entity of this embodiment can be the data writing component in the distributed system described in the foregoing embodiments.The management and allocation relationship between data groups and service nodes refers to the relationship formed when a service node is assigned to manage the data read and write operations corresponding to a data group. After determining the availability zone corresponding to each of the at least one data group based on the management and allocation relationship between the data group and the service node, the data writing component can write the data to be written corresponding to each of the at least one data group into the respective availability zone of the at least one data group. Furthermore, the data corresponding to any data group and the service node corresponding to that data group can be distributed in the same availability zone. For any service node, when reading the data corresponding to the data group it manages, it can perform the data read operation within its own availability zone without needing to cross availability zones, reducing the traffic overhead caused by data reads. Optionally, one method of writing the data to be written corresponding to each of the at least one data group into the respective availability zone of the at least one data group may include: splitting the data to be written corresponding to each of the at least one data group into data blocks corresponding to each of the at least one data group; and writing the data blocks corresponding to each of the at least one data group into the respective availability zone of the at least one data group. The size of the data block can be fixed. For example, taking data packets P1, P2, and P3 as examples, data packet P1 can be managed by service node A, while data packets P2 and P3 can be managed by service node B. After splitting data packet P1 into multiple data blocks, since service node A runs in the first availability zone 10, the data writing component 40 can store the multiple data blocks of data packet P1 on data nodes in the first availability zone 10. Correspondingly, after splitting data packet P2 into multiple data blocks, since service node B runs in the second availability zone 20, the data writing component 40 can store the multiple data blocks of data packet P2 on data nodes in the second availability zone 20. Based on this implementation method, the data to be written corresponding to any data packet can be stored in the corresponding availability zone in a block-based manner, allowing different data blocks of the data packet to be processed in parallel, thus accelerating data transmission and storage efficiency. Optionally, the method further includes: aggregating the data blocks corresponding to each of the at least one data group to obtain multiple data blocks; encoding the multiple data blocks to obtain multiple check codes; and storing the multiple check codes in a target availability zone other than the availability zone corresponding to each of the at least one data group. Optionally, when encoding the aggregated multiple data blocks, an erasure coding-based encoding method can be used, as described in the foregoing embodiments, and will not be repeated here.Optionally, a method for writing the data blocks corresponding to each of the at least one data group into the respective availability zones of the at least one data group may include: for any first data group among the at least one data group, distributing and writing the data blocks corresponding to the first data group to multiple data nodes in the availability zone corresponding to the first data group. Here, the first data group is any one of the at least one data group. Based on this implementation, the data blocks in the first data group can be distributed and stored across multiple data nodes in a decentralized storage model. Even if a single data node is compromised, the entire dataset cannot be easily destroyed, thus improving overall data security. Furthermore, when multiple data blocks are distributed and stored across multiple data nodes, the write operations of multiple data blocks can be executed in parallel, thereby significantly improving data processing speed and throughput. Especially in high-performance computing applications such as big data analytics and streaming media services, service quality can be guaranteed based on a large read / write throughput. Optionally, the method further includes: encoding the data blocks corresponding to the first data group to obtain at least one checksum corresponding to the first data group; and storing the at least one checksum in a target data node in the availability zone where the first data group is located. Optionally, when encoding the data blocks corresponding to the first data group, an erasure coding-based encoding method can be used. Based on this implementation, a fault tolerance mechanism can be set between different data nodes within the availability zone. After writing the data blocks of any data group into the availability zone, the data writing component 40 can generate verification data for multiple data blocks corresponding to the data group, and store the verification data and the data blocks corresponding to the data group in different data nodes in the availability zone, so that when some data nodes in the availability zone fail, the original data of the data group can be recovered based on the remaining data blocks and the verification data. The following will continue to use the first data group as an example for illustrative explanation. Optionally, the target data node is another data node in the availability zone where the first data group is located, excluding the data node used to store the data blocks corresponding to the first data group. Taking data group P1 as an example, the k data blocks of data group P1 are stored in data nodes D1, D2, ..., Dk in the first availability zone 10. The data writing component 40 can encode the k data blocks in data group P1 to generate m verification blocks. Subsequently, the m parity blocks can be stored in the target data nodes Dk+1, Dk+2-Dk+m in the first availability zone 10. Based on this implementation, data blocks and parity blocks can be stored in isolation. When a physical failure occurs on the data node where some data blocks reside, the parity blocks on other data nodes remain safe, thereby helping to quickly recover data and improve overall service stability.Optionally, a method for determining the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs may include: for any second data group among the at least one data group, querying metadata information in the distributed system based on the identifier of the service node corresponding to the second data group to obtain the availability zone to which the service node corresponding to the second data group belongs; and using the availability zone to which the service node corresponding to the second data group belongs as the availability zone corresponding to the second data group. In this embodiment, when different service nodes are distributed in different availability zones, after obtaining the data to be written corresponding to each of the at least one data group, the service node corresponding to each of the at least one data group can be determined according to the management and allocation relationship between the data group and the service node. Based on the availability zone to which the service node corresponding to each of the at least one data group belongs, the availability zone corresponding to each of the at least one data group can be determined. Then, the data to be written corresponding to each of the at least one data group can be written into the corresponding availability zone respectively. Furthermore, for any data packet, the corresponding service node and data can be distributed within the same availability zone. When a service node reads data from a data packet it manages, it does not need to perform cross-availability zone data read operations, reducing bandwidth overhead and thus saving data read costs. It should be noted that the execution subject of each step in the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 301 to 304 can be device A; or the execution subject of steps 301 and 302 can be device A, and the execution subject of step 303 can be device B; and so on. Additionally, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document or may be executed in parallel. The operation numbers, such as 301, 302, etc., are merely used to distinguish different operations, and the numbers themselves do not represent any execution order. Furthermore, these processes can include more or fewer operations, and these operations can be executed sequentially or in parallel. It should be noted that the terms "first," "second," etc., used in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. Figure 4 illustrates a schematic diagram of the structure of a server provided in an exemplary embodiment of this disclosure. This server is applicable to the data writing component provided in the foregoing embodiments. As shown in Figure 4, the server includes: a memory 401, a processor 402, and a communication component 403. The memory 401 is used to store computer programs and can be configured to store various other data to support operations on the server. Examples of this data include instructions for any application or method used to operate on the server. Processor 402, coupled to memory 401, is configured to execute a computer program in memory 401 for: responding to a data write request and acquiring data to be written corresponding to at least one data packet; determining the service node corresponding to each of the at least one data packet based on the management allocation relationship between the data packet and the service node; determining the availability zone corresponding to each of the at least one data packet based on the availability zone to which the service node corresponding to each of the at least one data packet belongs; and writing the data to be written corresponding to each of the at least one data packet into the availability zone corresponding to each of the at least one data packet, so that the service node corresponding to any data packet can read the written data corresponding to the data packet in its own availability zone. Optionally, when processor 402 writes the data to be written corresponding to each of the at least one data packet into the availability zone corresponding to each of the at least one data packet, it is specifically configured to: divide the data to be written corresponding to each of the at least one data packet into data blocks corresponding to each of the at least one data packet; and write the data blocks corresponding to each of the at least one data packet into the availability zone corresponding to each of the at least one data packet. Optionally, the processor 402 is further configured to: aggregate the data blocks corresponding to each of the at least one data group to obtain multiple data blocks; encode the multiple data blocks to obtain multiple check codes; and store the multiple check codes in a target availability zone other than the availability zone corresponding to each of the at least one data group.Optionally, when the processor 402 writes the data blocks corresponding to each of the at least one data group into the respective availability zones of the at least one data group, it specifically performs the following: for any first data group among the at least one data group, it distributes and writes the data blocks corresponding to the first data group into multiple data nodes in the availability zone corresponding to the first data group. Optionally, the processor 402 is further configured to: encode the data blocks corresponding to the first data group to obtain at least one checksum corresponding to the first data group; and store the at least one checksum in a target data node in the availability zone where the first data group is located. Optionally, the target data node is another data node in the availability zone where the first data group is located, excluding the data node used to store the data blocks corresponding to the first data group. Optionally, when the processor 402 determines the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs, it specifically performs the following: For any second data group among the at least one data group, based on the identifier of the service node corresponding to the second data group, queries the metadata information in the distributed system to obtain the availability zone to which the service node corresponding to the second data group belongs; and uses the availability zone to which the service node corresponding to the second data group belongs as the availability zone corresponding to the second data group. Further, as shown in FIG4, the server also includes other components such as a power supply component 404, a display component 405, and an audio component 406. FIG4 only schematically shows some components and does not mean that the server only includes the components shown in FIG4. In FIG4, the components within the dashed boxes are optional components, not mandatory components, and the specific components depend on the product form of the electronic device. The electronic device in this embodiment can be a terminal device such as a desktop computer, a laptop computer, a smartphone, or a 10T device, or a server device such as a conventional server, a cloud server, or a server array. If the electronic device in this embodiment is implemented as a desktop computer, laptop computer, smartphone or other terminal device, it may include the components within the dashed box in Figure 4; if the electronic device in this embodiment is implemented as a conventional server, cloud server or server array or other server-side device, it may not include the components within the dashed box in Figure 4.The memory 401 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The communication component 403 is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as Wi-Fi (wireless network communication technology), 2G (such as Global System for Mobile Communications (GSM), etc.), 3G (such as Wideband Code Division Multiple Access (WCDMA), 4G (such as Long Term Evolution (LTE), etc.), 4G+ (such as LTE-Advanced (LTE-A), etc.), or 5G (5th Generation Mobile Communication Technology), or a combination thereof. In an exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may be based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, etc. It is achieved through Infrared Data Association (IRDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.The power supply component 404 provides power to various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides. The display component includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, it may be implemented as a touchscreen to receive input signals from a user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of a touch or swipe action but also the duration and pressure associated with that touch or swipe operation. The audio component is configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device in which the audio component resides is in an operating mode, such as a call mode, recording mode, or voice recognition mode. The received audio signals may be further stored in memory or transmitted via a communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals. In this embodiment, when different service nodes are distributed in different availability zones, after obtaining the data to be written corresponding to each of at least one data packet, the service node corresponding to each of the at least one data packet can be determined according to the management and allocation relationship between the data packet and the service node. Based on the availability zone to which the service node corresponding to each of the at least one data packet belongs, the availability zone corresponding to each of the at least one data packet can be determined. Then, the data to be written corresponding to each of the at least one data packet can be written to the corresponding availability zone respectively. Furthermore, for any data packet, the service node and data corresponding to the data packet can be distributed in the same availability zone. When a service node reads the data of the data packet it manages, it does not need to perform cross-availability zone data reading operations, reducing traffic bandwidth overhead and thus saving data reading costs. Accordingly, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the server in the above method embodiments. This disclosure also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in this disclosure. Those skilled in the art will understand that the embodiments of this disclosure can be provided as methods, systems, or computer program products.Therefore, this disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM (Compact Disk Read-Only Memory), optical storage, etc.) containing computer-usable program code. This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable device, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams. In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in a computer-readable medium, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium. Computer-readable media include both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data.Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Vision Disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The above description is merely an embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

Claims 1. A method for writing data across multiple availability zones, wherein, include: Respond to data write requests and obtain the data to be written for each of at least one data group; Based on the management and allocation relationship between data packets and service nodes, determine the service node corresponding to each of the at least one data packet; based on the availability zone to which the service node corresponding to each of the at least one data packet belongs, determine the availability zone corresponding to each of the at least one data packet. The data to be written corresponding to each of the at least one data group is written into the availability zone corresponding to each of the at least one data group, so that the service node corresponding to any data group can read the data already written to the data group in its own availability zone.

2. The method according to claim 1, wherein, Writing the data to be written to each of the at least one data group into the corresponding availability zone of each of the at least one data group includes: splitting the data to be written to each of the at least one data group into data blocks corresponding to each of the at least one data group; and writing the data blocks corresponding to each of the at least one data group into the corresponding availability zone of each of the at least one data group.

3. The method according to claim 2, wherein, It also includes: aggregating the data blocks corresponding to each of the at least one data group to obtain multiple data blocks; encoding the multiple data blocks to obtain multiple check codes; and storing the multiple check codes in a target availability zone other than the availability zone corresponding to each of the at least one data group.

4. The method according to claim 2 or 3, wherein, Writing the data blocks corresponding to each of the at least one data group into the availability zone corresponding to each of the at least one data group includes: for any first data group in the at least one data group, distributing the data blocks corresponding to the first data group into multiple data nodes in the availability zone corresponding to the first data group.

5. The method according to claim 4, wherein, It also includes: encoding the data block corresponding to the first data group to obtain at least one check code corresponding to the first data group; and storing the at least one check code in the target data node in the availability zone where the first data group is located.

6. The method according to claim 5, wherein, The target data node is any data node in the availability zone where the first data packet is located, excluding the data node used to store the data block corresponding to the first data packet.

7. The method according to any one of claims 1-6, wherein, Determining the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs includes: for any second data group among the at least one data group, querying metadata information in the distributed system based on the identifier of the service node corresponding to the second data group to obtain the availability zone to which the service node corresponding to the second data group belongs; and taking the availability zone to which the service node corresponding to the second data group belongs as the availability zone corresponding to the second data group.

8. A distributed system, wherein, include: The system includes a data writing component and multiple availability zones, each of which comprises a service node and multiple data nodes. The service node in each availability zone is used to manage data read / write operations for at least one data packet. The data writing component is configured to: respond to a data write request and obtain the data to be written corresponding to at least one data group; determine the service node corresponding to each of the at least one data group based on the management and allocation relationship between the data group and the service node; determine the availability zone corresponding to each of the at least one data group based on the availability zone to which the service node corresponding to each of the at least one data group belongs; and write the data to be written corresponding to each of the at least one data group into the data nodes in the availability zones corresponding to the at least one data group, so that the service node corresponding to any data group can read the data already written corresponding to the data group in its own availability zone.

9. A server, wherein, include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions for: performing the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, wherein, When a computer program is executed by a processor, it can implement the data writing method for multiple availability zones as described in any one of claims 7.

11. A computer program product, wherein, include: A computer program / instruction, which, when executed by a processor, enables the implementation of the multi-availability zone data writing method as described in any one of claims 1-7.

Images