A data flow control method and related device

Abstract

The present application provides a data flow control method, which is applied to a database system. The database system includes a host, and the host is connected to M backup machines and a client, where M is a positive integer. The method includes: obtaining the consensus-reaching efficiency when performing log backup between the host and the M backup machines, and the consensus-reaching efficiency is related to at least one of the following information: the log bandwidth for reaching consensus and the time overhead for reaching consensus; determining the data flow information of the host based on the consensus-reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus-reaching efficiency. Based on the characteristics of the consensus algorithm, the present application uses the consensus-reaching efficiency (for example, including the time delay for reaching consensus, the log bandwidth for reaching consensus end-to-end, etc.) as the control target to determine and control the client's business injection flow, thereby improving the stability of the control effect, and does not require parameter adjustment, making it easy to implement.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a data flow control method and related devices. Background Art

[0002] In distributed systems, to improve availability and reliability, multiple backup servers are assigned to each master. Masters can replicate logs to these servers using a multi-replication mechanism. While multi-replication does improve reliability and availability, it also creates a new challenge: ensuring consensus among these replicas. Specifically, ensuring that the replicas agree on any given proposal. A consensus algorithm is essential for reaching consensus on a proposal. The Paxos algorithm is synonymous with distributed consensus, and most commonly used consensus algorithms are based on it. To put it simply, the master proposes a proposal and sends it to the other backup servers. These servers can accept or reject the proposal based on the consensus algorithm. When the master confirms a majority of the backup servers have accepted the proposal, consensus is reached on the proposal. The proposal is a log file. Consensus-based log replication involves the database master replicating the log file to the other backup servers using a consensus algorithm. Consensus is reached when a log file receives responses from more than half (or close to 50%) of the backup servers.

[0003] Since the data sender's rate isn't necessarily equal to the receiver's rate, if the sender's rate is too high, the receiver may be unable to process the data. The unprocessed data will be stored in the buffer, and if the traffic volume increases further, the packets will be discarded. Because system performance is limited, a large data backlog or packet loss can cause a sharp decline in end-to-end service performance. Flow control regulates the sender's traffic flow to match that of the receiver. This is especially true during high concurrency. By limiting traffic injection, we prevent excessive requests from overwhelming the system and potentially causing a system crash, thereby ensuring service availability.

[0004] In the field of database flow control, most existing technologies control the flow between the host and the client based on queue high and low watermarks, which has poor control effect. Summary of the Invention

[0005] In a first aspect, an embodiment of the present application provides a data flow control method, which is applied to a database system, wherein the database system includes a host, and the host is communicatively connected to M backup machines and a client, where M is a positive integer. The method includes: obtaining a consensus-reaching efficiency when performing log backup between the host and the M backup machines, wherein the consensus-reaching efficiency is related to at least one of the following information: the log bandwidth for reaching consensus and the time overhead for reaching consensus; determining the data flow information of the host based on the consensus-reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus-reaching efficiency.

[0006] The consensus efficiency can be understood as the efficiency of reaching consensus between the master and the slave during master-slave log replication.

[0007] Among them, reaching a consensus can be understood as the host receiving confirmation acks for a certain log sent by multiple backup machines. When the number of backup machines and hosts that send the above-mentioned confirmation acks exceeds half of the total number of hosts and backup machines in the current master-backup system (or other thresholds), it can be considered that a consensus has been reached between the master and backup machines for the log. For example, the master-backup system includes one master and two backup machines. When the host itself completes the consensus and the host receives the confirmation ack sent by at least one backup machine, it can be considered that a consensus has been reached between the master and backup machines. For another example, the master-backup system includes one host and four backup machines. When the host itself completes the consensus and the host receives the confirmation ack sent by at least two backup machines, it can be considered that a consensus has been reached between the master and backup machines.

[0008] The consensus-reaching efficiency may be related to the log bandwidth for reaching consensus. In one implementation, the consensus-reaching efficiency is positively correlated with the log bandwidth for reaching consensus. That is, when other information remains unchanged, the higher the log bandwidth for reaching consensus between the master and multiple backup machines, the higher the consensus-reaching efficiency.

[0009] Among them, the consensus reaching efficiency may be related to the time cost of reaching consensus. In one implementation, the consensus reaching efficiency is negatively correlated with the time cost of reaching consensus. That is, when other information remains unchanged, the higher the time cost of reaching consensus between the host and multiple backup machines, the lower the consensus reaching efficiency.

[0010] Among them, after obtaining the consensus efficiency, the data flow information of the host can be determined based on the consensus efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus efficiency. That is, when other information remains unchanged, the higher the consensus efficiency between the host and multiple backup machines, the higher the flow of data injected by the client received by the host.

[0011] In the embodiment of the present application, based on the characteristics of the consensus algorithm, the efficiency of consensus (for example, the delay in reaching consensus, the log bandwidth for end-to-end consensus, etc.) is used as the control target to determine and control the client's business injection traffic, and the control effect is certain and stable.

[0012] Existing technologies (such as MySQL group peering) use quotas for flow control. All nodes periodically communicate information such as queue statistics with each other, and each node performs flow control based on the received queue statistics. MySQL group peering has approximately 10 parameters that require tuning for different scenarios. However, in the embodiments of the present application, data transmission flow can be directly controlled based on historical log data, eliminating the need for parameter tuning and making implementation easier.

[0013] In a possible implementation, the method further includes: obtaining information on multiple logs on which consensus is reached between the host and the M backup machines within a historical sampling time period, the information on the multiple logs including the log sizes of the multiple logs; and determining the bandwidth of the consensus operation based on the log sizes of the multiple logs and the length of the historical sampling time period. Specifically, sampling can be performed according to a certain historical sampling time period (e.g., 1 second), sampling the size of each log on which consensus is reached within a period, and the time overhead of each log on which consensus is reached from the start of writing to the moment of reaching consensus. The log bandwidth of the consensus is calculated based on the size of each log on which consensus is reached within the historical sampling time period and the length of the historical sampling time period. The log bandwidth of the consensus can be recorded as bw1. For example, the log bandwidth of the consensus bw1 can be equal to the sum of the sizes of the logs on which consensus is reached within the historical sampling time period divided by the length of the historical sampling time period.

[0014] In a possible implementation, the method further includes: obtaining information on multiple logs on which consensus is reached between the host and the M backup machines within a historical sampling time period, the information on the multiple logs including the time overhead for each of the multiple logs to reach a consensus; and determining the time overhead for the consensus operation based on the time overhead for each of the multiple logs to reach a consensus. Specifically, sampling can be performed according to a certain historical sampling time period (e.g., 1 second), sampling the size of each log on which consensus is reached within a period, and the time overhead for each log on which consensus is reached from the moment of writing to the moment of reaching a consensus. The above-mentioned time overhead for reaching a consensus is calculated based on the average time overhead of each log on which consensus is reached within a historical sampling time period, which can be recorded as lat, for example.

[0015] In one possible implementation, the method further includes: obtaining the log playback bandwidth of the M backup machines when performing log backup between the host and the M backup machines; determining the data traffic information of the host based on the consensus reaching efficiency includes: determining the data traffic information of the host based on the consensus reaching efficiency and the log playback bandwidth, wherein the traffic is positively correlated with the log playback bandwidth.

[0016] In one possible implementation, obtaining the log playback bandwidth of the M backup machines when performing log backup between the master and the M backup machines includes: obtaining information of multiple logs on which consensus is reached between the master and the M backup machines and data playback is completed within a historical sampling time period, the information of the multiple logs on which consensus is reached between the master and the M backup machines and data playback is completed within the historical sampling time period includes the log size of the multiple logs on which data playback is completed; and determining the log playback bandwidth based on the log size of the multiple logs and the length of the historical sampling time period.

[0017] Log playback refers to applying logs to a data system.

[0018] In one possible implementation, information about multiple logs on which consensus is reached between the host and the M backup machines and data playback is completed within a historical sampling period can be obtained. The information about the multiple logs on which data playback is completed includes the log sizes of the multiple logs on which data playback is completed, and the log playback bandwidth is determined based on the log sizes of the multiple logs and the length of the historical sampling period.

[0019] For example, when the master copies logs to other slaves, the other slaves can reply with an acknowledgment ack message, and carry the log index that the slave has completed replaying in the ack message. The master can obtain the replayed log indexes of all slaves, and combine them with the size of each log to obtain the overall log replay bandwidth of the system during the sampling period, which is recorded as bw2.

[0020] In a possible implementation, the historical sampling time period is a sampling period whose time interval with the current moment is less than a threshold among multiple sampling periods before the current moment; or, the historical sampling time period is multiple sampling periods before the current moment.

[0021] For example, before the current moment include sampling period 1, sampling period 2, sampling period 3, sampling period 4, sampling period 5, sampling period 6, sampling period 7, sampling period 8, and sampling period 9, among which sampling period 1 is the sampling period closest to the current moment in time, then the above-mentioned historical sampling time period can be sampling period 1, and the historical sampling time period can also be the union of period 1, sampling period 2, sampling period 3, sampling period 4, sampling period 5, sampling period 6, sampling period 7, sampling period 8 and sampling period 9, or a subset of the union, which is not limited here.

[0022] Among them, the threshold value can be related to the time length of a sampling cycle. For example, the threshold value can be a sampling cycle or a multiple of a sampling cycle. When the threshold value is a sampling cycle, the historical sampling time period is the sampling cycle with the smallest time interval between the multiple sampling cycles before the current moment and the current moment. It should be understood that the specific threshold value is not limited in this application. The smaller the threshold value is set, the smaller the number of sampling cycles in the historical sampling time period. The larger the threshold value is set, the larger the number of sampling cycles in the historical sampling time period.

[0023] In one possible implementation, the data flow information may include flow control direction information and flow control step information. The flow control direction information is used to indicate the direction of change in the flow of data injected by the client received by the host (for example, it may indicate whether the flow increases or decreases), and the flow control step information is used to indicate the change in the value of the flow of data injected by the client received by the host. Based on the direction of change in the data flow and the flow control step information, the flow of data injected by the client received by the host can be determined. Based on the data flow information, the flow of data injected by the client received by the host can be controlled.

[0024] In a second aspect, the present application provides a data flow control device, which is applied to a database system. The database system includes a host computer, the host computer is in communication with M backup computers and a client computer, where M is a positive integer. The device includes:

[0025] an acquisition module, configured to acquire a consensus-reaching efficiency when performing log backup between the master and the M backup machines, wherein the consensus-reaching efficiency is related to at least one of the following information: a log bandwidth for reaching consensus and a time overhead for reaching consensus;

[0026] A data flow determination module is used to determine the data flow information of the host based on the consensus reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus reaching efficiency.

[0027] In a possible implementation, the consensus-reaching efficiency is positively correlated with the log bandwidth for reaching consensus, and the consensus-reaching efficiency is negatively correlated with the time overhead for reaching consensus.

[0028] In a possible implementation, the acquisition module is further configured to:

[0029] Obtain information about multiple logs that are agreed upon between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the log sizes of the multiple logs;

[0030] The bandwidth of the consensus operation is determined according to the log sizes of the multiple logs and the length of the historical sampling time period.

[0031] In a possible implementation, the acquisition module is further configured to:

[0032] Obtain information about multiple logs on which consensus is reached between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the time overhead for reaching consensus on each of the multiple logs;

[0033] The time cost of the consensus operation is determined according to the time cost of reaching consensus on each log in the multiple logs.

[0034] In a possible implementation, the acquisition module is further configured to:

[0035] Obtaining the log playback bandwidth of the M backup machines when performing log backup between the master and the M backup machines;

[0036] The data flow determination module is specifically configured to:

[0037] Based on the consensus reaching efficiency and the log replay bandwidth, data traffic information of the host is determined, wherein the traffic is positively correlated with the log replay bandwidth.

[0038] In a possible implementation, the acquisition module is specifically configured to:

[0039] Obtain information about multiple logs on which the master and the M backup machines reach a consensus and complete data playback within a historical sampling period. The information about multiple logs on which the master and the M backup machines reach a consensus and complete data playback within the historical sampling period includes the log sizes of the multiple logs on which data playback has been completed.

[0040] The log playback bandwidth is determined according to the log sizes of the multiple logs and the length of the historical sampling time period.

[0041] In a possible implementation, the historical sampling time period is a sampling period whose time interval with the current moment is less than a threshold among multiple sampling periods before the current moment; or, the historical sampling time period is multiple sampling periods before the current moment.

[0042] In one possible implementation, the data flow information includes flow control direction information and flow control step information. The flow control direction information is used to indicate the direction of change in the flow of data injected by the client received by the host, and the flow control step information is used to indicate the value of change in the flow of data injected by the client received by the host.

[0043] In a third aspect, the present application provides a data flow control device, which may include a processor coupled to a memory, wherein the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the first aspect and any of the methods described above are implemented. For details of the steps performed by the processor in each possible implementation of the first aspect, please refer to the first aspect and will not be repeated here.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0045] In a fifth aspect, the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the first aspect and any one of the methods described above.

[0046] In a sixth aspect, the present application provides a computer program which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0047] In a seventh aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the server or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices.

[0048] The embodiment of the present application provides a data flow control method, which is applied to a database system, wherein the database system includes a host, and the host is in communication connection with M backup machines and a client, wherein M is a positive integer, and the method includes: obtaining the consensus-reaching efficiency when performing log backup between the host and the M backup machines, wherein the consensus-reaching efficiency is related to at least one of the following information: the log bandwidth for reaching consensus and the time overhead for reaching consensus; determining the data flow information of the host based on the consensus-reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus-reaching efficiency. In the embodiment of the present application, according to the characteristics of the consensus algorithm, the optimal consensus-reaching efficiency (for example, including the end-to-end consensus delay, the end-to-end consensus log bandwidth, etc.) is used as the control target, and the flow injected by the business is adaptively controlled. The control effect is determined and stable, and does not require parameter adjustment, and is easy to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1A This is a schematic diagram of the architecture of a database system;

[0050] Figure 1B A schematic diagram of the architecture of a distributed database system provided in an embodiment of the present application;

[0051] Figure 1C Another schematic diagram of the architecture of the distributed database system provided in an embodiment of the present application;

[0052] Figure 1D An example of an application architecture provided in an embodiment of the present application;

[0053] Figure 1E A schematic diagram of a system architecture provided in an embodiment of the present application;

[0054] Figure 2 A flow chart of a data flow control method provided in an embodiment of the present application;

[0055] Figure 3 A flow chart of a data flow control method provided in an embodiment of the present application;

[0056] Figure 4This is a schematic diagram of determining a control amount provided in an embodiment of the present application;

[0057] Figure 5 This is a schematic diagram of the structure of a data flow control device shown in an embodiment of the present application;

[0058] Figure 6 A computer-readable storage medium is provided for this application;

[0059] Figure 7 This is a schematic diagram of a computer device provided in this application. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0061] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time or logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0062] The method provided in the embodiment of the present application can be applied to Figure 1AIn the illustrated database system 100, the database system 100 is in communication with an application server 300 to provide database services to the application server 300. The application server 300 is in communication with a client 200, which is typically an application program deployed on a user device. The client 200 initiates service requests to the application server 300 to implement specific functions, such as downloading or uploading data. The data associated with the client 200 is stored in the database system 100. When responding to the service request from the client 200, if the application server 300 needs to operate on the data associated with the client 200 (such as querying data, adding data, updating data, deleting data, etc.), it can do so by sending operation requests to the database system 100.

[0063] Figure 1A A typical logical architecture of the database system 100 is shown. Figure 1A The database system 100 includes a database 110 and a database management system (DBMS) 130 .

[0064] Among them, the database 110 is an organized data set stored in the data storage 120, that is, a related data set organized, stored and used according to a specific data model. According to the different data models used to organize data, data can be divided into multiple types, such as relational data, graph data, time series data, etc. Relational data is data modeled using a relational model, usually represented as a table, and the rows in the table represent a set of related values of an object or entity. Graph data, referred to as "graph", is used to represent the relationship between objects or entities, such as social relationships. Time series data, referred to as time series data, is a data column recorded and indexed in chronological order, used to describe the state change information of an object in the time dimension.

[0065] The database management system 130 is the core of the database system and is the system software used to organize, store, and maintain data. Clients 200 can access the database 110 through the database management system 130, and database administrators also use the database management system to perform database maintenance. The database management system 130 provides various functions for clients 200, which can be applications or user devices running applications, to create, modify, and query databases. The functions provided by the database management system 130 may include but are not limited to the following: (1) Data definition function. The database management system 130 provides a data definition language (DDL) to define the structure of the database 110. DDL is used to describe the database framework and can be saved in the data dictionary; (2) Data access function. The database management system 130 provides a data manipulation language (DML) to implement basic access operations on the database 110, such as retrieval, insertion, modification and deletion; (3) Database operation management function. The database management system 130 provides a data control function to effectively control and manage the operation of the database 110 to ensure that the data is correct and valid; (4) Database establishment and maintenance function, including loading the initial data of the database, dumping, restoring, and reorganizing the database, system performance monitoring, analysis and other functions; (5) Database transmission. The database management system provides transmission of processed data to realize communication between the client and the database management system, which is usually coordinated with the operating system.

[0066] The database storage 120 includes, but is not limited to, solid state drives (SSDs), disk arrays, cloud storage, or other types of non-transitory computer-readable storage media.

[0067] In an embodiment of the present application, client 200 can initiate a service request to application server 300. Application server 300 is deployed with a data service to respond to the service request initiated by client 200. In one embodiment, the data service deployed on application server 300 can verify the legitimacy of client 200's access. If the verification is successful, the session is recorded and the service request initiated by client 200 is converted into a data operation request, such as a query statement, for database 110. Furthermore, the data service can perform real-time statistics and control of the system resources occupied by different clients 200.

[0068] It should be understood that those skilled in the art will understand that a database system may include Figure 1A Fewer or more components than those shown in, or including Figure 1A The components shown are different components, Figure 1AOnly components more relevant to the implementation disclosed in the embodiment of the present invention are shown.

[0069] The following combination Figure 1B An embodiment of an application server 300 provided in this application is introduced.

[0070] The functions implemented by the application server 300 may include but are not limited to access control, session management, data management, resource monitoring, storage management, etc. Among them, access control can control the legitimacy of client access and bandwidth. Session management can perform session management on clients that have successfully accessed. Data management can convert client service requests into operation requests for the database. Resource monitoring can perform real-time statistics and control on system resources occupied by different clients. Storage management can convert operation requests for the database into operation requests supported or executable by the database system, such as database query statements (referred to as "queries"), which can be structured query language (SQL) queries. It should be noted that the application server 300 can convert service requests into queries supported or executable by the database system in one go or in multiple times, and the specific conversion process belongs to the existing technology in this field.

[0071] The database system provided in the embodiment of the present application may be a distributed database system (DDBS), such as a database system with a massively parallel processor (MPP) architecture. Figure 1C and Figure 1D Introducing DDBS.

[0072] Figure 1CThis diagram illustrates a distributed database system using a shared-storage architecture. The system includes one or more coordinator nodes (CNs) and multiple data nodes (DNs). The DDBS may also include other components, such as a global transaction manager (GTM). The CNs and DNs communicate via a network channel. Based on received queries, such as those from application servers, the CNs can generate execution plans for the queries and, based on the execution plans, distribute the queries to the corresponding DNs for execution. The CNs can also generate query results based on the execution results of the DNs. In one embodiment, the network channels can be composed of network devices such as switches, routers, and gateways. The CNs and DNs jointly implement the functions of a database management system, providing clients with services such as database retrieval, insertion, modification, and deletion. In one embodiment, a database management system is deployed on each CN and DN. A shared data storage system stores data shared by multiple DNs, and the DNs can perform read and write operations on the data in the data storage system via the network channel. The shared data storage system can be a shared disk array. The CN and DN in a distributed database system can be physical machines, such as database servers, or virtual machines (VMs) or containers running on abstract hardware resources. In one embodiment, the CN and DN are virtual machines or containers, and the network channel is a virtual switching network, which includes a virtual switch. The database management system deployed in the CN and DN is a DBMS instance, which can be a process or a thread. These DBMSs work together to complete the functions of the database relational system. In another embodiment, the CN and DN are physical machines, and the network channel includes one or more switches, which are storage area network (SAN) switches, Ethernet switches, fiber switches, or other physical switching devices.

[0073] Figure 1D This is a diagram of a distributed database system using a shared-nothing architecture. Each DN has its own dedicated hardware resources (such as CPU, memory, data storage, etc.). CN and DN communicate through a network channel. The network channel can be found in the above Figure 1C Under this system, data will be distributed to various DNs according to the database model and application characteristics. Query tasks will be divided into several parts by CN and executed in parallel on multiple DNs. They will coordinate calculations with each other to provide database services as a whole. All communication functions are implemented on a high-bandwidth network interconnection system. Figure 1CLike the distributed database system with shared-storage architecture described in the previous section, the CN and DN here can be either physical machines or virtual machines.

[0074] In all embodiments of the present application, the data storage of the database system includes but is not limited to solid state drives (SSDs), disk arrays, or other types of non-transitory computer-readable media. Figure 1C-1D Although the database is not shown in the figure, it should be understood that the database is stored in the data storage. Those skilled in the art will understand that a database system may include Figure 1A-Figure 1D Fewer or more components than those shown in, or including Figure 1A-Figure 1D The components shown in the figure are different components. Figure 1A-Figure 1D Only components more relevant to the implementation disclosed in the embodiments of this application are shown. However, those skilled in the art will appreciate that a distributed database system can include any number of CNs and DNs. The database management system functions of each CN and DN can be implemented by an appropriate combination of software, hardware, and / or firmware running on each CN and DN.

[0075] In the embodiment of the present application, the distributed database system can provide application developers with application (APP) deployment services. Specifically, the application can be deployed on a node in the distributed database system, such as a virtual machine. The virtual machine can include one or more data nodes DN and corresponding data storage (for example, Figure 1C The shared data memory in Figure 1D An application can send a service request to an application server. The application server processes the service request, for example, by converting it into one or more data operation requests supported or executable by the database system and sending the data operation requests to the distributed database system. In response to the data operation request, the DN corresponding to the application in the distributed database system performs the operation on the data in the data storage.

[0076] like Figure 1E As shown, Figure 1E This is an application architecture diagram of an embodiment of the present application, such as Figure 1E As shown, the application architecture may include: a host, a backup machine 1, and a backup machine 2 (it should be understood that more backup machines may be included). Figure 1E Two standby machines are used as an example), one or more clients (e.g. Figure 1EClients 1 and 2 in the example can inject business traffic into the master. The database replicates the master-slave logs based on a consensus algorithm and synchronizes the logs to slaves 1 and 2. Due to network congestion or physical distance, the network latency between the master and slave servers may be significant. Clients may inject traffic into the database at high concurrency.

[0077] It should be understood that Figure 1E The scenario shown is just an example, and the technical solutions in the embodiments of this application can be applied to any database master-slave replication scenario based on a consensus algorithm.

[0078] Figure 2 The flow diagram of a data flow control method provided in an embodiment of the present application is as follows. The data flow control method provided in an embodiment of the present application can be applied to a database system, wherein the database system includes a host computer, the host computer is connected to M backup computers and a client computer, and M is a positive integer, such as Figure 2 As shown, the method may include:

[0079] 201. Obtain consensus-reaching efficiency when performing log backup between the master and the M backup machines, where the consensus-reaching efficiency is related to at least one of the following information: log bandwidth for reaching consensus and time overhead for reaching consensus.

[0080] Among them, the execution subject of step 201 can be the host, that is, the host obtains the consensus efficiency when performing log backup between the host and the M backup machines, or the execution subject of step 201 can be other computing nodes in the host database, which is not limited here.

[0081] The consensus efficiency can be understood as the efficiency of reaching consensus between the master and the slave during master-slave log replication.

[0082] Among them, reaching a consensus can be understood as the host receiving confirmation acks for a certain log sent by multiple backup machines. When the number of backup machines and hosts that send the above-mentioned confirmation acks exceeds half of the total number of hosts and backup machines in the current master-backup system (or other thresholds), it can be considered that a consensus has been reached between the master and backup machines for the log. For example, the master-backup system includes one master and two backup machines. When the host itself completes the consensus and the host receives the confirmation ack sent by at least one backup machine, it can be considered that a consensus has been reached between the master and backup machines. For another example, the master-backup system includes one host and four backup machines. When the host itself completes the consensus and the host receives the confirmation ack sent by at least two backup machines, it can be considered that a consensus has been reached between the master and backup machines.

[0083] The consensus-reaching efficiency may be related to the log bandwidth for reaching consensus. In one implementation, the consensus-reaching efficiency is positively correlated with the log bandwidth for reaching consensus. That is, when other information remains unchanged, the higher the log bandwidth for reaching consensus between the master and multiple backup machines, the higher the consensus-reaching efficiency.

[0084] In one possible implementation, when a database system's main machine replicates logs based on a consensus algorithm, it can record the start time of each log entry, the log size, and the time when consensus is reached. The start time is when the business injects traffic into the main machine and the log entry is generated. The consensus time is when the main machine replicates the log entry to each backup machine, each backup machine receives the log entry and sends an acknowledgment, and the main machine receives acknowledgments from more than half of the backup machines.

[0085] In one possible implementation, information of multiple logs on which consensus is reached between the host and the M backup machines within a historical sampling time period can be obtained, where the information of the multiple logs includes the log sizes of the multiple logs; and the bandwidth of the consensus operation is determined based on the log sizes of the multiple logs and the length of the historical sampling time period.

[0086] Specifically, sampling can be performed at a certain sampling time period (e.g., 1 second). The size of each consensus-reached log within a sampling period and the time it takes for each consensus-reached log to be written to the time consensus is reached are sampled. The consensus log bandwidth is calculated based on the size of each consensus-reached log within the historical sampling time period and the length of the historical sampling time period. This consensus log bandwidth can be denoted as bw1. For example, this consensus log bandwidth bw1 can be equal to the sum of the sizes of the consensus-reached logs within the historical sampling time period divided by the length of the historical sampling time period.

[0087] Among them, the consensus reaching efficiency may be related to the time cost of reaching consensus. In one implementation, the consensus reaching efficiency is negatively correlated with the time cost of reaching consensus. That is, when other information remains unchanged, the higher the time cost of reaching consensus between the host and multiple backup machines, the lower the consensus reaching efficiency.

[0088] In one possible implementation, information of multiple logs on which consensus is reached between the host and the M backup machines within a historical sampling time period can be obtained. The information of the multiple logs includes the time overhead for each of the multiple logs to reach consensus. Based on the time overhead for each of the multiple logs to reach consensus, the time overhead of the consensus operation is determined.

[0089] Specifically, sampling can be performed at a specific sampling period (e.g., 1 second). The size of each consensus-reaching log and the time it takes for each consensus-reaching log to be written to the time consensus is reached within the sampling period are sampled. The consensus-reaching time cost, for example, can be calculated based on the average time cost of each consensus-reaching log within the historical sampling period.

[0090] In one possible implementation, the consensus-reaching efficiency may be related to the log bandwidth for reaching consensus. In one possible implementation, the consensus-reaching efficiency may be related to the time overhead for reaching consensus. In one possible implementation, the consensus-reaching efficiency may be related to the log bandwidth for reaching consensus and the time overhead for reaching consensus.

[0091] In a possible implementation, the historical sampling time period is a sampling period whose time interval with the current moment is less than a threshold among multiple sampling periods before the current moment; or, the historical sampling time period is multiple sampling periods before the current moment.

[0092] For example, before the current moment include sampling period 1, sampling period 2, sampling period 3, sampling period 4, sampling period 5, sampling period 6, sampling period 7, sampling period 8, and sampling period 9, among which sampling period 1 is the sampling period closest to the current moment in time, then the above-mentioned historical sampling time period can be sampling period 1, and the historical sampling time period can also be the union of period 1, sampling period 2, sampling period 3, sampling period 4, sampling period 5, sampling period 6, sampling period 7, sampling period 8 and sampling period 9, or a subset of the union, which is not limited here.

[0093] 202. Determine data flow information of the host based on the consensus reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus reaching efficiency.

[0094] In an embodiment of the present application, after obtaining the consensus reaching efficiency, the data flow information of the host can be determined based on the consensus reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus reaching efficiency. That is, when other information remains unchanged, the higher the consensus reaching efficiency between the host and multiple backup machines, the higher the flow of data injected by the client received by the host.

[0095] In one possible implementation, the data traffic information can also be determined based on the log playback bandwidth of the standby machine. Specifically, the log playback bandwidth of the M standby machines can be obtained when performing log backup between the host and the M standby machines, and then the data traffic information of the host can be determined based on the consensus efficiency and the log playback bandwidth, wherein the traffic is positively correlated with the log playback bandwidth, that is, when other information remains unchanged, the higher the log playback bandwidth of multiple standby machines, the higher the traffic of the host receiving the data injected by the client.

[0096] Log playback refers to applying logs to a data system.

[0097] In one possible implementation, information about multiple logs on which consensus is reached between the host and the M backup machines and data playback is completed within a historical sampling period can be obtained. The information about the multiple logs on which data playback is completed includes the log sizes of the multiple logs on which data playback is completed, and the log playback bandwidth is determined based on the log sizes of the multiple logs and the length of the historical sampling period.

[0098] For example, when the master copies logs to other slaves, the other slaves can reply with an acknowledgment ack message, and carry the log index that the slave has completed replaying in the ack message. The master can obtain the replayed log indexes of all slaves, and combine them with the size of each log to obtain the overall log replay bandwidth of the system during the sampling period, which is recorded as bw2.

[0099] In a possible implementation, the historical sampling time period is a time period before and adjacent to the current moment; or, the historical sampling time period is multiple time periods before the current moment.

[0100] In one possible implementation, the data flow information may include flow control direction information and flow control step information. The flow control direction information is used to indicate the direction of change in the flow of data injected by the client received by the host (for example, it may indicate whether the flow increases or decreases), and the flow control step information is used to indicate the change in the value of the flow of data injected by the client received by the host. Based on the direction of change in the data flow and the flow control step information, the flow of data injected by the client received by the host can be determined. Based on the data flow information, the flow of data injected by the client received by the host can be controlled.

[0101] For example, refer to Figure 3The system can record the time overhead (lat), log bandwidth (bw1), and log playback bandwidth (bw2) required to reach end-to-end consensus for a certain number of historical sampling periods (or adjacent historical sampling periods). It also records a certain number of historical control variables (ctrl_val), control directions (direction), and control steps (step). Then, based on the values of lat, bw1, and bw2 in the current sampling period and combined with the results from the historical periods, it calculates the new control direction (new_direction) and control step (new_step). Finally, it calculates the new control variable (new_ctrl_val) for adaptively controlling the injected traffic.

[0102] For example, you can Figure 4 As shown, Figure 4 This represents a certain number of historical sampling results (lat, bw1, bw2), the historical control variable (crtl_val), the control direction (direction), and the control step size (step), as well as the sampling results (new_lat, new_bw1, and new_bw2) for the current sampling period. Initially, the control direction (direction) and the control variable (ctrl_val) can each be set with an initial value. The control direction (direction) can be positive or negative. The new control direction (new_direction) can be determined by comparing the latest lat, bw1, and bw2 sampling results with the historical results. If performance deteriorates, the control direction is changed; otherwise, the original control direction is maintained. The new control step size (new_step) can be calculated based on the difference between the latest lat, bw1, and bw2 sampling results and the historical results, and adjusted based on the value of the historical control variable (crtl_val). The new control variable (new_ctrl_val) can be calculated by multiplying the historical control variable (new control direction) and the new control step size.

[0103] Specifically, it has been measured that in a business scenario with a 10ms network delay, after applying the solution of the present invention, the TPS performance of the 10ms business can be improved by more than 10 times.

[0104] Among them, you can refer to Table 1, such as Table 1 is the result of effect verification:

[0105] Table 1

[0106] Business TPS performance at 10ms No flow control solution 7386 Solution of the present invention 103487

[0107] The existing technology is basically based on queue statistics when doing flow control. The method of flow control through queue statistics requires the configuration of many parameters, and different scenarios require parameter adjustment, and the control effect is uncertain. In the embodiment of the present application, according to the characteristics of the consensus algorithm, the optimal consensus efficiency (for example, including the delay for reaching an end-to-end consensus, the log bandwidth for reaching an end-to-end consensus, etc.) is used as the control target, and the client's business injection flow is adaptively determined and controlled. The control effect is certain and stable. In the existing technology (such as MySQL group peplication), flow control is based on quotas. All nodes periodically transmit their own queue statistics and other information to each other. Each node performs flow control after calculation and processing based on the received queue statistics. MySQL group peplication is set with about 10 parameters, which need to be adjusted for different scenarios. In the embodiment of the present application, the size of the data transmission flow can be directly controlled through historical log data, without the need for parameter adjustment, and is easy to implement.

[0108] Combined with the above Figures 2 to 4 The data processing method provided in the embodiment of the present application is introduced in detail. The data processing device provided in the embodiment of the present application will be introduced from the perspective of functional units in conjunction with the accompanying drawings.

[0109] Reference Figure 5 , Figure 5 The structure of a data flow control device provided in an embodiment of the present application is schematically shown, and is applied to a database system. The database system includes a host computer, and the host computer is connected to M backup computers and a client computer, where M is a positive integer, such as Figure 5 As shown, the apparatus 500 may include:

[0110] An acquisition module 501 is configured to acquire a consensus-reaching efficiency when performing log backup between the master and the M backup machines, wherein the consensus-reaching efficiency is related to at least one of the following information: a log bandwidth for reaching consensus and a time overhead for reaching consensus;

[0111] The description of the acquisition module 501 can refer to the description of step 201 and will not be repeated here.

[0112] The data flow determination module 502 is used to determine the data flow information of the host based on the consensus reaching efficiency, wherein the data flow information is used to control the flow of the data injected by the client received by the host, and the flow is positively correlated with the consensus reaching efficiency.

[0113] The description of the data flow determination module 502 can refer to the description of step 202 and will not be repeated here.

[0114] In a possible implementation, the consensus-reaching efficiency is positively correlated with the log bandwidth for reaching consensus, and the consensus-reaching efficiency is negatively correlated with the time overhead for reaching consensus.

[0115] In a possible implementation, the acquisition module 501 is further configured to:

[0116] Obtain information about multiple logs that are agreed upon between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the log sizes of the multiple logs;

[0117] The bandwidth of the consensus operation is determined according to the log sizes of the multiple logs and the length of the historical sampling time period.

[0118] In a possible implementation, the acquisition module 501 is further configured to:

[0119] Obtain information about multiple logs on which consensus is reached between the master and the M backup machines during a historical sampling period, where the information about the multiple logs on which consensus is reached between the master and the M backup machines during the historical sampling period includes a time overhead for reaching consensus on each of the multiple logs;

[0120] The time cost of the consensus operation is determined according to the time cost of reaching consensus on each log in the multiple logs.

[0121] In a possible implementation, the acquisition module 501 is further configured to:

[0122] Obtaining the log playback bandwidth of the M backup machines when performing log backup between the master and the M backup machines;

[0123] The data flow determination module 502 is specifically configured to:

[0124] Based on the consensus reaching efficiency and the log replay bandwidth, data traffic information of the host is determined, wherein the traffic is positively correlated with the log replay bandwidth.

[0125] In a possible implementation, the acquisition module 501 is specifically configured to:

[0126] Obtain information about multiple logs that have reached a consensus between the master and the M backup machines and completed data playback within a historical sampling period, where the information about the multiple logs that have completed data playback includes the log sizes of the multiple logs that have completed data playback;

[0127] The log playback bandwidth is determined according to the log sizes of the multiple logs and the length of the historical sampling time period.

[0128] In a possible implementation, the historical sampling time period is a sampling period whose time interval with the current moment is less than a threshold among multiple sampling periods before the current moment; or, the historical sampling time period is multiple sampling periods before the current moment.

[0129] In one possible implementation, the data flow information includes flow control direction information and flow control step information. The flow control direction information is used to indicate the direction of change in the flow of data injected by the client received by the host, and the flow control step information is used to indicate the value of change in the flow of data injected by the client received by the host.

[0130] The embodiment of the present application provides a data flow control device, which is applied to a database system, wherein the database system includes a host, and the host is in communication connection with M backup machines and a client, wherein M is a positive integer, and the device may include: an acquisition module for acquiring the consensus-reaching efficiency when performing log backup between the host and the M backup machines, wherein the consensus-reaching efficiency is related to at least one of the following information: the log bandwidth for reaching consensus and the time overhead for reaching consensus; a data flow determination module for determining the data flow information of the host based on the consensus-reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus-reaching efficiency. According to the characteristics of the consensus algorithm, the present application takes the optimal consensus-reaching efficiency (for example, including the end-to-end consensus delay, the end-to-end consensus log bandwidth, etc.) as the control target, adaptively determines and controls the client's business injection flow, and the control effect is determined and stable, and does not require parameter adjustment, and is easy to implement.

[0131] Reference Figure 6 , the present application also provides a computer-readable storage medium, in some embodiments, the above Figure 2 The disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture. Figure 6 Schematically illustrates a conceptual partial view of an example computer program product arranged in accordance with at least some embodiments presented herein, the example computer program product comprising a computer program for executing a computer process on a computing device. In one embodiment, the example computer program product 600 is provided using a signal-bearing medium 601. The signal-bearing medium 601 may include one or more program instructions 602 that, when executed by one or more processors, may provide the above-described instructions for executing a computer process. Figure 2 Thus, for example, reference to Figure 2In the embodiment shown in , one or more features of steps 201 to 202 may be undertaken by one or more instructions associated with the signal bearing medium 601. In addition, Figure 6 Program instructions 602 in also describe example instructions.

[0132] In some examples, signal-bearing medium 601 may include computer-readable media 603, such as, but not limited to, a hard drive, a compact disc (CD), a digital video disc (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc. In some embodiments, signal-bearing medium 601 may include computer-recordable media 604, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, signal-bearing medium 601 may include communication media 605, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Thus, for example, signal-bearing medium 601 may be communicated via a wireless form of communication medium 605 (e.g., a wireless communication medium that complies with the IEEE 602.11 standard or other transmission protocols). One or more program instructions 602 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, the computing device of the computing device can be configured to provide various operations, functions, or actions in response to the program instructions 602 communicated to the computing device by one or more of the computer-readable medium 603, the computer-recordable medium 604, and / or the communication medium 605. It should be understood that the arrangement described herein is merely for illustrative purposes. Thus, it will be understood by those skilled in the art that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements can be omitted altogether according to the desired result. In addition, many of the described elements can be implemented as discrete or distributed components or in any appropriate combination and position to implement functional entities in conjunction with other components.

[0133] Figure 7 FIG. 1 is a possible logical structure diagram of the computer device 70 involved in the above embodiment provided in the embodiment of the present application. The computer device 70 may be Figure 5 The data flow control device 500 in the computer device 70 may include: a processor 701, a communication interface 702, a memory 703 and a bus 704. The processor 701, the communication interface 702 and the memory 703 are interconnected via the bus 704. In the embodiment of the present application, the processor 701 is used to execute Figure 2The communication interface 702 is used to support the computer device 70 in communication. For example, the communication interface 702 can perform the steps related to receiving or sending in the above-mentioned method embodiment. The memory 703 is used to store program code and data of the database server.

[0134] Among them, the processor 701 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute the various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. The bus 704 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0135] In another embodiment of the present application, a chip system is further provided. The chip system includes a processor for supporting the device for injecting time series data or the device for querying time series data to implement the above Figure 2 The data flow control method described in the embodiment. In one possible design, the chip system may also include a memory for managing the program instructions and data necessary for the application data. The chip system may be composed of a chip or may include a chip and other discrete devices.

[0136] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of this application.

[0137] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0138] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0142] The above is only a specific implementation of the embodiment of the present application, but the scope of protection of the embodiment of the present application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the embodiment of the present application, and they should be included in the scope of protection of the embodiment of the present application. Therefore, the scope of protection of the embodiment of the present application should be based on the scope of protection of the claims.

Claims

1. A data flow control method, characterized in that: Applied to a database system, the database system includes a host computer, the host computer is in communication with M standby computers and a client computer, where M is a positive integer, and the method includes: Obtaining a consensus-reaching efficiency when performing log backup between the master and the M backup machines, the consensus-reaching efficiency being related to at least one of the following information: a log bandwidth for reaching consensus and a time overhead for reaching consensus; reaching consensus comprising: the number of backup machines receiving confirmation acks from the master exceeding a threshold; the confirmation ack being sent by the backup machine after receiving the log copied by the master; the consensus-reaching efficiency being related to the log bandwidth for reaching consensus comprising: the consensus-reaching efficiency being positively correlated with the log bandwidth for reaching consensus; the consensus-reaching efficiency being related to the time overhead for reaching consensus comprising: the consensus-reaching efficiency being negatively correlated with the time overhead for reaching consensus; Data flow information of the host is determined based on the consensus reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus reaching efficiency.

2. The method according to claim 1, characterized in that The method further comprises: Obtain information about multiple logs that are agreed upon between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the log sizes of the multiple logs; A bandwidth of the consensus operation is determined according to log sizes of the multiple logs and a length of the historical sampling time period.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Obtain information about multiple logs on which consensus is reached between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the time spent on reaching consensus on each of the multiple logs; The time cost of the consensus operation is determined according to the time cost of reaching consensus on each log in the multiple logs.

4. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Obtaining the log playback bandwidth of the M backup machines when performing log backup between the master and the M backup machines; The determining of the data flow information of the host based on the consensus reaching efficiency includes: Based on the consensus reaching efficiency and the log replay bandwidth, data traffic information of the host is determined, wherein the traffic is positively correlated with the log replay bandwidth.

5. The method according to claim 4, characterized in that The obtaining of the log playback bandwidth of the M backup servers when performing log backup between the master server and the M backup servers includes: Obtain information about multiple logs on which the master and the M backup machines reach a consensus and complete data playback within a historical sampling period. The information about multiple logs on which the master and the M backup machines reach a consensus and complete data playback within the historical sampling period includes the log sizes of the multiple logs on which data playback has been completed. The log playback bandwidth is determined according to the log sizes of the multiple logs and the length of the historical sampling time period.

6. The method according to claim 2, characterized in that The historical sampling time period is a sampling period whose time interval with the current moment is less than a threshold among multiple sampling periods before the current moment; or, the historical sampling time period is multiple sampling periods before the current moment.

7. The method according to any one of claims 1 to 2, characterized in that: The data flow information includes flow control direction information and flow control step information. The flow control direction information is used to indicate the direction of change in the flow of data injected by the client received by the host, and the flow control step information is used to indicate the value of change in the flow of data injected by the client received by the host.

8. A data flow control device, characterized in that: Applied to a database system, the database system includes a host computer, the host computer is in communication with M standby computers and a client computer, where M is a positive integer, and the device includes: An acquisition module is configured to acquire a consensus-reaching efficiency when performing log backup between the master and the M backup machines, wherein the consensus-reaching efficiency is related to at least one of the following information: a log bandwidth for reaching consensus and a time overhead for reaching consensus; reaching consensus includes the number of backup machines receiving confirmation acks from the master exceeding a threshold; the confirmation ack is sent by the backup machine after receiving the log copied by the master; the consensus-reaching efficiency is related to the log bandwidth for reaching consensus, including: the consensus-reaching efficiency is positively correlated with the log bandwidth for reaching consensus; the consensus-reaching efficiency is related to the time overhead for reaching consensus, including: the consensus-reaching efficiency is negatively correlated with the time overhead for reaching consensus; A data flow determination module is used to determine the data flow information of the host based on the consensus reaching efficiency, wherein the data flow information is used to control the flow of data injected by the client received by the host, and the flow is positively correlated with the consensus reaching efficiency.

9. The device according to claim 8, characterized in that The acquisition module is further used to: Obtain information about multiple logs that are agreed upon between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the log sizes of the multiple logs; A bandwidth of the consensus operation is determined according to log sizes of the multiple logs and a length of the historical sampling time period.

10. The device according to any one of claims 8 to 9, characterized in that The acquisition module is further used to: Obtain information about multiple logs on which consensus is reached between the master and the M backup machines within a historical sampling period, where the information about the multiple logs includes the time spent on reaching consensus on each of the multiple logs; The time cost of the consensus operation is determined according to the time cost of reaching consensus on each log in the multiple logs.

11. The device according to any one of claims 8 to 9, characterized in that The acquisition module is further used to: Obtaining the log playback bandwidth of the M backup machines when performing log backup between the master and the M backup machines; The data flow determination module is specifically configured to: Based on the consensus reaching efficiency and the log replay bandwidth, data traffic information of the host is determined, wherein the traffic is positively correlated with the log replay bandwidth.

12. The device according to claim 11, characterized in that The acquisition module is specifically used to: Obtain information about multiple logs on which the master and the M backup machines reach a consensus and complete data playback within a historical sampling period. The information about multiple logs on which the master and the M backup machines reach a consensus and complete data playback within the historical sampling period includes the log sizes of the multiple logs on which data playback has been completed. The log playback bandwidth is determined according to the log sizes of the multiple logs and the length of the historical sampling time period.

13. The device according to claim 9, characterized in that The historical sampling time period is a sampling period whose time interval with the current moment is less than a threshold among multiple sampling periods before the current moment; or, the historical sampling time period is multiple sampling periods before the current moment.

14. The device according to any one of claims 8 to 9, characterized in that The data flow information includes flow control direction information and flow control step information. The flow control direction information is used to indicate the direction of change in the flow of data injected by the client received by the host, and the flow control step information is used to indicate the value of change in the flow of data injected by the client received by the host.

15. A data flow control device, characterized in that: The device includes at least one processor, a memory, and instructions stored in the memory and executable by the at least one processor, wherein the at least one processor executes the instructions to implement the steps of the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that A computer program is stored, and when the program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

17. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 7.

Images