Traffic playback method and apparatus, storage medium, and computer program product

Abstract

The present disclosure relates to the technical field of computers, and provides a traffic playback method and apparatus, a storage medium, and a computer program product. In the present disclosure, during traffic playback, for recorded statements, determinations in terms of a parallel execution condition are performed in the order of end times, so that different statements operating on the same data can only be executed serially, and statements having earlier end times are executed first. In this way, during target database playback, it can be ensured that the execution order of a plurality of operations on the same data is the same as the actual execution order during source database recording, thereby ensuring data consistency during target database playback and source database recording as much as possible, further ensuring as much as possible that the complexity of executing transactions in the target database is consistent with the actual complexity of executing transactions in the source database, and thus reducing test load distortion.

Description

Traffic playback methods, devices, storage media and computer program products

[0001] This disclosure claims priority to Chinese Patent Application No. 202411545132.2, filed on October 31, 2024, entitled "Flow Playback Method, Apparatus, Storage Medium and Computer Program Product", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and computer program product for playback of data. Background Technology

[0003] With the advent of the big data era, databases have been widely used. Traffic replay technology is a commonly used testing technique in the database field, used to test the database's hardware and software. The corresponding testing process can consist of traffic recording and traffic replay. This testing process can simulate the transaction execution process of the source database within the target database over a period of time, achieving the purpose of testing the target database. It eliminates the need for technicians to write additional test scripts and offers better realism than using test scripts.

[0004] In many cases, database testing requires replaying transactions recorded over a long period of time within a short timeframe; for example, recording for a week and replaying for 3 hours. To address this need, the time-window concurrency method is commonly used. This method involves dividing the recorded transactions into multiple groups based on their time slices, and executing these transactions in parallel within each group and serially between groups.

[0005] The time window concurrency method described above may result in discrepancies between the execution order of multiple operations on the same data and the actual order during the parallel execution of multiple transactions. This can lead to the data in the target database not being consistent with the data in the source database during actual operation. After data inconsistency occurs, the complexity of a transaction executed by the target database may be inconsistent with the actual complexity of the transaction executed by the source database, thus causing test load distortion. Summary of the Invention

[0006] This disclosure provides a traffic replay method, apparatus, storage medium, and computer program product to reduce test load distortion during traffic replay.

[0007] Firstly, a traffic replay method is provided. This method includes: firstly, obtaining at least one statement executed by the source database within a target time period, the end time of each statement, and the start time and / or the commit time of the transaction to which each statement belongs; then, sequentially obtaining statements from the at least one statement in chronological order of their end times, and determining whether each obtained statement and each reference statement in a reference queue meet the conditions for parallel execution. For each determination, based on specified information of the currently obtained statement and the current reference statement, or based on specified information of the associated statements of the currently obtained statement and the associated statements of the current reference statement, it is determined whether the currently obtained statement and the current reference statement meet the conditions for parallel execution. The reference queue includes... The target database contains currently executing statements and statements waiting to be executed. Specified information includes at least one of the following: statement start time, statement end time, and the commit time of the transaction to which the statement belongs. Parallel execution conditions are used to indicate that the currently obtained statement and the current reference statement belong to different transactions, and that the data to be operated on by the currently obtained statement when executed in the source database is not locked by the current reference statement. Finally, if the currently obtained statement and each reference statement in the reference queue satisfy the parallel execution conditions, the currently obtained statement is executed in parallel in the target database. If the currently obtained statement and at least one reference statement in the reference queue do not satisfy the parallel execution conditions, the currently obtained statement is executed serially in the target database.

[0008] In this embodiment of the disclosure, during the traffic replay process, the recorded statements are judged for parallel execution conditions according to their end time order. This ensures that different statements operating on the same data can only be executed serially, and the statement with the earlier end time will be executed first. In this way, during the target database replay process, it can be guaranteed that the execution order of multiple operations on the same data is the same as the actual execution order during the source database recording process. Therefore, it can ensure data consistency between the target database replay process and the source database recording process as much as possible. Furthermore, it can ensure that the complexity of the transaction executed in the target database is consistent with the actual complexity of the transaction executed in the source database as much as possible, thereby reducing test load distortion.

[0009] In one possible implementation, satisfying the parallel execution conditions includes satisfying a first parallel execution condition and a second parallel execution condition. The first parallel execution condition is that the currently retrieved statement and the currently referenced statement belong to different transactions. The second parallel execution condition is that at least one of the following sub-conditions is satisfied: the end time of the currently retrieved statement is earlier than the commit time of the transaction to which the currently referenced statement belongs; the commit time of the transaction to which the currently retrieved statement belongs is earlier than the commit time of the transaction to which the currently referenced statement belongs; the start time of the next statement in the transaction to which the currently retrieved statement belongs is earlier than the start time of the next statement in the transaction to which the currently referenced statement belongs, wherein the next statement in the transaction to which the currently retrieved statement belongs is a related statement of the currently retrieved statement, and the next statement in the transaction to which the currently referenced statement belongs is a related statement of the currently referenced statement.

[0010] In this way, the first parallel execution condition ensures that statements within the same transaction are executed sequentially, and the second parallel execution condition ensures that the data to be operated on by the currently retrieved statement when it is executed in the source database is not locked by the currently referenced statement, thus filtering out statements that can be executed in parallel.

[0011] In one possible implementation, if the currently retrieved statement is a read operation statement, the reference queue includes currently executed write operation statements and pending write operation statements in the target database; if the currently retrieved statement is a write operation statement, the reference queue includes currently executed read operation statements and write operation statements in the target database, as well as pending read operation statements and write operation statements.

[0012] In this way, based on the characteristics of the currently acquired statement (whether it is a read operation statement or a write operation statement), statements that may conflict with the currently acquired statement are selected from the currently executing statements and the statements waiting to be executed as reference statements, thereby reducing the number of reference statements and improving the efficiency of the judgment.

[0013] In one possible implementation, if the currently retrieved statement is a statement that operates on the target data table, the reference queue includes statements that are currently being executed and waiting to be executed in the target database that operate on the target data table.

[0014] In this way, based on the scope of the currently acquired statement (operating on the target data table), statements that may conflict with the currently acquired statement are selected as reference statements from the currently executing statements and statements waiting to be executed, thereby reducing the number of reference statements and improving the efficiency of the judgment.

[0015] In one possible implementation, the process of parallel execution of the currently acquired statement includes: adding the currently acquired statement to the execution statement queue, where statements in the execution statement queue are allowed to be executed in parallel.

[0016] By placing statements that can be executed in parallel into the execution statement queue, the system can dynamically adjust according to the current load, enabling the system to allocate and use resources more efficiently.

[0017] In one possible implementation, the process of serially executing the currently acquired statement includes: after at least one reference statement (which may be called a conflicting statement) in the reference queue that does not meet the parallel execution condition with the currently acquired statement has been executed, the currently acquired statement is added to the execution statement queue, and the statements in the execution statement queue are allowed to be executed in parallel.

[0018] In this way, for statements that cannot be executed in parallel, the correct execution order is ensured by waiting for conflicting statements to finish executing.

[0019] In one possible implementation, multiple statements can be retrieved from the execution statement queue to execute multiple statements in parallel.

[0020] In this way, by executing multiple statements in parallel, the utilization rate of system resources is improved and the execution efficiency of traffic replay is increased.

[0021] In one possible implementation, the start time, end time, and submission time are all logical times representing the sequence of events.

[0022] In this way, the processing of logical time is simpler, reducing the system's synchronization overhead.

[0023] In a second aspect, a traffic playback device is provided, the device comprising at least one module for implementing the method provided in the first aspect or any possible implementation thereof.

[0024] Thirdly, a computing device cluster is provided, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster performs the method provided by the first aspect or any possible implementation thereof.

[0025] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method provided by the first aspect or any possible implementation thereof.

[0026] Fifthly, a computer program product containing instructions is provided, which, when run by a cluster of computing devices, causes the cluster of computing devices to perform the method provided by the first aspect or any possible implementation thereof. Attached Figure Description

[0027] Figure 1 is a schematic diagram of a process for performing traffic replay according to an embodiment of this disclosure;

[0028] Figure 2 is a schematic diagram of a process for performing traffic replay according to an embodiment of this disclosure;

[0029] Figure 3 is a schematic diagram of a serial execution process provided in an embodiment of this disclosure;

[0030] Figure 4 is a schematic diagram of selecting a reference statement according to an embodiment of this disclosure;

[0031] Figure 5 is a schematic diagram of selecting a reference statement according to an embodiment of this disclosure;

[0032] Figure 6 is a schematic diagram of selecting a reference statement according to an embodiment of this disclosure;

[0033] Figure 7 is a schematic diagram of a transaction provided in an embodiment of this disclosure;

[0034] Figure 8 is a schematic diagram of a flow chart for performing traffic replay according to an embodiment of this disclosure;

[0035] Figure 9 is a schematic diagram of a flow chart for performing traffic replay according to an embodiment of this disclosure;

[0036] Figure 10 is a schematic diagram of a process for performing traffic replay according to an embodiment of this disclosure;

[0037] Figure 11 is a schematic diagram of the structure of a flow playback device provided in an embodiment of this disclosure;

[0038] Figure 12 is a schematic diagram of a computing device provided in an embodiment of this disclosure;

[0039] Figure 13 is a schematic diagram of a computing device cluster provided in an embodiment of this disclosure;

[0040] Figure 14 is a schematic diagram of a computing device cluster provided in an embodiment of this disclosure. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0042] The concepts involved in this disclosure are explained below:

[0043] database

[0044] A database can be considered a warehouse for storing data according to certain rules, built using computers. It has a very large storage capacity, capable of holding millions, tens of millions, hundreds of millions, or even more data entries.

[0045] The database's main engine (or main program) is responsible for executing various operations within the database. The data in the database is generally stored on the computer's storage media (such as a hard drive).

[0046] Databases are generally divided into relational databases and non-relational databases. Relational databases typically store data in tables. Non-relational databases typically store data in non-tabular formats, such as key-value pairs or graph structures.

[0047] Statement

[0048] Statements are commands for manipulating data in a database.

[0049] Statements can have various possible operation types. Operation types can include CRUD (Create, Read, Update, Query), etc. CRUD operations write new data to the database. CRUD operations delete existing data from the database. CRUD operations modify existing data in the database. Query operations retrieve data from the database that meets certain conditions. CRUD operations can be collectively referred to as write operations, and statements that perform write operations are called write operation statements; Query operations can be referred to as read operations, and statements that perform read operations are called read operation statements.

[0050] In this embodiment of the disclosure, a structured query language (SQL) statement is used as an example to illustrate some aspects. Other cases are similar and will not be described in detail.

[0051] transactions

[0052] A transaction is the basic unit of actual database operations. A transaction can include one or more statements; that is, a transaction can consist of a series of operations. A transaction is considered an indivisible unit of operation execution, meaning that the operations within a transaction are either all executed or none are executed.

[0053] The process of creating and executing a transaction can be as follows:

[0054] Users initiate a transaction using the `begin` statement. Then, users can enter statements outlining their desired operations. Each statement entered is executed by the database in memory. After all desired operations are completed, users can enter a `commit` statement. Upon receiving the `commit` statement, the database persists the completed data from memory to disk.

[0055] Traffic replay technology

[0056] Traffic replay technology is a commonly used testing technique in the database field, used to test the database's hardware and software. For example, testing is required before replacing the main engine of a database (or upgrading the main engine version), or before replacing the hardware devices of a database.

[0057] The corresponding testing process can consist of traffic recording and traffic playback. Traffic recording refers to recording the transactions executed in the source database over a period of time (such as one day, one month, etc.) (traffic recording can be considered as recording the actual transaction flow in the source database); traffic playback refers to executing all the recorded transactions in the target database. Before playback, the data in the target database is exactly the same as the data in the source database at the start of recording. During traffic playback, technicians can monitor the operation of the target database to achieve various testing objectives, such as monitoring whether errors occur during processing, whether the processing latency of each statement meets the standard, and whether the real-time utilization of hardware resources meets the standard, etc.

[0058] This method simulates the transaction execution process of the source database over a period of time within the target database, achieving the purpose of testing the target database without requiring technicians to write additional test scripts, and it offers better realism than using test scripts.

[0059] Source database and target database

[0060] In traffic replay technology, the source database is used for traffic recording, and the target database is used for traffic replay. The source database provides the actual transaction flow, which is then used to test the target database.

[0061] To ensure data consistency between the target database testing process and the source database recording process, it is generally necessary that the data in the target database is exactly the same as the data in the source database at the start of recording before playback. Therefore, the data in the target database can typically be a backup of the data in the source database at the start time of recording.

[0062] Depending on the specific testing scenario, the relationship between the source and target databases can have several possibilities:

[0063] Scenario 1: Conduct main engine testing, for example, testing a new version of the main engine.

[0064] In this case, the source database and the target database can have the same hardware configuration but use different main engines.

[0065] Scenario 2: Conduct hardware testing, for example, testing new hardware before a hardware upgrade.

[0066] In this scenario, the source and target databases can have different hardware configurations but use the same main engine.

[0067] Scenario 3: Simultaneous testing of the main engine and hardware, such as testing before database migration. One possible specific scenario is that the database was originally maintained by the user, but was subsequently handed over to the cloud service provider for maintenance.

[0068] In this case, the source database and the target database can have different hardware configurations and use different main engines.

[0069] Scenario 4: Conduct stress tests without changing the main engine and hardware configuration.

[0070] In this case, the source database and the target database can have the same hardware configuration and use the same main engine.

[0071] Locking mechanism

[0072] Locking is a crucial mechanism in databases, primarily used to prevent data corruption caused by simultaneous operations on the same data by different transactions. The general operation of locking is as follows: when a transaction operates on certain data, it locks that data to restrict other transactions from accessing it. Specifically, there are several scenarios: when a transaction performs a write operation on certain data, locking that data to prevent other transactions from writing or reading it is called a write lock; when a transaction performs a read operation on certain data, locking that data to prevent other transactions from writing to it is called a read lock.

[0073] After a transaction has finished operating on the data, it can release the lock, allowing other transactions to operate on the data.

[0074] A transaction may involve operations on multiple data. Generally, the locks on these data are released when the transaction is committed. The processing of locking these data can be done uniformly or distributed across different statements.

[0075] Two-phase locking (2PL) protocol

[0076] 2PL is a classic database concurrency control technique used to solve data consistency problems when transactions are executed concurrently. The main purpose of 2PL is to ensure the isolation and consistency of transactions through locking mechanisms, thus avoiding data conflicts during concurrent execution.

[0077] The 2PL protocol divides transaction execution into two phases:

[0078] During the locking phase, a transaction can request any lock it needs (i.e., acquire a lock), but cannot release any locks.

[0079] During the unlocking phase, after a transaction has completed all necessary operations, it begins to release all held locks (i.e., unlock). Once a transaction enters the unlocking phase, it cannot request new locks.

[0080] When a transaction needs to read or write data, it must first acquire the corresponding lock. If the data needed by a transaction is already locked by another transaction, then the transaction must wait until the lock is released. If the waiting time is too long (known as lock timeout) or a circular wait occurs (known as deadlock), the system may take measures to resolve the issue (such as rolling back a transaction, i.e., canceling all operations already performed by that transaction).

[0081] Traffic replay technology is a commonly used testing technique in the database field, used to test the database's hardware and software. The corresponding testing process can consist of traffic recording and traffic replay.

[0082] In many cases, database testing requires replaying transactions recorded over a long period in a short time, for example, recording for a week and replaying for 3 hours. This need may be to test the database's resilience or to improve testing efficiency. For this requirement, the time window concurrency method is generally used. The process of replaying traffic in this method involves dividing the recorded transactions into multiple groups according to their time slices, and executing these transactions in parallel within each group and serially between groups. That is, multiple transactions within the same group are executed in parallel, without waiting for one transaction to complete before starting another. For different groups, all transactions within one group are executed first, in chronological order, before starting the next group.

[0083] The aforementioned time-window concurrency method, when executing multiple transactions in parallel, may result in discrepancies between the execution order of multiple operations on the same data and the actual order. This is because after adding multiple statements to the execution statement queue, the processor cannot guarantee that these statements will be executed in the order they appear in the queue; the execution order is somewhat random, meaning it's impossible to pre-arrange the execution order of these parallel statements. This can lead to inconsistencies between the data in the target database and the data in the source database during actual operation. For example, if the source database performs an increment operation (+1) and a multiplication operation (+2) on a certain data during recording, and then reverses the order of these operations during playback, the data in the target database playback will be inconsistent with the data in the source database recording. Once this inconsistency occurs, the complexity of a subsequent transaction in the target database may differ from the actual complexity in the source database, resulting in distorted test load.

[0084] This disclosure provides a traffic replay method. The method determines whether operations can be executed in parallel on a statement-by-statement basis. The goal is to ensure that the order of multiple operations on the same data is not disordered and to maximize concurrency. The execution entity of this method can be a server of the target database or a separate device for performing traffic replay. The processing flow of this method is shown in Figure 1, including the following steps:

[0085] Step 101: Obtain at least one statement executed by the source database within the target time period, the end time of each statement, and the start time of each statement and / or the commit time of the transaction to which each statement belongs.

[0086] Before recording and replaying database traffic, technicians first determine the target time period to be replayed, which is usually determined based on testing needs. For example, if a website plans to launch a new promotional activity during the upcoming "Double Eleven" shopping festival, and expects a large number of users to visit the website and make purchases simultaneously, it needs to ensure that the existing database system can withstand the pressure of high concurrency requests and remain stable under prolonged high load. To test the application performance of the database system during peak hours, the target time period could be a period from a past peak period, such as the entire day of last year's "Double Eleven" (let's assume it's from 0:00 to 23:59 on November 11th).

[0087] To perform database traffic replay, ensure that the target database and the source database are consistent. The following steps can be taken:

[0088] First, back up the data in the source database at the start of the target time period.

[0089] Then, initialize the target database using the backup data. Ensure that the initial state of the target database is consistent with the state of the source database at the aforementioned start time.

[0090] Next, obtain the traffic data for all database operations within the target time period. The obtained traffic data may include the executed statements, the transaction to which each statement belongs, the end time of each statement, and the start time of each statement and / or the commit time of each transaction. In addition, the traffic data may also include at least one of the following: the operation type of each statement (such as write or read operation), and the data source information of each statement (such as the table or graph to which the data belongs).

[0091] The statements (taking SQL statements as an example) can be of types such as INSERT, UPDATE, DELETE, and SELECT. Optionally, BEGIN statements may not be recorded. The statement's end time, start time, and transaction commit time can use logical times indicating sequence. These logical times can be any set of identifiers defined by order, such as numerical codes, as shown in Figure 7, where logical times in numerical form represent the statement's end time. Alternatively, the statement's end time, start time, and transaction commit time can also be clock times.

[0092] The aforementioned traffic data is typically recorded in database operation logs. This data can be obtained by reading the database management system's logs (this process can be performed by the source database server or other devices). Besides using the database's built-in logging functionality, third-party tools can also be used to record traffic data. For example, some database management systems provide SQL tracing tools or database proxy tools (such as ProxySQL, Pgbouncer, etc.), which can capture SQL statements and execution time information.

[0093] After obtaining the traffic data, it can be preprocessed, such as formatting it into a specified format and removing abnormal data. Then, the preprocessed data can be recorded.

[0094] Step 102: Sequentially retrieve statements from the at least one set of statements according to their end times, and determine whether each retrieved statement satisfies the parallel execution condition with each reference statement in the reference queue. The reference queue includes currently executing statements and statements waiting to be executed in the target database. For each determination, based on specified information about the currently retrieved statement and the current reference statement, or based on specified information about the associated statements of the currently retrieved statement and the associated statements of the current reference statement, determine whether the currently retrieved statement and the current reference statement satisfy the parallel execution condition.

[0095] The specified information includes at least one of the following: the start time of the statement, the end time of the statement, and the commit time of the transaction to which the statement belongs. The parallel execution condition indicates that the currently retrieved statement and the currently referenced statement belong to different transactions, and the data that the currently retrieved statement needs to operate on when executed in the source database is not locked by the currently referenced statement (i.e., there is no lock conflict between the currently retrieved statement and the currently referenced statement). Statements waiting to be executed can include two categories: one is statements that are determined to be able to be executed in parallel but have not yet been executed due to insufficient system resources; the other is statements that are determined not to be able to be executed in parallel and need to wait for serial execution.

[0096] First, all statements retrieved from the source database can be sorted by their end time. Then, following this sorting, that is, retrieving statements one by one from earliest to latest end time. Each retrieved statement (which can be called the target statement) is added to a pre-established buffer. This buffer records all statements that have started undergoing "parallel execution condition" checks but have not yet finished executing; in other words, the buffer includes the target statement, all statements waiting to be executed, and statements currently being executed. Next, statements other than the target statement are treated as reference statements in the buffer, forming a reference queue. A conflict check is then performed between the target statement and each reference statement in the reference queue, that is, to determine whether the target statement meets the parallel execution condition with each reference statement.

[0097] Optionally, at this time, all statements in the cache except the target statement can be selected to form a reference queue, or statements that meet certain conditions can be selected from the cache to form a reference queue. This will be described in detail in the following content of the embodiments of this disclosure.

[0098] The aforementioned "statements for retrieving records in order of their end time" and "parallel execution conditions" are to ensure that, during playback, multiple operations on any data can be executed in the order they were recorded.

[0099] For example, as shown in Figure 2, statements A1, B1, C1, D1, and E1 are sorted according to their end times. Then, statements are retrieved one by one in order of their end times (i.e., in the order of A1, B1, C1, D1, and E1). For each retrieved statement (taking A1 as an example), a reference statement is selected from all statements awaiting execution and those currently being executed. Statement A1 is then compared with each reference statement to determine if it meets the conditions for parallel execution. If statement A1 meets the parallel execution conditions with all reference statements, it is executed in parallel. If statement A1 does not meet the parallel execution conditions with at least one reference statement, it is executed sequentially. Then, the next statement (B1) is retrieved, and the above process is repeated.

[0100] Step 103: If the currently obtained statement and each reference statement in the reference queue meet the conditions for parallel execution, perform parallel execution processing on the currently obtained statement in the target database.

[0101] If both the target statement and the reference statements in the aforementioned cache meet the conditions for parallel execution, then the target statement can be executed in parallel with these reference statements. The target statement can then be added to the execution statement queue of the target database. Any number of statements in the execution statement queue are allowed to be executed in parallel. At this time, there may still be several statements in the target database that have completed the above checks and are in the process of execution, which will consume execution resources to varying degrees. If there are sufficient execution resources, the target statement can be executed directly; if there are insufficient execution resources, the target statement can wait in the execution statement queue for execution.

[0102] This parallel execution can make full use of the system's multi-core, multi-processor, or multi-device resources to improve execution efficiency.

[0103] Step 104: If the currently obtained statement does not meet the parallel execution condition with at least one reference statement in the reference queue, the currently obtained statement is executed serially in the target database.

[0104] If the target statement does not meet the parallel execution conditions with one or more reference statements in the aforementioned cache (reference statements that do not meet the parallel execution conditions can be called conflicting statements), it means that the target statement cannot be executed in parallel with this conflicting statement. In this case, the target statement and the conflicting statement can be executed serially. The execution status of conflicting statements can be monitored. When all conflicting statements have been executed, the target statement is added to the execution statement queue of the target database. Any number of statements in the execution statement queue are allowed to be executed in parallel. At this time, there may still be several statements in the target database that have completed the above judgment and are in the process of execution, which will occupy execution resources to varying degrees. If there are enough execution resources, the target statement can be executed directly. If there are not enough execution resources, the target statement can wait in the execution statement queue for execution.

[0105] In actual processing, the processor can retrieve multiple statements from the above-mentioned execution statement queue to execute these multiple statements in parallel.

[0106] For example, as shown in Figure 3, assume the target statement is statement A1, and the reference statements are statements X1, X2, X4, Xn, Y2, Y4, and Yn. Statement A1 does not satisfy the parallel execution condition with statements X1, X4, and Y4 in the reference queue; that is, statements X1, X4, and Y4 are conflicting statements with statement A1. Record the conflicting statements of statement A1, and after all statements X1, X4, and Y4 have been executed, add statement A1 to the execution statement queue.

[0107] In this embodiment of the disclosure, during the traffic replay process, the recorded statements are judged for parallel execution conditions according to their end time order. This ensures that different statements operating on the same data can only be executed serially, and the statement with the earlier end time will be executed first. In this way, during the target database replay process, it can be guaranteed that the execution order of multiple operations on the same data is the same as the actual execution order during the source database recording process. Therefore, it can ensure data consistency between the target database replay process and the source database recording process as much as possible. Furthermore, it can ensure that the complexity of the transaction executed in the target database is consistent with the actual complexity of the transaction executed in the source database as much as possible, thereby reducing test load distortion.

[0108] To improve the efficiency of traffic replay processing, certain conditions can be set for the selection of reference statements. The corresponding processing method for selecting reference statements can be as follows:

[0109] Method 1

[0110] If the currently retrieved statement is a read operation statement, a reference queue is formed using the currently executing write operation statement and the write operation statement waiting to be executed in the target database as reference statements; if the currently retrieved statement is a write operation statement, a reference queue is formed using the currently executing read operation statement and the write operation statement, as well as the read operation statement and the write operation statement waiting to be executed in the target database as reference statements.

[0111] The core idea behind this approach is:

[0112] For read operations, only the currently executing write operation statement and the write operation statement waiting to be executed are considered as reference statements.

[0113] For write operations, all currently executing read and write operations, as well as all read and write operations waiting to be executed, need to be considered as reference statements.

[0114] This is because read operations do not modify data, meaning multiple read operations can run concurrently without interfering with each other. Therefore, read operations do not require a "parallel execution condition" check. Write operations, on the other hand, typically involve modifying data in the database, such as inserting new data, updating existing data, or deleting existing data. These operations change the database state, potentially affecting the data read by ongoing read operations or interfering with other write operations. Therefore, read and write operations may not be able to execute in parallel and require a "parallel execution condition" check. Similarly, write operations may not be able to execute in parallel with read operations or other write operations and also require a "parallel execution condition" check.

[0115] During traffic replay, the type of SQL statement can be determined by examining keywords within the statement. For example, read operations primarily include SELECT statements, while write operations include INSERT, UPDATE, and DELETE statements. Alternatively, database management tools can be used during traffic recording to assist in identifying and recording the operation type of the SQL statement.

[0116] For example, as shown in Figure 4, assuming the target statement is statement A1, and statement A1 is a read operation statement, then all write operation statements in the statements being executed and those waiting to be executed are taken as reference statements, namely statements X1, X2, X4, Xn, Y2, Y4, and Yn in Figure 4.

[0117] For example, as shown in Figure 5, assuming the target statement is statement A1, and statement A1 is a write operation statement, then all read operation statements and write operation statements in the currently executing statements, as well as all read operation statements and write operation statements in the statements waiting to be executed, are used as reference statements, namely statements X1, X2, X3, X4, Xn, Y1, Y2, Y3, Y4, and Yn in Figure 5.

[0118] Method 2

[0119] If the currently executing statement operates on the target data table, a reference queue is formed by using currently executing and waiting statements in the target database that operate on the target data table as reference statements.

[0120] If the currently obtained statement operates on a specific data table, you can focus only on the statements that operate on that data table, selecting only the statements currently being executed and those waiting to be executed that operate on that data table as reference statements.

[0121] For example, as shown in Figure 6, assuming the target statement is statement A1, which is a statement that operates on Table 1, then the statements that operate on Table 1 among the currently executing statements and the statements waiting to be executed are taken as reference statements, namely statements X1, X2, X4, Xn, Y2, Y4 and Yn in Figure 6.

[0122] Method 3

[0123] If the currently executing statement operates on the target graph data, a reference queue is formed by using currently executing and waiting statements in the target database that operate on the target graph data as reference statements.

[0124] If the currently obtained statements operate on specific graph data, you can focus only on the statements that operate on that graph data. Select only the statements currently being executed and those waiting to be executed that operate on the graph data as reference statements.

[0125] The following is a detailed explanation of the above-mentioned conditions for parallel execution:

[0126] The corresponding parallel execution conditions may include a first parallel execution condition and a second parallel execution condition. Meeting both the first and second parallel execution conditions (both conditions being met simultaneously) constitutes a parallel execution condition.

[0127] The first condition for parallel execution can be that the currently retrieved statement (which can be called the target statement) and the currently referenced statement belong to different transactions.

[0128] During traffic recording, the transaction to which the statement belongs is recorded. Based on this record, the transaction to which the target statement and the current reference statement belong can be determined. If the two transactions are the same, the first parallel execution condition is not met; if the two transactions are different, the first parallel execution condition is met. This condition ensures that operations within the same transaction are executed sequentially, preventing the order of operations within a transaction from being disrupted due to parallel execution.

[0129] The second parallel execution condition is based on the 2PL concurrency control theory. It analyzes the lock contention relationship between statements by using at least one of the following information: the commit time of the transaction to which the statement belongs, the start time of the statement, and the end time of the statement. The second parallel execution condition is to satisfy at least one of the following sub-conditions:

[0130] Sub-condition one: The end time of the currently retrieved statement is earlier than the commit time of the transaction to which the currently referenced statement belongs. Here, the transaction commit time is the end time of the transaction's commit statement.

[0131] If the data to be operated on by the target statement is locked by the current reference statement, the lock will not be released until the transaction to which the current reference statement belongs is committed. Only after that can the target statement acquire the lock on the data and further operate on the data. Therefore, in this case, the end time of the target statement will definitely be later than the end time of the transaction to which the current reference statement belongs.

[0132] In other words, if the target statement has already been executed before the transaction to which the current reference statement belongs is committed, it means that the data operated on by the target statement cannot be locked by the current reference statement, that is, there is no lock conflict between the target statement and the current reference statement.

[0133] Sub-condition two: The commit time of the transaction to which the currently retrieved statement belongs is earlier than the commit time of the transaction to which the currently referenced statement belongs.

[0134] The end time of the target statement should be earlier than the end time of the transaction to which the target statement belongs. Therefore, if the commit time of the transaction to which the target statement belongs is earlier than the commit time of the transaction to which the current reference statement belongs, then the end time of the target statement must be earlier than the commit time of the transaction to which the current reference statement belongs. In this case, combined with the conclusion in sub-condition one, the data operated on by the target statement cannot be locked by the current reference statement, that is, there is no lock conflict between the target statement and the current reference statement.

[0135] Sub-condition three: The start time of the next statement in the transaction to which the currently retrieved statement belongs is earlier than the start time of the next statement in the transaction to which the currently referenced statement belongs.

[0136] The end time of the target statement should be earlier than the start time of the next statement in the transaction to which the target statement belongs. The start time of the next statement in the transaction to which the current reference statement belongs should be earlier than the commit time of the current reference statement's transaction. Therefore, if the start time of the next statement in the transaction to which the target statement belongs is earlier than the start time of the next statement in the transaction to which the current reference statement belongs, then the end time of the target statement must be earlier than the commit time of the current reference statement's transaction. In this case, combined with the conclusion in sub-condition one, the data operated on by the target statement cannot be locked by the current reference statement; that is, there is no lock conflict between the target statement and the current reference statement.

[0137] The following concrete example illustrates the application of these conditions:

[0138] For example, as shown in Figure 7, suppose there are three transactions: transaction A, transaction B, and transaction C. Transaction A contains 5 statements: A1, A2, A3, A4, and commit. Transaction B contains 3 statements: B1, B2, and commit. Transaction C contains 3 statements: C1, C2, and commit.

[0139] Scenario 1: The currently retrieved statement is C1, and the currently referenced statement is A1. C1 and A1 belong to different transactions, satisfying the first condition for parallel execution. The end time of C1 is earlier than the commit time of the transaction to which A1 belongs, satisfying the first sub-condition of the second condition for parallel execution. Therefore, C1 and A1 can be executed in parallel.

[0140] Scenario 2: The currently retrieved statement is A2, and the current reference statements are A1 and C1. A2 and A1 belong to the same transaction, so the first parallel execution condition is not met. A2 and C1 belong to different transactions, thus meeting the first parallel execution condition. The end time of A2 is earlier than the commit time of the transaction to which C1 belongs, satisfying sub-condition one of the second parallel execution condition. Therefore, A2 and C1 can be executed in parallel. Thus, A2 is executed serially, and the fact that A2 and A1 do not meet the parallel execution condition is recorded. After A1 finishes execution, A2 is added to the execution statement queue.

[0141] Scenario 3: The currently retrieved statement is B1, and the currently referenced statements are A2, A3, and C2 (i.e., A1 and C1 have already completed execution). B1 and A2 belong to different transactions, satisfying the first parallel execution condition. The end time of B1 is earlier than the commit time of the transaction to which A2 belongs, satisfying sub-condition one of the second parallel execution condition. Therefore, B1 and A2 can be executed in parallel. Similarly, B1 and A3 can be executed in parallel. B1 and C2 belong to different transactions, satisfying the first parallel execution condition. The end time of B1 is later than the commit time of the transaction to which C2 belongs, not satisfying sub-condition one of the second parallel execution condition. The commit time of transaction B to which B1 belongs is later than the commit time of transaction C, not satisfying sub-condition two of the second parallel execution condition. The start time of the next statement B2 of transaction B is also later than the start time of the next statement commit of transaction C, not satisfying sub-condition three of the second parallel execution condition. Therefore, B1 is executed serially, and it is recorded that B1 and C2 do not meet the parallel execution condition. After C2 completes execution, B1 is added to the execution statement queue.

[0142] In one possible implementation of this disclosure, the traffic replay method can be performed collaboratively by two executable programs. These programs can include a first executable program and a second executable program. The first executable program is the main program for recording and replaying, and the second executable program is a program for determining whether statements can be executed in parallel. Both the first and second executable programs can run on the client side or in the cloud. In both cases, the first executable program can call the second executable program via a library reference. Alternatively, the first executable program can run on the client side, and the second executable program can run in the cloud. In this case, the first executable program can remotely call the second executable program via network protocols or other means. The first executable program can run on the server of the target database or on other devices outside of that server.

[0143] During the entire traffic replay process, there are multiple ways to divide the processing responsibilities between the first and second execution programs.

[0144] The following describes one division of labor between the first and second execution programs. The corresponding traffic replay processing flow can be shown in Figure 8, including the following steps:

[0145] Step 801: The first execution program obtains at least one statement executed by the source database within the target time period, the transaction to which each statement belongs, the end time of each statement, and the start time of each statement and / or the commit time of each transaction.

[0146] Step 802: The first execution program notifies the second execution program through the first interface of at least one of the following three time information: the transaction to which each statement belongs, the end time of each statement, the start time of each statement, and the commit time of each transaction.

[0147] When sending the relevant information for each statement, the statement can be represented by a unique identifier or by the complete statement content.

[0148] Step 803: The first execution program retrieves statements sequentially from at least one of the above statements according to the order of their end times. For each statement retrieved, the program notifies the second execution program of the currently retrieved statement through the second interface.

[0149] When notifying the currently retrieved statement (hereinafter referred to as the target statement), the target statement can be represented by a unique identifier or by the complete statement content.

[0150] Step 804: The second execution program determines whether the currently obtained statement and each reference statement in the reference queue meet the conditions for parallel execution. The reference queue includes statements currently being executed and statements waiting to be executed in the target database.

[0151] The currently executing statements and the statements waiting to be executed include statements that have completed the parallel execution condition judgment but have not received an execution completion notification (which will be introduced later).

[0152] For the processing of each judgment, please refer to step 102 and its related explanations, which will not be repeated here.

[0153] The second executor can pre-establish a buffer. Whenever it receives a statement from the first executor via the second interface, it adds that statement to the buffer. Whenever it receives a statement from the first executor via the third interface (indicating that the statement has finished executing), it removes that statement from the buffer. Thus, the buffer contains both the target statement and all reference statements (i.e., the currently executing statement and statements waiting to be executed).

[0154] Step 805: If the currently acquired statement and each reference statement in the reference queue meet the conditions for parallel execution, the second execution program sends an execution notification corresponding to the currently acquired statement to the first execution program; if the currently acquired statement and at least one reference statement in the reference queue do not meet the conditions for parallel execution, the second execution program records the at least one reference statement, and after all at least one reference statement has been executed, sends an execution notification corresponding to the currently acquired statement to the first execution program.

[0155] Step 806: Whenever the first execution program receives an execution notification corresponding to any statement, it executes the statement in the target database.

[0156] Executing this statement means adding it to the statement execution queue.

[0157] Step 807: Whenever the first execution program detects that any statement in the target database has been executed, it sends a notification of the completion of the execution of that statement to the second execution program through the third interface.

[0158] This step can occur at any time during actual operation and has no necessary sequential relationship with other steps.

[0159] Some steps in the above process are similar to those in the process shown in Figure 1. For the corresponding processing, please refer to the relevant descriptions of the process shown in Figure 1; they will not be repeated here. The first interface, second interface, and third interface in the above process can be different interfaces or the same interface.

[0160] The following describes another division of labor between the first and second execution programs. The corresponding traffic replay processing flow can be shown in Figure 9, including the following steps:

[0161] Step 901: The first execution program obtains at least one statement executed by the source database within the target time period, the transaction to which each statement belongs, the end time of each statement, and the start time of each statement and / or the commit time of each transaction.

[0162] Step 902: The first execution program notifies the second execution program through the first interface of at least one statement, the transaction to which each statement belongs, the end time of each statement, and the start time of each statement and / or the commit time of each transaction.

[0163] Step 903: The second execution program sequentially retrieves statements from at least one of the above statements according to the order of their end times. For each retrieved statement, it determines whether the currently retrieved statement and each reference statement in the reference queue meet the conditions for parallel execution. The reference queue includes statements currently being executed and statements waiting to be executed in the target database.

[0164] The currently executing statements and the statements waiting to be executed include statements that have completed the parallel execution condition judgment but have not received an execution completion notification (the notification will be introduced in the following content).

[0165] For the processing of each judgment, please refer to step 102 and its related explanations, which will not be repeated here.

[0166] The second executor can sort the statements received through the first interface according to their completion time, and then retrieve each statement as a target statement. The second executor can pre-establish a buffer. Whenever a statement is retrieved as a target statement, it is added to the buffer. Whenever the first executor receives a statement notification (indicating that the statement has completed execution) through the second interface, it removes the statement from the buffer. Thus, the buffer contains both target statements and all reference statements (i.e., the currently executing statement and statements waiting to be executed).

[0167] Step 904: If the currently acquired statement and each reference statement in the reference queue meet the conditions for parallel execution, the second execution program sends an execution notification corresponding to the currently acquired statement to the first execution program; if the currently acquired statement and at least one reference statement in the reference queue do not meet the conditions for parallel execution, the second execution program records the at least one reference statement, and after all at least one reference statement has been executed, sends an execution notification corresponding to the currently acquired statement to the first execution program.

[0168] Step 905: Whenever the first execution program receives an execution notification corresponding to any statement, it executes the statement in the target database.

[0169] Executing this statement means adding it to the statement execution queue.

[0170] Step 906: Whenever the first execution program detects that any statement in the target database has been executed, it sends a notification of the completion of the execution of that statement to the second execution program through the second interface.

[0171] Some steps in the above process are similar to those shown in Figures 8 and 1. For details on the corresponding processes, please refer to the relevant descriptions in Figures 8 and 1; they will not be repeated here. The first interface and the second interface in the above process can be different interfaces or the same interface.

[0172] The following describes another division of labor between the first and second execution programs. The corresponding traffic replay processing flow can be shown in Figure 10, including the following steps:

[0173] Step 1001: The first execution program obtains at least one statement executed by the source database within the target time period, the transaction to which each statement belongs, the end time of each statement, and the start time of each statement and / or the commit time of each transaction.

[0174] Step 1002: The first execution program notifies the second execution program through the first interface of at least one of the following three time information: the transaction to which each statement belongs, the end time of each statement, the start time of each statement, and the commit time of each transaction.

[0175] Step 1003: The first execution program sequentially retrieves statements from at least one of the above statements according to the order of their end times. For each retrieved statement, it notifies the second execution program of the currently retrieved statement through the second interface.

[0176] Step 1004: The second execution program determines whether the currently acquired statement and each reference statement in the reference queue meet the conditions for parallel execution. The reference queue includes statements currently being executed and statements waiting to be executed in the target database.

[0177] The currently executing statements and the statements waiting to be executed include statements that have completed the parallel execution condition judgment but have not received a completion notification.

[0178] For the processing of each judgment, please refer to step 102 and its related explanations, which will not be repeated here.

[0179] Step 1005: If the currently acquired statement and each reference statement in the reference queue meet the parallel execution conditions, the second execution program sends a parallel execution notification corresponding to the currently acquired statement to the first execution program; if the currently acquired statement and at least one reference statement in the reference queue do not meet the parallel execution conditions, the second execution program records the at least one reference statement and sends a serial execution notification corresponding to the currently acquired statement to the first execution program.

[0180] The serial execution notification carries at least one of the aforementioned reference statements.

[0181] That is, regardless of whether the target statement meets the conditions for parallel execution, the second executor can return the judgment result (parallel execution notification or serial execution notification) to the first executor after completing the judgment. The first executor can then perform different processing on the target statement according to the parallel execution notification or serial execution notification.

[0182] Step 1006: Whenever the first execution program receives a parallel execution notification corresponding to any statement, it executes the statement in the target database; whenever the first execution program receives a serial execution notification corresponding to any statement, it waits for all reference statements in the serial execution notification to be executed before executing the statement in the target database.

[0183] Executing this statement means adding it to the statement execution queue.

[0184] Step 1007: Whenever the first execution program detects that any statement in the target database has been executed, it sends a notification of the completion of the execution of that statement to the second execution program through the third interface.

[0185] Some steps in the above process are similar to those shown in Figures 8 and 1, and can be found in the relevant descriptions of the processes shown in Figures 8 and 1, which will not be repeated here. The first interface, second interface, and third interface in the above process can be different interfaces or the same interface.

[0186] In the above three processes, when the first executable program and the second executable program are on the same device, the interface used between the first executable program and the second executable program (such as the first interface and the second interface mentioned above) can be the internal communication interface of the device; when the first executable program and the second executable program are not on the same device, the interface used between the first executable program and the second executable program (such as the first interface and the second interface mentioned above) can be the network communication interface.

[0187] Based on the same technical concept, this disclosure provides a traffic replay device, which is deployed on a database server, as shown in FIG11. The device includes:

[0188] The acquisition module 1110 is used to acquire at least one statement executed by the source database within the target time period, the end time of each statement, and the start time of each statement and / or the commit time of the transaction to which each statement belongs. Specifically, it can implement the processing function of step 101 above, as well as other implicit steps.

[0189] The judgment module 1120 is used to sequentially retrieve statements from at least one set of statements according to their end times, and to determine whether each retrieved statement and each reference statement in the reference queue meet the conditions for parallel execution. The reference queue includes currently executing statements and statements waiting to be executed in the target database. For each judgment, based on specified information of the currently retrieved statement and the current reference statement, or based on specified information of the associated statements of the currently retrieved statement and the associated statements of the current reference statement, it determines whether the currently retrieved statement and the current reference statement meet the conditions for parallel execution. The specified information includes at least one of the statement's start time, end time, and the commit time of the transaction to which the statement belongs. The parallel execution condition indicates that the currently retrieved statement and the current reference statement belong to different transactions, and that the data to be operated on by the currently retrieved statement when executed in the source database is not locked by the current reference statement. Specifically, this can implement the processing function of step 102 above, as well as other implicit steps.

[0190] Execution module 1130 is used to perform parallel execution processing on the currently acquired statement in the target database if the parallel execution conditions are met between the currently acquired statement and each reference statement in the reference queue. If the parallel execution conditions are not met between the currently acquired statement and at least one reference statement in the reference queue, the currently acquired statement is executed serially in the target database. Specifically, it can implement the processing functions of steps 103 and 104 above, as well as other implicit steps.

[0191] In one possible implementation, satisfying the parallel execution conditions includes satisfying both the first and second parallel execution conditions.

[0192] The first condition for parallel execution is that the currently retrieved statement and the currently referenced statement belong to different transactions.

[0193] The second parallel execution condition is that at least one of the following sub-conditions is met:

[0194] The end time of the currently retrieved statement is earlier than the commit time of the transaction to which the currently referenced statement belongs.

[0195] The commit time of the transaction to which the currently retrieved statement belongs is earlier than the commit time of the transaction to which the currently referenced statement belongs.

[0196] The start time of the next statement in the transaction to which the currently retrieved statement belongs is earlier than the start time of the next statement in the transaction to which the currently referenced statement belongs. The next statement in the transaction to which the currently retrieved statement belongs is the related statement of the currently retrieved statement, and the next statement in the transaction to which the currently referenced statement belongs is the related statement of the currently referenced statement.

[0197] In one possible implementation, if the currently retrieved statement is a read operation statement, the reference queue includes currently executing write operation statements and pending write operation statements in the target database.

[0198] If the currently retrieved statement is a write operation statement, the reference queue includes currently executing read and write operations in the target database, as well as read and write operations waiting to be executed.

[0199] In one possible implementation, if the currently retrieved statement is a statement that operates on the target data table, the reference queue includes statements that are currently being executed and those waiting to be executed in the target database that operate on the target data table.

[0200] In one possible implementation, execution module 1130 is used for:

[0201] The currently retrieved statement is added to the execution statement queue, and the statements in the execution statement queue are allowed to be executed in parallel.

[0202] In one possible implementation, execution module 1130 is used for:

[0203] After at least one reference statement in the reference queue that does not meet the conditions for parallel execution with the currently obtained statement has been executed, the currently obtained statement is added to the execution statement queue, and the statements in the execution statement queue are allowed to be executed in parallel.

[0204] In one possible implementation, execution module 1130 is further configured to:

[0205] Multiple statements are retrieved from the execution statement queue to execute multiple statements in parallel.

[0206] In one possible implementation, the start time, end time, and submission time are all logical times representing the sequence of events.

[0207] In this embodiment of the disclosure, during the traffic replay process, the recorded statements are judged for parallel execution conditions according to their end time order. This ensures that different statements operating on the same data can only be executed serially, and the statement with the earlier end time will be executed first. In this way, during the target database replay process, it can be guaranteed that the execution order of multiple operations on the same data is the same as the actual execution order during the source database recording process. Therefore, it can ensure data consistency between the target database replay process and the source database recording process as much as possible. Furthermore, it can ensure that the complexity of the transaction executed in the target database is consistent with the actual complexity of the transaction executed in the source database as much as possible, thereby reducing test load distortion.

[0208] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0209] The acquisition module 1110, the judgment module 1120, and the execution module 1130 can all be implemented in software or in hardware. For example, the implementation of the acquisition module 1110 will be described below. Similarly, the implementation of the judgment module 1120 and the execution module 1130 can refer to the implementation of the acquisition module 1110.

[0210] As an example of a software functional unit, module 1110 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 1110 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0211] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0212] As an example of a hardware functional unit, the acquisition module 1110 may include at least one computing device, such as a server. Alternatively, the acquisition module 1110 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0213] The multiple computing devices included in the acquisition module 1110 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 1110 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 1110 can be distributed in the same VPC or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0214] It should be noted that, in other embodiments, the acquisition module 1110, the judgment module 1120, and the execution module 1130 can be used for any step in the traffic playback method. The steps implemented by the acquisition module 1110, the judgment module 1120, and the execution module 1130 can be specified as needed. By implementing different steps in the traffic playback method through the acquisition module 1110, the judgment module 1120, and the execution module 1130, all functions of the traffic playback device can be realized.

[0215] This disclosure also provides a computing device 100. As shown in FIG12, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that this disclosure does not limit the number of processors and memories in the computing device 100.

[0216] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 12, but this does not imply that there is only one bus or one type of bus. Bus 102 can include pathways for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, communication interface 108).

[0217] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0218] The memory 106 may include volatile memory, such as random access memory (RAM). The memory 106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0219] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the aforementioned acquisition module 1110, judgment module 1120, and execution module 1130, thereby realizing the traffic playback method. That is, the memory 106 stores the instructions for the traffic playback method.

[0220] Alternatively, the memory 106 stores executable code, which the processor 104 executes to implement the functions of the aforementioned acquisition module 1110, judgment module 1120, and execution module 1130, thereby realizing the traffic playback method. That is, the memory 106 stores the instructions for the traffic playback method.

[0221] The communication interface 108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 100 and other devices or communication networks.

[0222] This disclosure also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0223] As shown in Figure 13, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for the traffic replay method.

[0224] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for the traffic replay method. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for the traffic replay method.

[0225] It should be noted that the memory 106 in different computing devices 100 within the computing device cluster can store different instructions, each used for a portion of the functions of the traffic playback device. That is, the instructions stored in the memory 106 of different computing devices 100 can implement the functions of one or more of the aforementioned acquisition module 1110, judgment module 1120, and execution module 1130.

[0226] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), and can be a Transmission Control Protocol (TCP) network or a Remote Direct Memory Access (RDMA) network. Figure 14 illustrates one possible implementation. As shown in Figure 14, two computing devices 100A and 100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 106 in computing device 100A stores instructions for executing the functions of the acquisition module 1110 and the judgment module 1120. Simultaneously, the memory 106 in computing device 100B stores instructions for executing the functions of the execution module 1130.

[0227] The connection method between the computing device clusters shown in Figure 14 can be considered as follows: taking into account that the traffic playback method provided in this disclosure requires a large amount of data storage, the function implemented by the execution module 1130 is to be executed by the computing device 100B.

[0228] It should be understood that the functions of computing device 100A shown in Figure 14 can also be performed by multiple computing devices 100. Similarly, the functions of computing device 100B can also be performed by multiple computing devices 100.

[0229] This disclosure also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection method of the computing device cluster described in Figures 13 and 14. The difference is that the memory 106 of one or more computing devices 100 in this computing device cluster can store the same instructions for the traffic replay method.

[0230] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for the traffic replay method. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for the traffic replay method.

[0231] This disclosure also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions capable of running on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a traffic replay method.

[0232] This disclosure also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disk (DVD)), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the stream playback method provided in this application embodiment.

[0233] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A traffic playback method characterized by, The method comprises: acquiring at least one statement executed by a source database in a target time period, an end time of each statement, and a start time of each statement and / or a commit time of a transaction to which each statement belongs; acquiring statements from the at least one statement in turn according to the order of the end times of the statements, and determining whether each acquired statement and each reference statement in a reference queue satisfy a parallel execution condition, wherein the reference queue comprises statements currently executed and statements waiting to be executed in a target database; for each determination, determining whether the currently acquired statement and the currently reference statement satisfy the parallel execution condition based on specified information of the currently acquired statement and the currently reference statement, or based on specified information of associated statements of the currently acquired statement and the currently reference statement, wherein the specified information comprises at least one of a start time of a statement, an end time of a statement, and a commit time of a transaction to which a statement belongs, and the parallel execution condition is used to indicate that the currently acquired statement and the currently reference statement belong to different transactions, and data required by the currently acquired statement for operation when executed in the source database is not locked by the currently reference statement; in a case where the currently acquired statement and each reference statement in the reference queue satisfy the parallel execution condition, performing parallel execution processing on the currently acquired statement in the target database; in a case where the currently acquired statement and at least one reference statement in the reference queue do not satisfy the parallel execution condition, performing serial execution processing on the currently acquired statement in the target database.

2. The method of claim 1, wherein, The satisfying of the parallel execution condition comprises satisfying a first parallel execution condition and a second parallel execution condition, wherein the first parallel execution condition is: the currently acquired statement and the currently reference statement belong to different transactions; the second parallel execution condition is at least one of the following sub-conditions: the end time of the currently acquired statement is earlier than the commit time of the transaction to which the currently reference statement belongs; the commit time of the transaction to which the currently acquired statement belongs is earlier than the commit time of the transaction to which the currently reference statement belongs; the start time of a next statement of the transaction to which the currently acquired statement belongs is earlier than the start time of a next statement of the transaction to which the currently reference statement belongs, wherein the next statement of the transaction to which the currently acquired statement belongs is an associated statement of the currently acquired statement, and the next statement of the transaction to which the currently reference statement belongs is an associated statement of the currently reference statement.

3. The method according to claim 1 or 2, characterized in that, in a case where the currently acquired statement is a read operation statement, the reference queue comprises write operation statements currently executed and waiting to be executed in the target database; in a case where the currently acquired statement is a write operation statement, the reference queue comprises read operation statements and write operation statements currently executed and waiting to be executed in the target database.

4. The method according to claim 1 or 2, characterized in that, In a case where the current acquired statement is a statement operating on a target data table, the reference queue includes statements operating on the target data table that are currently executing and waiting to execute in the target database.

5. The method according to any one of claims 1 to 4, characterized in that, The parallel execution processing of the current acquired statement includes: The current acquired statement is added to an execution statement queue, and the statements in the execution statement queue are allowed to be executed in parallel.

6. The method according to any one of claims 1 to 4, characterized in that, The serial execution processing of the current acquired statement includes: After the at least one reference statement in the reference queue that does not satisfy the parallel execution condition with the current acquired statement is executed, the current acquired statement is added to an execution statement queue, and the statements in the execution statement queue are allowed to be executed in parallel.

7. The method according to claim 5 or 6, characterized in that, The method further includes acquiring a plurality of statements from the execution statement queue to execute the plurality of statements in parallel.

8. The method according to any one of claims 1 to 7, characterized in that, The start time, the end time, and the commit time are all logical times representing time sequence.

9. A flow playback device, characterized by The device is deployed in a database server, and the device includes: An acquisition module is configured to acquire at least one statement executed by a source database within a target time period, an end time of each statement, and a start time of each statement and / or a commit time of a transaction to which each statement belongs. A judgment module is configured to acquire statements from the at least one statement in a sequence of end times of the statements and to determine whether each acquired statement satisfies a parallel execution condition with each reference statement in a reference queue, where the reference queue includes statements currently executing and statements waiting to execute in a target database; for each determination, whether the current acquired statement and the current reference statement satisfy the parallel execution condition is determined based on specified information of the current acquired statement and the current reference statement or based on specified information of associated statements of the current acquired statement and the current reference statement, where the specified information includes at least one of a start time of a statement, an end time of a statement, and a commit time of a transaction to which a statement belongs, and the parallel execution condition indicates that the current acquired statement and the current reference statement belong to different transactions and that data required for operation of the current acquired statement when executed in the source database is not locked by the current reference statement. An execution module is configured to perform parallel execution processing of the current acquired statement in the target database in a case where the current acquired statement satisfies the parallel execution condition with each reference statement in the reference queue, and to perform serial execution processing of the current acquired statement in the target database in a case where the current acquired statement does not satisfy the parallel execution condition with at least one reference statement in the reference queue.

10. The apparatus of claim 9, wherein, The satisfying the parallel execution condition includes satisfying a first parallel execution condition and a second parallel execution condition, where The first parallel execution condition is: The current acquired statement and the current reference statement belong to different transactions. The second parallel execution condition is at least one of the following sub-conditions: The current acquired statement and the current reference statement belong to different transactions. the end time of the current acquired statement is earlier than the commit time of the transaction to which the current reference statement belongs; the commit time of the transaction to which the current acquired statement belongs is earlier than the commit time of the transaction to which the current reference statement belongs; the start time of the next statement of the transaction to which the current acquired statement belongs is earlier than the start time of the next statement of the transaction to which the current reference statement belongs, wherein the next statement of the transaction to which the current acquired statement belongs is an associated statement of the current acquired statement, and the next statement of the transaction to which the current reference statement belongs is an associated statement of the current reference statement.

11. The apparatus of claim 9 or 10, wherein, in a case where the current acquired statement is a read operation statement, the reference queue includes write operation statements that are currently executed and waiting to be executed in the target database; in a case where the current acquired statement is a write operation statement, the reference queue includes read operation statements and write operation statements that are currently executed and waiting to be executed in the target database.

12. The apparatus of claim 9 or 10, wherein, in a case where the current acquired statement is a statement for operating on a target data table, the reference queue includes statements for operating on the target data table that are currently executed and waiting to be executed in the target database.

13. The device of any of claims 9-12, wherein, the execution module is configured to: add the current acquired statement to an execution statement queue, and the statements in the execution statement queue are allowed to be executed in parallel.

14. The apparatus of any one of claims 9-12, wherein, the execution module is configured to: add the current acquired statement to an execution statement queue after the at least one reference statement in the reference queue that does not satisfy the parallel execution condition with the current acquired statement is executed, and the statements in the execution statement queue are allowed to be executed in parallel.

15. The apparatus of claim 13 or 14, wherein, the execution module is further configured to acquire a plurality of statements from the execution statement queue to execute the plurality of statements in parallel.

16. The apparatus of any one of claims 9-15, wherein, the start time, the end time, and the commit time are all logical times representing time sequence.

17. A cluster of computing devices, characterized in that, at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any one of claims 1-8.

18. A computer-readable storage medium, characterized in that, computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any one of claims 1-8.

19. A computer program product comprising instructions, characterized in that, computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any one of claims 1-8.

Images