Database simulation test method, device and equipment

Abstract

According to the database simulation test method, device and equipment provided by the embodiment of the invention, in response to a playback request operation, a database traffic log is obtained, and the database traffic log comprises interaction data between a back-end service and a target database; performing a simulation test on a simulation database based on the database traffic log, wherein the simulation database and the target database are different in communication protocol and store the same data; and generating a function performance report based on a result of the simulation test, wherein the performance report is used for indicating a difference between the target database and the simulation database. In the embodiment of the invention, the database traffic log can cover each scene of the target database, and the service logic which is the same as that of the target database is executed on the simulation database in a simulation test mode, so that the advantages and disadvantages of the simulation database can be efficiently determined at low cost.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a database simulation test method, device and equipment. Background Art

[0002] As business needs grow and technology advances, database systems become increasingly complex. When new database types emerge, users need to test their databases to ensure that existing businesses can operate properly within them. This testing verifies the new database's compatibility and support for applications. Compatibility issues can arise between different database types, and data migration between them can present various risks, including unsupported features, performance bottlenecks, and data inconsistencies. Traditional testing methods may not fully cover all scenarios, and their high testing costs and low efficiency make them difficult to meet user testing needs. Summary of the Invention

[0003] In view of this, the present application provides a database simulation test method, device and equipment to help solve the problem of high cost and low efficiency of database testing in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a database simulation testing method, comprising:

[0005] In response to the playback request operation, obtaining a database traffic file, wherein the database traffic file includes interaction data between the backend service and the target database;

[0006] Performing a simulation test on a simulation database based on the database traffic file, wherein the simulation database has a different communication protocol from the target database and stores the same data;

[0007] A performance report is generated based on the results of the simulation test, wherein the performance report is used to indicate the differences between the target database and the simulation database.

[0008] In an optional embodiment, before obtaining the database traffic file in response to the playback request operation, the method further includes:

[0009] In response to a recording request operation, capturing the interaction data during the interaction between the backend service and the target database;

[0010] The interaction data is stored in the database flow file.

[0011] In an optional embodiment, in response to the recording request operation, capturing the interaction data during the interaction between the backend service and the target database includes:

[0012] Based on the data tags of the target database, capturing the interaction data during the interaction between the backend service and the target database;

[0013] The data tag of the target database is determined by the communication protocol of the target database.

[0014] In an optional embodiment, the interaction data includes: command statements, execution start time, execution end time, time consumption, number of concurrent operations, metadata information or connection information.

[0015] In an optional embodiment, a recording period is set in the recording request operation;

[0016] The step of capturing interaction data related to the target database in a system log in response to the recording request operation includes:

[0017] determining target interaction data within the recording period in the system log based on a timestamp of the system log;

[0018] The target interaction data is captured.

[0019] In an optional embodiment, the interaction data includes an operation instruction sent by the backend service to the target database or first response data fed back by the target database;

[0020] The simulation test of the simulation database based on the database flow file includes:

[0021] The operation instruction is sent to the simulation database, so that the simulation database feeds back second response data.

[0022] In an optional embodiment, generating a performance report based on the simulation test results includes:

[0023] The performance report is generated based on the operation instruction, the first response data, or the second response data.

[0024] In an optional embodiment, sending the operation instruction to a simulation database so that the simulation database feeds back second response data includes:

[0025] Allocating the operation instruction to a corresponding transmission queue based on the data type of the operation instruction;

[0026] Controlling each transmission queue to send each operation instruction to the simulation database in parallel.

[0027] In a second aspect, an embodiment of the present application provides a database simulation test device, comprising:

[0028] An acquisition module, configured to acquire a database traffic file in response to a playback request operation, wherein the database traffic file includes interaction data between a backend service and a target database;

[0029] A testing module, configured to perform a simulation test on a simulation database based on the database traffic file, wherein the simulation database has a different communication protocol from the target database and stores the same data;

[0030] The processing module is used to generate a performance report based on the result of the simulation test, wherein the performance report is used to indicate the difference between the target database and the simulation database.

[0031] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the electronic device is triggered to execute any of the methods described in the first aspect above.

[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any method described in the first aspect.

[0033] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes any one of the methods described in the first aspect.

[0034] Using the solution provided in the embodiment of the present application, in response to a playback request operation, a database traffic file is obtained, wherein the database traffic file contains interaction data between a backend service and a target database; a simulation test is performed on a simulation database based on the database traffic file, wherein the simulation database and the target database have different communication protocols and store the same data; a performance report is generated based on the results of the simulation test, wherein the performance report is used to indicate the differences between the target database and the simulation database. In the embodiment of the present application, the database traffic file can cover various scenarios of the target database, and by executing the same business logic on the simulation database as the target database through simulation testing, the advantages and disadvantages of the simulation database can be determined efficiently and at a low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 An example schematic diagram of a database simulation test method provided in an embodiment of the present application;

[0037] Figure 2 A flowchart of a database simulation test method provided in an embodiment of the present application;

[0038] Figure 3 A flowchart of another database simulation test method provided in an embodiment of the present application;

[0039] Figure 4 A flowchart of another database simulation test method provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of a database simulation test device provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0043] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0044] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0045] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0046] In response to the data migration needs between different databases, an embodiment of the present application provides a database simulation testing method, which records the interaction data during the interaction between the back-end service and the target database, and replays the interaction data in a simulation environment to analyze the improvement points of the simulation database, thereby improving database testing efficiency and reducing testing costs.

[0047] Figure 1 This is an example diagram of a database simulation method provided in an embodiment of the present application. Figure 1 As shown, the user issues an operation instruction through the client, and the load balancer distributes the operation instruction to the appropriate queue. The backend service calls the database cluster based on the operation instruction, and the database cluster returns the response data. Optionally, the database cluster can include various types of databases, such as MySQL, PostgreSQL, Sybase, etc. The database simulation device includes a recording module and a playback module. The process involved mainly includes:

[0048] (1) Recording phase: The user sends an operation instruction through the client, which passes through the load balancer to the backend service. The backend service processes the operation instruction and interacts with the database. The recording module captures and records all the interaction data between the backend service and the database, such as the operation instruction sent by the backend service and the response data fed back by the database. The recorded interaction data is stored in the database traffic file for subsequent playback.

[0049] (2) Playback phase: When verification or troubleshooting of other types of databases is required, the playback module is started. The playback module reads the interaction data from the database traffic file and builds the corresponding simulation environment, including the simulation database. The playback module tests the simulation database based on the interaction data, sends operation instructions to the simulation database, and checks whether the simulation database behaves as expected to determine whether there are any compatibility issues between the two databases.

[0050] Taking the railway system as an example, the specific steps of the recording and playback stages are explained. The steps of the recording stage can be as follows: Figure 2 As shown, it mainly includes:

[0051] Step 201: The user initiates a request.

[0052] Users initiate requests through front-end applications (such as computer web pages or mobile applications). For example, users use railway apps on their mobile terminals to check train schedules and book tickets.

[0053] Step 202: forward the request based on the load balancing policy.

[0054] The front-end application sends the user's request to the load balancer. For example, when a user clicks the "Search" button on a computer webpage, the front-end application sends a Hypertext Transfer Protocol (HTTP) request to the load balancer. After receiving the request, the load balancer distributes the request to an instance of the back-end service based on a preset strategy (such as round-robin or minimum number of connections). For example, the load balancer forwards the request to an Application Programming Interface (API) server instance of the back-end service.

[0055] Step 203: The backend service calls the database.

[0056] After receiving the request, the backend service processes the business logic and interacts with the database. For example, the backend service may query the database for train schedule information and return the results to the user.

[0057] Step 204: Record the interaction process.

[0058] During interactions between backend services and the database, the recording module captures and records all database operation instructions and response data in bypass mode, without interfering with existing data requests. For example, it records the command statement (such as an SQL statement) used to query the train schedule, along with the execution start and end times, execution duration, number of concurrent connections, database nodes, database schema, metadata (such as data field type, length, and precision), connection information, and the resulting response.

[0059] Step 205: store the interaction data.

[0060] The recording module stores the captured interaction data. For example, it saves SQL query statements and response results to a distributed file system or object storage, which can be used for subsequent playback and analysis.

[0061] The steps of the playback phase can be as follows Figure 3 As shown, it mainly includes:

[0062] Step 301: Read interaction data.

[0063] When playback is required, the playback module reads the recorded interaction data from the database traffic file. For example, the playback module reads the previously saved SQL query statements and response data.

[0064] Step 302: Simulate and replay interactive data to the simulation database.

[0065] The replay module simulates the original database operation request, sending the operation instructions to the simulated database. For example, the replay module sends SQL query statements to the simulated database and obtains the corresponding response results, thereby checking whether the system behavior meets expectations. For example, the replay module verifies whether the query results are correct and whether the system can correctly process the requests. If any problems are found, further analysis and debugging can be carried out.

[0066] In this embodiment, the simulation database and the target database (i.e., the database used during the recording phase) are of different types and have different communication protocols. By recording the interaction data and replaying the entire interaction process in a simulation environment, the simulation database's response to operational instructions and response speed can be tested. This covers most interaction scenarios, improving database testing efficiency.

[0067] Figure 4 This is a flow chart of a database simulation test method provided in an embodiment of the present application. The method can be executed by the above-mentioned database simulation device, such as Figure 4 As shown, the method may include:

[0068] Step 401: In response to a playback request operation, a database traffic file is obtained, where the database traffic file includes interaction data between a backend service and a target database.

[0069] Step 402: Perform simulation testing on a simulation database based on the database traffic file. The simulation database has a different communication protocol from the target database and stores the same data.

[0070] Step 403: Generate a performance report based on the result of the simulation test. The performance report is used to indicate the difference between the target database and the simulation database.

[0071] The database traffic file is pre-recorded and generated by the database simulation device. Specifically, prior to step 401, in response to a user-initiated recording request, the database simulation device captures interaction data related to the target database in the system log and stores the interaction data in the database traffic file. The interaction data related to the target database in the system log is tagged with a data tag, which is determined by the target database's communication protocol. Based on the target database type, the database simulation device can parse the corresponding communication protocol and successfully identify interaction data with a specific data tag.

[0072] In the embodiments of the present application, the database emulation device can parse and understand the communication protocols of different database types to accurately capture the interaction data. The interaction data in the system log is specially marked, and the database emulation device can successfully identify and process interaction data related to Sybase, PostgreSQL, and MySQL.

[0073] In an optional embodiment, a recording period is further provided in the recording request operation to filter interaction data within a specific time period. Specifically, the database emulation device is capable of understanding the log timestamp formats of different databases and, based on the system log timestamp, identifies target interaction data within the recording period in the system log, thereby capturing the target interaction data and storing it in chronological order.

[0074] The interaction data includes an operation instruction sent by the backend service to the target database or first response data fed back by the target database. In step 402, the database simulation device performs the same operation on the simulation database as on the target database based on the database traffic file. Specifically, the database simulation device sends the operation instruction to the simulation database, causing the simulation database to feed back second response data.

[0075] In an optional embodiment, when the database simulation device sends an operation instruction, it can allocate the operation instruction to a corresponding transmission queue based on the data type of the operation instruction. Then, each transmission queue is controlled to send the operation instruction to the simulation database in parallel to meet the load balancing strategy, improve playback efficiency, and optimize playback performance.

[0076] Based on the comparison of the first response data and the second response data, the database simulation device can determine the difference in responses between the target database and the simulated database for the same operation instruction. In step 403, based on the operation instruction, the first response data, and the second response data, the database simulation device can generate a corresponding performance report to indicate the performance of the simulated database and whether there are any usage issues.

[0077] In the embodiments of the present application, automated testing of simulated data using a database simulation device can adapt to the characteristics and behaviors of different databases to ensure the comprehensiveness and accuracy of the test. The performance report can include various parameters related to the target database and the simulated database, such as load parameters during the interaction process, response time, and the accuracy of the response data. Based on the performance report, users can understand the advantages and disadvantages of different database types and make corresponding adjustments to avoid potential problems during data migration.

[0078] The above database simulation test method is described below through specific embodiments.

[0079] In the railway ticketing system, Database A stores a large amount of business data. Database B is a newly developed database of a different type than Database A. If the railway ticketing system subsequently plans to store data in Database B, Database B must first be tested to ensure that it can respond properly to various operational commands. The database simulation device creates a simulation environment. By testing a simulated database (of the same type as Database B), the performance of Database B can be comprehensively and efficiently determined at low cost.

[0080] The database emulation device determines the database type used by the railway ticketing system, such as Sybase, PostgreSQL, or MySQL. Based on the database type, it parses the corresponding database communication protocol to accurately capture SQL statements. During normal operation of the railway ticketing system, the database emulation device captures interaction data, including database requests for user queries, ticket purchases, and refunds, based on data tags, for subsequent analysis.

[0081] Users can set a specific time period, such as peak holiday periods. According to the time period set by the user, the database simulation device can capture interaction data within the time period and generate a database traffic file.

[0082] In the embodiment of the present application, by recording and generating database traffic files, the database simulation device can cover most of the interaction scenarios of database A during testing, making the test more comprehensive.

[0083] During testing, the database emulator can playback the recorded data in its original format or filter out specific operations from the database traffic file, such as replaying only ticket purchase operations. Optionally, a proxy can be used to intercept interaction data at multiple levels to simulate real user operations. The proxy server processes the interaction data, simulating the user's ticket purchase operation. Based on testing requirements, non-critical interaction data, such as query operations not required for testing, can be filtered out. Furthermore, the database emulator can replay traffic proportionally, such as replaying only one-third of the traffic after dividing it into thirds.

[0084] The database simulation device calls the simulation database based on the filtered operation instructions, so that the simulation database performs the same operation as database A. Optionally, the database simulation device assigns the operation instructions to different queues according to the number and type of operation instructions to simulate a high-concurrency scenario where different users purchase tickets at the same time and achieve load balancing.

[0085] After the test is completed, the database simulation device generates a report containing response data and performance data based on the feedback from the simulated database, such as the average response time of the ticket purchase operation and whether the ticket purchase operation is executed smoothly. Optionally, the database simulation device can generate a compatibility assessment table, indicating incompatible SQL statements such as add, delete, modify, and query, as shown in the following table:

[0086] Task ID Statement Type Number of runs Number of passes Number of failed Pass rate date 001 Select 1292 1251 41 97.00％ XX Insert 154 142 12 92.00％ Update 216 213 3 99.00％ Delete 52 52 0 100.00％ Other 13 9 4 69.00％ total 1727 1667 60 97.00％

[0087] The above table can be used to visually view compatibility issues between different databases, which is convenient for subsequent debugging.

[0088] In the embodiments of this application, the database simulation device uses an automated testing framework to reduce manual intervention, thereby improving testing efficiency and reducing testing costs. Automated testing helps identify and fix potential issues, optimizes the system based on test results, reduces issues after the new version of the database is launched, and improves the stability of the railway passenger ticket system and the user's ticket purchasing experience.

[0089] Figure 5 This is a schematic diagram of the structure of another database simulation test device provided in an embodiment of the present application. Figure 5 As shown, the device may include:

[0090] The acquisition module 510 is configured to acquire a database traffic file in response to a playback request operation, wherein the database traffic file includes interaction data between a backend service and a target database.

[0091] The testing module 520 is used to perform a simulation test on a simulation database based on the database traffic file. The simulation database has a different communication protocol from the target database and stores the same data.

[0092] The processing module 530 is configured to generate a performance report based on the result of the simulation test, wherein the performance report is configured to indicate the difference between the target database and the simulation database.

[0093] Corresponding to the above embodiments, the present application also provides an electronic device. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 600 may include: a processor 601, a memory 602, and a communication unit 603. These components communicate via one or more buses. Those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiments of the present application. It can be a bus structure or a star structure, and can also include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0094] The communication unit 603 is configured to establish a communication channel so that the electronic device can communicate with other devices, receive user data sent by other devices, or send user data to other devices.

[0095] The processor 601 is the control center of the electronic device. It uses various interfaces and lines to connect the various parts of the entire electronic device. It runs or executes software programs, instructions, and / or modules stored in the memory 602, and calls the data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 601 can only include a central processing unit (CPU). In the embodiment of the present application, the CPU can be a single computing core or multiple computing cores.

[0096] The memory 602 is used to store the execution instructions of the processor 601. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0097] When the execution instructions in the memory 602 are executed by the processor 601 , the electronic device 600 is enabled to execute part or all of the steps in the above embodiments.

[0098] In a specific implementation, the present application further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps of each embodiment of the database simulation test method provided in the present application. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0099] In a specific implementation, the present application also provides a computer program product, wherein the computer program product includes executable instructions, which, when executed on a computer, enable the computer to execute some or all of the steps in each embodiment of the database simulation testing method provided in the present application.

[0100] An embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable the computer to execute the database simulation test method provided by the embodiment of the present application.

[0101] The above-mentioned non-temporary computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (Read Only Memory; hereinafter referred to as: ROM), an erasable programmable read-only memory (ErasableProgrammable Read Only Memory; hereinafter referred to as: EPROM) or flash memory, optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0102] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0103] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0104] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments.

[0105] In this specification, reference can be made to the same or similar parts between the various embodiments. In particular, for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

Claims

1. A database simulation test method, characterized in that: include: In response to the playback request operation, obtaining a database traffic file, wherein the database traffic file includes interaction data between the backend service and the target database; Performing a simulation test on a simulation database based on the database traffic file, wherein the simulation database has a different communication protocol from the target database and stores the same data; A performance report is generated based on the results of the simulation test, wherein the performance report is used to indicate the differences between the target database and the simulation database.

2. The method according to claim 1, characterized in that Before obtaining the database traffic file in response to the playback request operation, the method further includes: In response to a recording request operation, capturing the interaction data during the interaction between the backend service and the target database; The interaction data is stored in the database flow file.

3. The method according to claim 2, characterized in that The step of capturing the interaction data during the interaction between the backend service and the target database in response to the recording request operation includes: Based on the data tags of the target database, capturing the interaction data during the interaction between the backend service and the target database; The data tag of the target database is determined by the communication protocol of the target database.

4. The method according to claim 2, characterized in that The interaction data includes: command statements, execution start time, execution end time, time consumption, number of concurrent users, metadata information or connection information.

5. The method according to claim 1, wherein The interaction data includes an operation instruction sent by the backend service to the target database or first response data fed back by the target database; The simulation test of the simulation database based on the database flow file includes: The operation instruction is sent to the simulation database, so that the simulation database feeds back second response data.

6. The method according to claim 5, characterized in that Generating a performance report based on the simulation test results includes: The performance report is generated based on the operation instruction, the first response data, or the second response data.

7. The method according to claim 5, characterized in that The step of sending the operation instruction to the simulation database so that the simulation database feeds back second response data includes: Allocating the operation instruction to a corresponding transmission queue based on the data type of the operation instruction; Controlling each transmission queue to send each operation instruction to the simulation database in parallel.

8. A database simulation test device, characterized in that: include: An acquisition module, configured to acquire a database traffic file in response to a playback request operation, wherein the database traffic file includes interaction data between a backend service and a target database; A testing module, configured to perform a simulation test on a simulation database based on the database traffic file, wherein the simulation database has a different communication protocol from the target database and stores the same data; The processing module is used to generate a performance report based on the result of the simulation test, wherein the performance report is used to indicate the difference between the target database and the simulation database.

9. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images