Data processing method and device

Abstract

The embodiments of this specification provide a data processing method and device, wherein the data processing method includes: obtaining and executing an initial SQL statement submitted by a target user for a target business, processing a first data table related to the target business in a database, generating a first processing result, determining the data storage space of the target user in the database, and adding a space identifier of the data storage space in the initial SQL statement, generating a target SQL statement, determining a second data table to be processed in the database according to the target SQL statement, executing the target SQL statement, processing the second data table, generating a second processing result, and determining a processing result of the target business according to the first processing result and the second processing result, which is conducive to ensuring that the proxy capabilities of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification results.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a data processing system, a computing device, and a computer-readable storage medium. Background Art

[0002] The database proxy component is a network proxy service located between the database server and the application server. It is used to proxy the application server's requests when accessing the database. It has the characteristics of high availability, high performance, operability, and ease of use.

[0003] To enable widespread use of database proxy components, it's often necessary to fully verify their capabilities. This involves verifying (simulating) their ability to process database access requests from application servers. Therefore, how to verify this processing capability and improve the accuracy of the verification results are pressing challenges. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a data processing system, a computing device, and a computer-readable storage medium to address technical deficiencies in the prior art.

[0005] According to a first aspect of an embodiment of this specification, there is provided a data processing method, including:

[0006] Obtaining and executing an initial SQL statement submitted by a target user for a target business, processing a first data table related to the target business in a database, and generating a first processing result;

[0007] Determine the data storage space of the target user in the database, and add a space identifier of the data storage space to the initial SQL statement to generate a target SQL statement;

[0008] Determine a second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, and generate a second processing result;

[0009] A processing result of the target business is determined according to the first processing result and the second processing result.

[0010] According to a second aspect of the embodiments of this specification, there is provided a data processing device, including:

[0011] an acquisition module configured to acquire and execute an initial SQL statement submitted by a target user for a target business, process a first data table related to the target business in a database, and generate a first processing result;

[0012] A generating module configured to determine the data storage space of the target user in the database, and add a space identifier of the data storage space to the initial SQL statement to generate a target SQL statement;

[0013] a processing module configured to determine a second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, and generate a second processing result;

[0014] The determining module is configured to determine a processing result of the target service according to the first processing result and the second processing result.

[0015] According to a third aspect of an embodiment of this specification, there is provided a data processing system, including:

[0016] Database proxy component and SQL playback component;

[0017] The database proxy component is configured to obtain and execute an initial SQL statement submitted by a target user for a target business, process a first data table in a database related to the target business, and generate a first processing result;

[0018] The SQL playback component is used to determine the data storage space of the target user in the database, and add the space identifier of the data access storage space in the initial SQL statement to generate a target SQL statement;

[0019] The database proxy component is used to determine the second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, generate a second processing result, and determine the processing result of the target business according to the first processing result and the second processing result.

[0020] According to a fourth aspect of the embodiments of this specification, a computing device is provided, including:

[0021] memory and processor;

[0022] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the data processing method.

[0023] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data processing method are implemented.

[0024] An embodiment of the present specification obtains and executes an initial SQL statement submitted by a target user for a target business, processes a first data table related to the target business in a database, generates a first processing result, determines the data storage space of the target user in the database, adds a space identifier of the data storage space in the initial SQL statement, generates a target SQL statement, determines a second data table to be processed in the database according to the target SQL statement, executes the target SQL statement, processes the second data table, generates a second processing result, and determines a processing result of the target business according to the first processing result and the second processing result.

[0025] The embodiment of this specification first processes the data in the first data table of the database by executing an initial SQL statement to generate a first processing result. Then, when it is determined that the initial SQL statement does not contain a space identifier of the data storage space, SQL replay is performed, that is, a space identifier is added to the initial SQL statement, so that the database proxy component can process the simulated data of the data storage space in the database (the data in the second data table) according to the space identifier to generate a second processing result. Then, the processing result of the target business is determined based on the first processing result and the second processing result, which is conducive to ensuring that the proxy capability of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is an architectural diagram of a data processing system provided by one embodiment of this specification;

[0027] Figure 2 is a processing flow chart of a data processing method provided by one embodiment of this specification;

[0028] Figure 3 This is a schematic diagram of a data processing process provided by an embodiment of this specification;

[0029] Figure 4 This is a flowchart of a data processing method provided by one embodiment of this specification;

[0030] Figure 5 is a schematic diagram of a data processing device provided by one embodiment of this specification;

[0031] Figure 6 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0032] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0033] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] First, the terms involved in one or more embodiments of this specification are explained.

[0036] Relational database: refers to a database that uses a relational model to organize data. It stores data in the form of rows and columns to facilitate user understanding. This series of rows and columns in a relational database is called a table, and a group of tables constitutes a database.

[0037] The database proxy component is a network proxy service located between the database server and the application server. It is used to proxy the application server's requests when accessing the database. It has the characteristics of high availability, high performance, operability, and ease of use.

[0038] In order to use the database proxy component on a large scale, it is usually necessary to conduct a complete coverage verification of the proxy capability of the database proxy component, that is, to verify (simulate) the database proxy component's ability to process database access requests from the application server.

[0039] Currently, the simulation environment database used to store simulation data needs to synchronize simulation data from the production end. After the simulation data is synchronized to the simulation environment database, it is stored in the baseline table. Some testing and development users (target users) need to add and delete simulation data stored in the simulation environment database to simulate the proxy capabilities of the database proxy component based on the operation results. However, since the simulation environment database usually synchronizes simulation data according to a certain data synchronization cycle, if the user modifies the simulation data of the previous cycle in the simulation environment database, the modified data will be overwritten when synchronizing the simulation data of the next cycle, that is, the simulation data of the next cycle will overwrite the simulation data of the previous cycle.

[0040] Furthermore, requests for operations such as adding and deleting simulation data are typically sent as SQL statements. When processing these requests, the database proxy component must convert them into SQL statements to determine how to process the simulation data in the baseline table and the data storage space. To ensure the stability of the simulation environment, SQL conversion is typically performed only for specific traffic (operation requests from development or test users). Development and test users are then required to participate in comparing the operation results to determine whether the database proxy component's proxy capabilities meet business requirements. However, this approach fails to fully verify the database proxy component's proxy capabilities, preventing its widespread use.

[0041] Faced with the contradiction between ensuring the stability of the simulation environment and the large-scale use of database proxy components, the embodiments of this specification provide a data processing method to solve the success rate of the database proxy component in each application, and can use this data to improve the success rate of the database proxy component in each application without the need for business testing and development intervention to process.

[0042] In this specification, a data processing method is provided. This specification also relates to a data processing apparatus, a data processing system, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0043] Figure 1 The following is an architecture diagram of a data processing system provided according to one embodiment of the present specification, including:

[0044] Database proxy component and SQL playback component;

[0045] The database proxy component is configured to obtain and execute an initial SQL statement submitted by a target user for a target business, process a first data table in a database related to the target business, and generate a first processing result;

[0046] The SQL playback component is used to determine the data storage space of the target user in the database, and add the space identifier of the data access storage space in the initial SQL statement to generate a target SQL statement;

[0047] The database proxy component is used to determine the second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, generate a second processing result, and determine the processing result of the target business according to the first processing result and the second processing result.

[0048] Optionally, the database proxy component obtains and executes the initial SQL statement submitted by the target user for the target business;

[0049] The SQL playback component determines the data storage space of the target user in the database.

[0050] Optionally, the database proxy component parses the target SQL statement according to a preset protocol parsing rule, and determines the second data table to be processed in the database according to the space identifier included in the parsing result.

[0051] Optionally, the data processing system further includes an SQL sampling component;

[0052] The SQL sampling component samples the initial SQL statement according to a preset sampling period;

[0053] The SQL playback component determines the data storage space of the target user in the database, and adds the space identifier of the data storage space to the initial SQL statement included in the sampling result to generate a target SQL statement.

[0054] Optionally, the data processing system further includes a SQL agent execution component;

[0055] The SQL playback component sends the target SQL statement to the SQL agent execution component;

[0056] The SQL agent execution component calls the database agent component, and determines the second data table to be processed in the database according to the target SQL statement through the database agent component.

[0057] Optionally, the SQL playback component cleans the initial SQL statements contained in the sampling results and generates intermediate SQL statements;

[0058] The SQL playback component determines the data storage space of the target user in the database, and adds the space identifier of the data storage space in the intermediate SQL statement to generate a target SQL statement.

[0059] Optionally, the SQL playback component classifies the initial SQL statements contained in the sampling result and generates corresponding classification results;

[0060] The initial SQL statements contained in each of the classification results are screened according to a preset quantity threshold to generate intermediate SQL statements.

[0061] Optionally, the database performs a syntax check on the parsing result, and executes the target SQL statement if the check passes;

[0062] Correspondingly, the database proxy component compares the first processing result with the second processing result, and determines the processing result of the target business according to the comparison result.

[0063] An embodiment of the present specification obtains and executes an initial SQL statement submitted by a target user for a target business, processes a first data table related to the target business in a database, generates a first processing result, determines the data storage space of the target user in the database, adds a space identifier of the data storage space in the initial SQL statement, generates a target SQL statement, determines a second data table to be processed in the database according to the target SQL statement, executes the target SQL statement, processes the second data table, generates a second processing result, and determines a processing result of the target business according to the first processing result and the second processing result.

[0064] The embodiment of this specification first processes the data in the first data table of the database by executing an initial SQL statement to generate a first processing result. Then, when it is determined that the initial SQL statement does not contain a space identifier of the data storage space, SQL replay is performed, that is, a space identifier is added to the initial SQL statement, so that the database proxy component can process the simulated data of the data storage space in the database (the data in the second data table) according to the space identifier to generate a second processing result. Then, the processing result of the target business is determined based on the first processing result and the second processing result, which is conducive to ensuring that the proxy capability of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification result.

[0065] The above is a schematic diagram of a data processing system according to this embodiment. It should be noted that the technical solution of the data processing system and the technical solution of the data processing method described below are based on the same concept. For details not described in detail in the technical solution of the data processing system, please refer to the description of the technical solution of the data processing method described below.

[0066] Figure 2A processing flow chart of a data processing method provided according to an embodiment of this specification is shown, including steps 202 to 208.

[0067] Step 202 : Obtain and execute an initial SQL statement submitted by a target user for a target business, process a first data table related to the target business in a database, and generate a first processing result.

[0068] Specifically, the first data table can be a baseline table. In computer terminology, a baseline is a stable version of a software document, source code, or other output. A baseline is a "snapshot" of each artifact version in the database at a specific time. Different "snapshots" can together constitute a baseline table.

[0069] Target businesses include but are not limited to trading business, insurance business, wealth management business, etc.

[0070] Currently, the database proxy component is a network proxy service located between the database server and the application server. It is used to proxy the application server's requests when accessing the database. It has the characteristics of high availability, high performance, maintainability, and ease of use.

[0071] In order to use the database proxy component on a large scale, it is usually necessary to conduct a complete coverage verification of the proxy capability of the database proxy component, that is, to verify (simulate) the database proxy component's ability to process database access requests from the application server.

[0072] Currently, the simulation environment database used to store simulation data needs to synchronize simulation data from the production end. After the simulation data is synchronized to the simulation environment database, it is stored in the baseline table. Some testing and development users (target users) need to add and delete simulation data stored in the simulation environment database to simulate the proxy capabilities of the database proxy component based on the operation results. However, since the simulation environment database usually synchronizes simulation data according to a certain data synchronization cycle, if the user modifies the simulation data of the previous cycle in the simulation environment database, the modified data will be overwritten when synchronizing the simulation data of the next cycle, that is, the simulation data of the next cycle will overwrite the simulation data of the previous cycle.

[0073] To solve this problem, a certain amount of data storage space can be allocated to test and development users respectively in the simulation environment database for storing simulation environment data. When test and development users need to add or delete simulation data stored in the simulation environment database to simulate the proxy capabilities of the database proxy component based on the operation results, they can add or delete data in the data storage space allocated to them, and the operation results can still be saved in the data storage space. Since the data storage space is exclusively used by a user after being allocated to it, the problem of data coverage can be solved in this way.

[0074] Based on this, after synchronizing the simulation data of the current cycle to the simulation environment data, if the simulation data of the next cycle is not synchronized, the baseline table and data storage space in the simulation environment database store the simulation data of the current cycle. In this case, if the test and development users send a request to add and delete simulation data, the request can be processed by the database proxy component to add and delete the simulation data of the current cycle in the baseline table and data storage space, and the operation results can be compared to simulate the proxy capabilities of the database proxy component.

[0075] However, in real-world applications, requests for operations such as adding and deleting simulation data are typically sent as SQL statements. When processing these requests, the database proxy component must convert them into SQL statements to determine whether to process the simulation data in the baseline table or the data storage space. To ensure the stability of the simulation environment, SQL conversion is typically performed only for specific traffic (operation requests from development or test users). Development and test users are then required to compare the operation results to determine whether the database proxy component's proxy capabilities meet business requirements. However, this approach fails to fully verify the database proxy component's proxy capabilities, preventing its widespread use.

[0076] Among them, when SQL conversion is performed on specific traffic to operate on the simulated data of the data storage space according to the conversion result, since the data storage space is allocated to a certain development user or test user and is used exclusively by the user, when operating on the simulated data in the data storage space, the database proxy component must first determine the data storage space of the development or test user according to the SQL statement, and then operate on the simulated data of the data storage space. However, since the SQL statement usually does not carry relevant information about the data storage space, it is impossible to perform complete coverage verification of the proxy capability of the database proxy component.

[0077] Therefore, in order to ensure that the proxy capabilities of the database proxy component can be fully covered, the embodiment of this specification can perform SQL playback, that is, first execute the initial SQL statement to process the simulation data in the baseline table (first data table) of the simulation environment database. The processing methods include but are not limited to adding, deleting, modifying, checking, etc., to generate a first processing result, and then, when it is determined that the initial SQL statement does not contain the space identifier of the data storage space, add a space identifier to the initial SQL statement so that the database proxy component can process the simulation data of the data storage space according to the space identifier.

[0078] Step 204 : Determine the data storage space of the target user in the database, and add the space identifier of the data storage space to the initial SQL statement to generate a target SQL statement.

[0079] Specifically, as described above, in the embodiment of this specification, after obtaining the initial SQL statement submitted by the target user for the target business, the database proxy component can first send the initial SQL statement to the simulation environment database, so as to process the simulation data in the baseline table (first data table) of the simulation environment database by executing the initial SQL statement, and the processing methods include but are not limited to adding, deleting, modifying, checking, etc., to generate a first processing result. Then, if it is determined that the initial SQL statement does not contain the space identifier of the data storage space, the data storage space pre-allocated for the target user in the simulation environment database can be determined, and the space identifier of the data storage space can be added to the initial SQL statement to generate the target SQL statement, so that the database proxy component can process the simulation data of the data storage space according to the space identifier.

[0080] In actual applications, the space identifier is virtual_id, and the simulation data in the data storage space can be stored in the form of a virtual table. The simulation data in the virtual table is consistent with the baseline table, but compared with the baseline table, the virtual table has a new vitual_id, which is the space identifier of the data storage space, which is used to represent the data storage space pre-allocated to the target user in the simulation environment database.

[0081] In specific implementation, the initial SQL statement submitted by the target user for the target business is obtained and executed, specifically, the initial SQL statement submitted by the target user for the target business is obtained and executed through the database proxy component;

[0082] Accordingly, the data storage space of the target user in the database is determined, specifically, the data storage space of the target user in the database is determined by using an SQL playback component.

[0083] Specifically, after obtaining and executing the initial SQL statement submitted by the target user for the target business through the database proxy component, when it is determined that the initial SQL statement does not contain the space identifier of the data storage space, the SQL replay component can determine the data storage space pre-allocated to the target user in the simulation environment database, and add the space identifier of the data storage space to the initial SQL statement to generate the target SQL statement, so that the database proxy component can process the simulation data of the data storage space in the database (the data in the second data table) according to the space identifier.

[0084] In addition, after executing the initial SQL statement submitted by the target user for the target business, processing the first data table related to the target business in the database, and generating a first processing result, the SQL sampling component can also perform sampling processing on the initial SQL statement according to a preset sampling period;

[0085] Accordingly, determining the data storage space of the target user in the database, adding the space identifier of the data storage space to the initial SQL statement, and generating the target SQL statement includes:

[0086] The data storage space of the target user in the database is determined by the SQL playback component, and a space identifier of the data storage space is added to the initial SQL statement included in the sampling result to generate a target SQL statement.

[0087] Specifically, to ensure the stability of the simulation environment, the embodiments of this specification can sample and process the initial SQL statement according to a preset sampling period through the SQL sampling component, and determine the data storage space pre-allocated for the target user in the simulation environment database through the SQL playback component, so as to add the space identifier of the data storage space to the initial SQL statement in the sampling result, and generate the target SQL statement, so that the database agent component can process the simulation data of the data storage space in the database (the data in the second data table) according to the space identifier.

[0088] Alternatively, after sampling the initial SQL statement, the SQL playback component determines the data storage space of the target user in the database, and adds the space identifier of the data storage space to the initial SQL statement included in the sampling result to generate the target SQL statement, including:

[0089] The SQL playback component cleans the initial SQL statements contained in the sampling results and generates intermediate SQL statements.

[0090] The SQL playback component determines the data storage space of the target user in the database, and adds a space identifier of the data storage space in the intermediate SQL statement to generate a target SQL statement.

[0091] In specific implementation, the SQL playback component is used to clean the initial SQL statements contained in the sampling results and generate intermediate SQL statements, including:

[0092] Classify the initial SQL statements contained in the sampling results through the SQL playback component to generate corresponding classification results;

[0093] The initial SQL statements contained in each of the classification results are screened according to a preset quantity threshold to generate intermediate SQL statements.

[0094] Specifically, after sampling the initial SQL statements, the initial SQL statements contained in the sampling results may be cleaned to add space identifiers of data storage spaces to the SQL statements in the cleaned results to generate target SQL statements.

[0095] Among them, the sampling period for sampling the initial SQL statements can be determined according to actual needs and is not limited here. After the sampling is completed, the SQL statements contained in the sampling results can be further cleaned. Specifically, the SQL statements contained in the sampling results can be divided into types, for example, into four categories: add, delete, modify, and check. Then, a certain number of SQL statements can be screened from each category as the cleaning results. The SQL statements contained in the cleaning results, that is, the remaining SQL statements, are the intermediate SQL statements.

[0096] After the intermediate SQL statement is generated, the data storage space of the target user in the database can be determined by the SQL playback component, and the space identifier of the data storage space is added to the intermediate SQL statement to generate the target SQL statement.

[0097] After sampling the SQL statements, the embodiments of this specification may further clean the SQL statements in the sampling results to reduce the number of SQL statements executed, thereby facilitating the stability of the simulation environment.

[0098] In specific implementation, after generating the target SQL statement, the target SQL statement can be sent to the SQL agent execution component through the SQL playback component;

[0099] The database proxy component is called by the SQL proxy execution component, and the second data table to be processed in the database is determined by the database proxy component according to the target SQL statement.

[0100] Specifically, after the SQL playback component generates the target SQL statement, it can transmit the target SQL statement to the database proxy component through the SQL proxy execution component, and the database proxy component parses the target SQL statement to determine the data storage space to be processed, that is, to determine the second data table (virtual table) to be processed in the simulation environment database.

[0101] Step 206 : determining a second data table to be processed in the database according to the target SQL statement, executing the target SQL statement, processing the second data table, and generating a second processing result.

[0102] During specific implementation, the second data table to be processed in the database is determined according to the target SQL statement, specifically, the second data table to be processed in the database is determined through the database proxy component and the target SQL statement.

[0103] Furthermore, determining a second data table to be processed in the database through the database proxy component and the target SQL statement includes:

[0104] The target SQL statement is parsed by the database proxy component according to a preset protocol parsing rule, and a second data table to be processed in the database is determined according to the space identifier included in the parsing result.

[0105] Specifically, after the database proxy component receives the target SQL statement, it can parse it according to the preset protocol parsing rules to determine the data storage space to be processed in the simulation environment database based on the space identifier contained in the parsing result, that is, the second data table to be processed, so as to process the second data table (for example, add, delete, modify, query, etc. the data in the second data table) by executing the target SQL statement to generate a second processing result.

[0106] In the embodiment of this specification, when it is determined that the initial SQL statement does not contain the space identifier of the data storage space, SQL replay is performed, that is, a space identifier is added to the initial SQL statement, so that the database proxy component can process the simulation data of the data storage space in the database (the data in the second data table) according to the space identifier to generate a second processing result. Subsequently, the processing result of the target business can be determined based on the first processing result and the second processing result, which is conducive to ensuring that the proxy capability of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification result.

[0107] Step 208: Determine a processing result of the target service according to the first processing result and the second processing result.

[0108] Specifically, after generating the first processing result and the second processing result, the processing result of the target business can be determined based on the first processing result and the second processing result. In actual applications, the processing result of the target business can be either a successful execution or a failed execution. Therefore, the proxy capability of the database proxy component can be evaluated based on the processing result, and the evaluation result can be used as a simulation result of its proxy capability.

[0109] For example, if the target user submits x initial SQL statements for the target business, and the processing result of the target business determined based on x1 initial SQL statements is successful, the proxy capability (execution success rate) of the database proxy component is x1 / x*100%.

[0110] In a specific implementation, before executing the target SQL statement, the following steps are also included:

[0111] Performing grammar checking on the parsing result;

[0112] If the test passes, execute the target SQL statement;

[0113] Accordingly, determining the processing result of the target service according to the first processing result and the second processing result includes:

[0114] The first processing result is compared with the second processing result, and a processing result of the target business is determined according to the comparison result.

[0115] Specifically, after the database proxy component parses the target SQL statement to obtain the corresponding parsing result and sends the parsing result to the simulation environment database, the simulation environment database can first determine whether the syntax of the parsing result can pass the syntax detection of the simulation environment database itself. If it fails to pass the detection, the processing result of the target business is determined to be an execution failure; if it passes the detection, after generating the second processing result, the first processing result and the second processing result can be compared. If the comparison is consistent, the processing result of the target business can be determined to be an execution success. If the comparison is inconsistent, the processing result of the target business can be determined to be an execution failure.

[0116] In addition, the embodiments of this specification can solidify SQL statements by sampling SQL statements, so that when it is determined that the proxy capability of the database proxy component does not meet the requirements based on the processing results of the target business, the database proxy component can be adjusted and the sampled SQL statements can be re-executed based on the adjustment results to test the proxy capability of the database proxy component based on the execution results, thereby ensuring the accuracy of the test results.

[0117] A schematic diagram of a data processing process provided in the embodiment of this specification is as follows Figure 3As shown, Figure 3 It includes application server, database proxy component, data flow pipeline, SQL playback component and database. The database proxy component also includes SQL proxy execution component, SQL sampling component and protocol parsing SQL conversion component.

[0118] Specifically, the application server first receives the user's data processing request, which is submitted in the form of an SQL statement. The application server sends the SQL statement (SQL1) to the database proxy component, and then the database proxy component executes the SQL statement to process the simulation data in the first data table, i.e., the baseline table, in the database (simulation environment database) to generate a first processing result (SQL1 execution result).

[0119] Then, if the SQL1 does not carry the space identifier of the data storage space, that is, if it does not carry the virtual_id, the SQL sampling component in the database proxy component can sample the part of the SQL statements, and then send the sampling results to the SQL playback component through the data flow pipeline. The SQL playback component cleans the SQL statements contained in the sampling results, and then replays the SQL statements in the cleaned results to add SQL proxy execution marks, that is, the space identifier of the data storage space, to the SQL statements contained in the cleaned results. The SQL proxy execution component sends the marked SQL statements to the protocol parsing SQL conversion component for parsing (SQL conversion). Then, after the parsing is completed, the target virtual table to be processed can be determined according to the virtual_id in the parsing result, and then the converted SQL statement can be executed to process the simulation data in the target virtual table in the simulation environment database to generate a corresponding second processing result (the execution result of the converted SQL1). Then, the execution success rate of the SQL statement can be determined by comparing the first processing result and the second processing result to determine whether the two are consistent. Among them, if the comparison is consistent, the execution result of SQL1 can be determined as successful execution. If the comparison is inconsistent, the execution result of SQL1 can be determined as failed execution. Then the ratio of the number of successfully executed SQL statements in the SQL statement to the total number of SQL statements can be used as the execution success rate of the SQL statement. The SQL statement execution success rate can be used to evaluate the proxy capability of the database proxy component, and its proxy capability can be proportional to the SQL statement execution success rate.

[0120] Among them, for the SQL statement sampling function, the sampled data is distributed on each single machine to ensure that the SQL request of each application has the opportunity to be written into the data flow pipeline. The SQL statements are batch cleaned through streaming data processing. After the cleaning is completed, the SQL statements are replayed. During the replay, the query virtual table mark is required, that is, the space identifier of the data storage space. This mark can trigger the SQL conversion function, so that the database agent component parses the SQL statement according to the preset parsing protocol rules, and determines the second data table (virtual table) to be processed in the simulation environment database based on the parsing results, and then executes the SQL statement to process the simulation data in the second data table to generate a second processing result; after the SQL statement is executed, the result verification is required, specifically, the consistency of the first processing result and the second processing result is verified, and the execution success rate is given according to the verification result.

[0121] Therefore, in this link, testers or developers do not need to directly participate in verifying the proxy capabilities of the database proxy component. At the same time, since the sampled SQL statements are actually used in the simulation environment, the authenticity of the data is guaranteed.

[0122] In this link, there is actually no impact on the execution of the simulation normalized traffic in the database, which is conducive to ensuring the stability of the existing simulation environment.

[0123] This data processing method eliminates the need for testers or developers to directly verify the database proxy component, reducing labor costs. The SQL playback and normalized SQL execution processes are isolated, and runtime does not affect the stability of the simulation environment. Result verification allows comparison of actual and expected execution results, effectively determining the success rate of the database proxy component. Furthermore, since the aforementioned process is universal, it can be verified across different applications, effectively capturing application coverage.

[0124] An embodiment of the present specification obtains and executes an initial SQL statement submitted by a target user for a target business, processes a first data table related to the target business in a database, generates a first processing result, determines the data storage space of the target user in the database, adds a space identifier of the data storage space in the initial SQL statement, generates a target SQL statement, determines a second data table to be processed in the database according to the target SQL statement, executes the target SQL statement, processes the second data table, generates a second processing result, and determines a processing result of the target business according to the first processing result and the second processing result.

[0125] The embodiment of this specification first processes the data in the first data table of the database by executing an initial SQL statement to generate a first processing result. Then, when it is determined that the initial SQL statement does not contain a space identifier of the data storage space, SQL replay is performed, that is, a space identifier is added to the initial SQL statement, so that the database proxy component can process the simulated data of the data storage space in the database (the data in the second data table) according to the space identifier to generate a second processing result. Then, the processing result of the target business is determined based on the first processing result and the second processing result, which is conducive to ensuring that the proxy capability of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification result.

[0126] The following combined Figure 4 , taking the application of the data processing method provided in this specification in a practical scenario as an example, the data processing method is further explained. Figure 4 A flowchart of a data processing method provided in one embodiment of the present specification is shown, wherein the specific steps include step 402 to step 422 .

[0127] Step 402: Obtain and execute the initial SQL statement submitted by the target user for the target business through the database proxy component, process the first data table related to the target business in the database, and generate a first processing result.

[0128] Step 404: The SQL sampling component performs sampling processing on the initial SQL statement according to a preset sampling period.

[0129] Step 406: The SQL playback component cleans the initial SQL statements contained in the sampling results to generate intermediate SQL statements.

[0130] Step 408: Determine the target user's data storage space in the database through the SQL playback component.

[0131] Step 410: Add the space identifier of the data storage space to the intermediate SQL statement to generate a target SQL statement.

[0132] Step 412: Send the target SQL statement to the SQL agent execution component through the SQL playback component.

[0133] Step 414: Call the database proxy component through the SQL proxy execution component.

[0134] Step 416: The target SQL statement is parsed by the database proxy component according to the preset protocol parsing rules, and the second data table to be processed in the database is determined based on the space identifier included in the parsing result.

[0135] Step 418: Perform grammar check on the parsing result.

[0136] Step 420: If the detection passes, execute the target SQL statement to process the second data table and generate a second processing result.

[0137] Step 422: Compare the first processing result with the second processing result, and determine the processing result of the target business based on the comparison result.

[0138] The embodiment of this specification first processes the data in the first data table of the database by executing an initial SQL statement to generate a first processing result. Then, when it is determined that the initial SQL statement does not contain a space identifier of the data storage space, SQL replay is performed, that is, a space identifier is added to the initial SQL statement, so that the database proxy component can process the simulated data of the data storage space in the database (the data in the second data table) according to the space identifier to generate a second processing result. Then, the processing result of the target business is determined based on the first processing result and the second processing result, which is conducive to ensuring that the proxy capability of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification result.

[0139] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 5 FIG1 shows a schematic diagram of a data processing device provided by an embodiment of this specification. Figure 5 As shown, the device includes:

[0140] The acquisition module 502 is configured to acquire and execute an initial SQL statement submitted by a target user for a target business, process a first data table related to the target business in a database, and generate a first processing result;

[0141] A generating module 504 is configured to determine the data storage space of the target user in the database, and add a space identifier of the data storage space to the initial SQL statement to generate a target SQL statement;

[0142] The processing module 506 is configured to determine a second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, and generate a second processing result;

[0143] The determination module 508 is configured to determine a processing result of the target service according to the first processing result and the second processing result.

[0144] Optionally, the acquisition module 502 is further configured to:

[0145] Obtain and execute the initial SQL statement submitted by the target user for the target business through the database proxy component;

[0146] Accordingly, the generating module 504 is further configured to:

[0147] The data storage space of the target user in the database is determined by the SQL playback component.

[0148] Optionally, the processing module 506 is further configured to:

[0149] A second data table to be processed in the database is determined by using the database proxy component and the target SQL statement.

[0150] Optionally, the processing module 506 is further configured to:

[0151] The target SQL statement is parsed by the database proxy component according to a preset protocol parsing rule, and a second data table to be processed in the database is determined according to the space identifier included in the parsing result.

[0152] Optionally, the data processing device further includes a sampling module configured to:

[0153] Sampling the initial SQL statement according to a preset sampling period by the SQL sampling component;

[0154] Accordingly, the generating module 504 is further configured to:

[0155] The data storage space of the target user in the database is determined by the SQL playback component, and a space identifier of the data storage space is added to the initial SQL statement included in the sampling result to generate a target SQL statement.

[0156] Optionally, the data processing device further includes a sending module configured to:

[0157] Sending the target SQL statement to the SQL agent execution component through the SQL playback component;

[0158] The database proxy component is called by the SQL proxy execution component, and the second data table to be processed in the database is determined by the database proxy component according to the target SQL statement.

[0159] Optionally, the generating module 504 is further configured to:

[0160] The SQL playback component cleans the initial SQL statements contained in the sampling results and generates intermediate SQL statements.

[0161] The SQL playback component determines the data storage space of the target user in the database, and adds a space identifier of the data storage space in the intermediate SQL statement to generate a target SQL statement.

[0162] Optionally, the generating module 504 is further configured to:

[0163] Classify the initial SQL statements contained in the sampling results through the SQL playback component to generate corresponding classification results;

[0164] The initial SQL statements contained in each of the classification results are screened according to a preset quantity threshold to generate intermediate SQL statements.

[0165] Optionally, the processing module 506 is further configured to:

[0166] Performing grammar checking on the parsing result;

[0167] If the test passes, execute the target SQL statement;

[0168] Accordingly, the determining module 508 is further configured to:

[0169] The first processing result is compared with the second processing result, and a processing result of the target business is determined according to the comparison result.

[0170] A data processing device provided by an embodiment of the present specification obtains and executes an initial SQL statement submitted by a target user for a target business through an acquisition module, processes a first data table related to the target business in a database, and generates a first processing result; determines the data storage space of the target user in the database through a generation module, and adds a space identifier of the data storage space to the initial SQL statement to generate a target SQL statement; determines a second data table to be processed in the database according to the target SQL statement through a processing module, executes the target SQL statement, processes the second data table, and generates a second processing result; determines a processing result of the target business according to the first processing result and the second processing result through a determination module.

[0171] The embodiments of this specification obtain and execute an initial SQL statement submitted by a target user for a target business, process a first data table related to the target business in a database, generate a first processing result, determine the data storage space of the target user in the database, add a space identifier of the data storage space in the initial SQL statement, generate a target SQL statement, determine a second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, generate a second processing result, and determine a processing result of the target business according to the first processing result and the second processing result.

[0172] The embodiment of this specification first processes the data in the first data table of the database by executing an initial SQL statement to generate a first processing result. Then, when it is determined that the initial SQL statement does not contain a space identifier of the data storage space, SQL replay is performed, that is, a space identifier is added to the initial SQL statement, so that the database proxy component can process the simulated data of the data storage space in the database (the data in the second data table) according to the space identifier to generate a second processing result. Then, the processing result of the target business is determined based on the first processing result and the second processing result, which is conducive to ensuring that the proxy capability of the database proxy component can be fully covered and verified, thereby improving the accuracy of the verification result.

[0173] The above is a schematic diagram of a data processing device according to this embodiment. It should be noted that the technical solution of the data processing device and the technical solution of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the above-mentioned data processing method.

[0174] Figure 6 6 shows a block diagram of a computing device 600 according to one embodiment of the present disclosure. Components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0175] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0176] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 6 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0177] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 600 may also be a mobile or stationary server.

[0178] The memory 610 is used to store computer-executable instructions, and the processor 620 is used to execute the following computer-executable instructions to implement the steps of the data processing method.

[0179] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned data processing method are of the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the above-mentioned data processing method.

[0180] An embodiment of the present specification further provides a computer-readable storage medium storing computer instructions, which are used to implement the steps of the data processing method when executed by a processor.

[0181] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the above-mentioned data processing method.

[0182] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0183] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0184] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0185] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Obtaining and executing an initial SQL statement submitted by a target user for a target business through a database proxy component, processing a first data table related to the target business in a database, and generating a first processing result; Determining the data storage space of the target user in the database through the SQL playback component, and adding a space identifier of the data storage space to the initial SQL statement to generate a target SQL statement, wherein the data storage space is allocated to the target user and is exclusively used by the target user; Determining, by a database proxy component, a second data table to be processed in the database according to the target SQL statement, and executing the target SQL statement to process the second data table and generate a second processing result, wherein the data in the second data table is simulation data in the data storage space; The database proxy component determines the processing result of the target business according to the first processing result and the second processing result.

2. The data processing method according to claim 1, wherein determining the second data table to be processed in the database according to the target SQL statement comprises: A second data table to be processed in the database is determined by using the database proxy component and the target SQL statement.

3. The data processing method according to claim 2, wherein determining the second data table to be processed in the database by using the database proxy component and the target SQL statement comprises: The target SQL statement is parsed by the database proxy component according to a preset protocol parsing rule, and a second data table to be processed in the database is determined according to the space identifier included in the parsing result.

4. The data processing method according to any one of claims 1 to 3, further comprising: Sampling the initial SQL statement according to a preset sampling period by the SQL sampling component; Accordingly, determining the data storage space of the target user in the database, adding the space identifier of the data storage space to the initial SQL statement, and generating the target SQL statement includes: The data storage space of the target user in the database is determined by the SQL playback component, and a space identifier of the data storage space is added to the initial SQL statement included in the sampling result to generate a target SQL statement.

5. The data processing method according to claim 4, further comprising: Sending the target SQL statement to the SQL agent execution component through the SQL playback component; The database proxy component is called by the SQL proxy execution component, and the second data table to be processed in the database is determined by the database proxy component according to the target SQL statement.

6. The data processing method according to claim 4, wherein determining the data storage space of the target user in the database by the SQL playback component and adding the space identifier of the data storage space to the initial SQL statement included in the sampling result to generate the target SQL statement comprises: The SQL playback component cleans the initial SQL statements contained in the sampling results and generates intermediate SQL statements. The SQL playback component determines the data storage space of the target user in the database, and adds a space identifier of the data storage space in the intermediate SQL statement to generate a target SQL statement.

7. The data processing method according to claim 6, wherein the step of cleaning the initial SQL statements contained in the sampling results by the SQL playback component to generate intermediate SQL statements comprises: Classify the initial SQL statements contained in the sampling results through the SQL playback component to generate corresponding classification results; The initial SQL statements contained in each of the classification results are screened according to a preset quantity threshold to generate intermediate SQL statements.

8. The data processing method according to claim 3, before executing the target SQL statement, further comprising: Performing grammar checking on the parsing result; If the test passes, execute the target SQL statement; Accordingly, determining the processing result of the target service according to the first processing result and the second processing result includes: The first processing result is compared with the second processing result, and a processing result of the target business is determined according to the comparison result.

9. A data processing device comprising: an acquisition module configured to acquire and execute an initial SQL statement submitted by a target user for a target business through a database proxy component, process a first data table related to the target business in a database, and generate a first processing result, wherein the data storage space is allocated to the target user and is exclusively used by the target user; A generating module is configured to determine the data storage space of the target user in the database through the SQL playback component, and add a space identifier of the data storage space to the initial SQL statement to generate a target SQL statement; a processing module configured to determine a second data table to be processed in the database according to the target SQL statement, execute the target SQL statement, process the second data table, and generate a second processing result, wherein the data in the second data table is simulation data in the data storage space; The determining module is configured to determine a processing result of the target service according to the first processing result and the second processing result.

10. A data processing system comprising: Database proxy component and SQL playback component; The database proxy component is configured to obtain and execute an initial SQL statement submitted by a target user for a target business, process a first data table in a database related to the target business, and generate a first processing result; The SQL playback component is used to determine the data storage space of the target user in the database, and add the space identifier of the data access storage space to the initial SQL statement to generate a target SQL statement, wherein the data storage space is allocated to the target user and is used exclusively by the target user; The database proxy component is used to determine the second data table to be processed in the database according to the target SQL statement, and execute the target SQL statement, wherein the data in the second data table is simulation data in the data storage space, the second data table is processed to generate a second processing result, and the processing result of the target business is determined according to the first processing result and the second processing result.

11. A computing device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the data processing method according to any one of claims 1 to 8.

12. A computer-readable storage medium storing computer instructions, wherein when the instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 8 are implemented.

13. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 8.

Images