Database upgrading method and device, equipment, medium and program product

Abstract

The invention provides a database upgrading method. The method can be applied to the technical fields of artificial intelligence and big data. The method comprises the steps of obtaining first structured query language request traffic of an initial database, performing key information extraction on n first structured query language statements in the first structured query language request traffic, and generating key information of the n first structured query language statements. And analyzing the key information of the n first structured query language statements, and generating a deformation rule of the n first structured query language statements. And based on the deformation rule of the n first structured query language statements, performing data deformation on the n first structured query language statements to generate n second structured query language statements. And based on the n second structured query language statements, performing database upgrading on the initial database to generate a target database. The invention further provides a database upgrading device and equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, in particular to the technical field of artificial intelligence, and specifically to a database upgrading method and device, equipment, medium and program product. BACKGROUND

[0002] With the progress of computer technology, database systems are also optimized and iterated with technology updates. Enterprises including banks gradually increase the demand for database upgrading and transformation. In the process of database upgrading and transformation, sensitive data in the initial database needs to be data morphed to improve data security.

[0003] In the prior art, in the process of database upgrading and transformation, sensitive data in the initial database needs to be data morphed, which is generally manually marked in the initial database by a data stream recording and playing tool, and data morphing operations are performed in the test database. This method requires manual marking of sensitive fields, which needs to be analyzed piece by piece, is time-consuming and prone to errors, resulting in low efficiency. And un-morphed sensitive data may be leaked to the test environment for data morphing, causing data security risks. At the same time, since the existing recording and playing tool is separated from the data morphing tool, the data cannot be dynamically adapted to the morphing requirements of different scenarios, resulting in the technical problem of reduced reliability of database upgrading. SUMMARY

[0004] In view of the above problems, the present application provides a database upgrading method, device, equipment, medium and program product.

[0005] According to a first aspect of the present application, a database upgrading method is provided, which comprises: obtaining a first structured query language request flow of an initial database, extracting key information of n first structured query language statements in the first structured query language request flow, and generating key information of the n first structured query language statements, wherein n is an integer and n is greater than 1; analyzing the key information of the n first structured query language statements to generate morphing rules of the n first structured query language statements; based on the morphing rules of the n first structured query language statements, data morphing is performed on the n first structured query language statements to generate n second structured query language statements; and based on the n second structured query language statements, the initial database is upgraded to generate a target database.

[0006] According to an embodiment of the present application, the text extraction large language model is generated, including: key information extraction is performed on n first structured query language sentences in the first structured query language request traffic, and key information of the n first structured query language sentences is generated, including: the n first structured query language sentences in the first structured query language request traffic are parsed, and database instance identifiers of the n first structured query language sentences in the first structured query language request traffic are obtained; and according to the database instance identifiers of the n first structured query language sentences, key information extraction is performed on the n first structured query language sentences, and the key information of the n first structured query language sentences is generated, the key information including: table name, field name and parameter value.

[0007] According to an embodiment of the present application, the key information of the n first structured query language sentences is analyzed, and the deformation rules of the n first structured query language sentences are generated, including: the key information of the n first structured query language sentences is parsed, and the table name and the field name of the n first structured query language sentences are determined; and based on the database instance identifiers, the table names and the field names of the n first structured query language sentences, the deformation rules of the n first structured query language sentences are obtained from a preset deformation rule library, wherein the preset deformation rule library includes: m rule libraries stored in the form of key-value pairs with database instance identifiers, table names and field names as keys and deformation rules as values, wherein m is an integer, and m is greater than 1.

[0008] According to an embodiment of the present application, the key information of the n first structured query language sentences is analyzed, and the deformation rules of the n first structured query language sentences are generated, including: the key information of the n first structured query language sentences is parsed, and the parameter values of the n first structured query language sentences are determined; the parameter values of the n first structured query language sentences are analyzed, and sensitive data of the n first structured query language sentences are generated; and the sensitive data of the n first structured query language sentences are input into a pre-trained local deformation rule matching model, and the deformation rules of the n first structured query language sentences are output.

[0009] According to an embodiment of the present application, based on the n second structured query language sentences, performing database upgrade on the initial database to generate a target database comprises: performing traffic feature extraction on the first structured query language request traffic to generate traffic features of the first structured query language request traffic; based on the traffic features of the first structured query language request traffic, generating second structured query language request traffic from the n second structured query language sentences; performing database upgrade on the initial database to generate a test database; performing data playback on the test database using the second structured query language request traffic to obtain a performance index of the data playback; and if the performance index of the data playback is less than a preset threshold, determining the test database as the target database.

[0010] According to an embodiment of the present application, performing database upgrade on the initial database to generate a test database comprises: performing initial data backup on data of the initial database to generate a first database after initial data backup; performing full data deformation on data of the first database to generate a second database after full data deformation; performing deformation data backup on data of the second database to generate a third database after deformation data backup; and performing database upgrade on the third database to generate a test database.

[0011] According to an embodiment of the present application, the method further comprises: if the performance index of the data playback is greater than or equal to a preset threshold, performing database rollback and generating a performance index report for sending.

[0012] According to a second aspect of the present application, a database upgrade apparatus is provided, which comprises: a first generation module configured to obtain first structured query language request traffic of an initial database, and extract key information of n first structured query language sentences in the first structured query language request traffic to generate key information of the n first structured query language sentences, wherein n is an integer and n is greater than 1; a second generation module configured to analyze the key information of the n first structured query language sentences to generate deformation rules of the n first structured query language sentences; a third generation module configured to perform data deformation on the n first structured query language sentences based on the deformation rules of the n first structured query language sentences to generate n second structured query language sentences; and a fourth generation module configured to perform database upgrade on the initial database based on the n second structured query language sentences to generate a target database.

[0013] According to a third aspect of the present application, an electronic device is provided, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0014] According to a fourth aspect of the present application, a computer-readable storage medium is also provided, which stores a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0015] According to a fifth aspect of the present application, a computer program product is also provided, comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0016] The present application obtains the SQL request traffic of the initial database, extracts the key information in the SQL statement in the SQL request traffic, determines the corresponding deformation rule, deforms the SQL statement in the SQL request traffic, generates the deformed SQL statement, and upgrades the initial database based on the deformed SQL statement to generate the target database. The technical means of the database upgrade process is closely combined with the request traffic analysis and the actual application scenario. By extracting the key information from the request traffic of the initial database, it is ensured that the upgraded database can continue to support the query mode of the existing application, and the risk of the business terminal is reduced. By deforming the original key information through the deformation rule, the controllability of the database structure can be realized, and the consistency and security of the data before and after the database upgrade are ensured. Overall, the smooth transition of the database upgrade is realized, which effectively improves the database upgrade efficiency and data security while ensuring the data consistency, continuity and reliability. The technical problems of the existing database upgrade process are solved, such as manual marking of sensitive fields, time-consuming and error-prone, low efficiency, untransformed sensitive data may be leaked to the test environment, causing data security risk, and the existing recording and playing tools are separated from the data deformation tools, which cannot dynamically adapt to the deformation requirements of different scenes, resulting in reduced reliability of the database upgrade. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application, taken in conjunction with the accompanying drawings, in which:

[0018] Figure 1 The application scenario diagram of the database upgrade method, device, equipment, medium and program product according to the embodiments of the present application is schematically shown;

[0019] Figure 2 The flowchart of the database upgrade method according to the embodiments of the present application is schematically shown;

[0020] Figure 3 FIG. 5 is a flowchart illustrating a method of generating key information of n first structured query language statements in a database upgrade method according to an embodiment of the present application;

[0021] Figure 4 FIG. 6 is a flowchart illustrating a method of generating a transformation rule of n first structured query language statements in a database upgrade method according to an embodiment of the present application;

[0022] Figure 5 FIG. 7 is a flowchart illustrating a method of generating a transformation rule of n first structured query language statements in a database upgrade method according to an embodiment of the present application;

[0023] Figure 6 FIG. 8 is a flowchart illustrating a method of generating a target database in a database upgrade method according to an embodiment of the present application;

[0024] Figure 7 FIG. 9 is a flowchart illustrating a method of generating a test database in a database upgrade method according to an embodiment of the present application;

[0025] Figure 8 FIG. 10 is a block diagram illustrating a structure of a database upgrade apparatus according to an embodiment of the present application;

[0026] Figure 9 FIG. 11 is a block diagram illustrating an electronic device suitable for implementing a database upgrade method according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely illustrative and is not intended to limit the scope of the present application. In the following detailed description of embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0028] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present application. The terms "include" and "have" and the like used herein indicate the presence of the described features, steps, operations, and / or components but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein, including technical and scientific terms, have the meanings as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein are to be interpreted as having a meaning that is consistent with the understanding of that term by those having ordinary skill in the art and not in an overly legal or overly formal sense unless otherwise defined.

[0030] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted that the meaning of the expression is at least one of A, B, or C (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0031] Some block diagrams and / or flowcharts are shown in the drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable control apparatus, so that the instructions executed by the processor can create a means for implementing the functions / operations described in the block diagrams and / or flowcharts.

[0032] First, the technical terms appearing in this paper are explained as follows:

[0033] SQL (Structured Query Language), i.e. structured query language, is a standard programming language for managing and operating relational databases. It allows users to store, query, update and manage data.

[0034] SQL request traffic is the sum of real-time, continuous SQL queries and operations initiated by application programs to databases in actual running, user-oriented production environment business systems.

[0035] Database instance identifier is a string or data used to identify and locate a running database instance. It is the key metadata for database connection, management and monitoring.

[0036] SQL recording tool: a tool for recording the SQL request traffic of the initial database in the production environment and playing it back in the target database.

[0037] Data transformation tool: a tool for desensitizing or encrypting sensitive data through a rule-based algorithm to ensure that the data meets the security requirements in the test environment.

[0038] Sensitive field: a data field in the database that contains user privacy, trade secrets, and other data that needs to be protected, such as name, address, card number, etc.

[0039] De-identification or encryption algorithm and parameters defined for specific fields, such as "use mask algorithm for card number field, keep the first 4 digits and the last 4 digits".

[0040] Embodiments of the present application provide a database upgrade method, which comprises: obtaining first structured query language request traffic of an initial database, extracting key information of n first structured query language statements in the first structured query language request traffic, and generating key information of the n first structured query language statements, wherein n is an integer and n is greater than 1. Analyzing the key information of the n first structured query language statements, generating a transformation rule of the n first structured query language statements. Based on the transformation rule of the n first structured query language statements, data transformation is performed on the n first structured query language statements to generate n second structured query language statements. And based on the n second structured query language statements, the initial database is upgraded to generate a target database.

[0041] According to embodiments of the present application, by obtaining SQL request traffic of an initial database, extracting key information in SQL statements in the SQL request traffic, determining the corresponding transformation rule, performing data transformation on the SQL statements in the SQL request traffic, generating transformed SQL statements, and based on the transformed SQL statements, the initial database is upgraded to generate a target database. Technical means, the database library upgrade process is closely combined with the actual application scene through request traffic analysis. By extracting key information from the request traffic of the initial database, it is ensured that the upgraded database can continue to support the query mode of the existing application, reducing the risk of business terminals. By transforming the original key information according to the transformation rule, the controllability of the database structure can be realized, and the consistency and security of the data before and after the database upgrade are guaranteed. Overall, it realizes the smooth transition of database upgrade, effectively improves the database upgrade efficiency and data security, and guarantees the data consistency, continuity and reliability. The technical problems of the prior art are solved, such as the need for manual marking of sensitive fields, which requires analysis of each SQL, is time-consuming and prone to errors, resulting in low efficiency; untransformed sensitive data may be leaked to the test environment, causing data security risks; the existing recording and playing tools are separated from the data transformation tools, and cannot dynamically adapt to the transformation needs of different scenes, resulting in reduced reliability of database upgrade.

[0042] Figure 1 The application scenario diagram of the database upgrade method, device, equipment, medium and program product according to embodiments of the present application is schematically shown. It should be noted that, Figure 1The examples shown are merely examples of scenarios in which the embodiments of this application can be applied, to help those skilled in the art understand the technical content of this application, but do not mean that the embodiments of this application cannot be used in other devices, systems, environments or scenarios.

[0043] like Figure 1 As shown, application scenario 100 according to this embodiment may include a database upgrade application scenario. Network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0044] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0045] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0046] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0047] It should be noted that the database upgrade method provided in this application embodiment can generally be executed by server 105. Correspondingly, the database upgrade device provided in this application embodiment can generally be located in server 105. The database upgrade method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the database upgrade device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0048] It should be understood that Figure 1 the number of terminal devices, networks and servers in the above-mentioned application scenarios is merely illustrative. Any number of terminal devices, networks and servers can be provided according to implementation needs.

[0049] The database upgrade method of the disclosed embodiments will be described in detail based on Figure 1 the scenario described above. Figures 2-7 It should be noted that the above-mentioned application scenarios are merely shown for the purpose of facilitating understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in any way in this respect. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0050] Figure 2 A flowchart of the database upgrade method according to an embodiment of the present application is schematically shown.

[0051] As shown in Figure 2 , the method 200 includes steps S201-S204.

[0052] In step S201, the first structured query language request traffic of an initial database is acquired, and key information of n first structured query language statements in the first structured query language request traffic is extracted to generate key information of the n first structured query language statements, wherein n is an integer and n is greater than 1.

[0053] For example, the traffic recording can be performed by a SQL recording tool, the SQL request traffic of a production environment can be captured by a packet capturing tool or a database agent, and the key information of n SQL statements in the SQL request traffic can be extracted to generate the key information of the n SQL statements.

[0054] Figure 3 A flowchart of generating key information of n first structured query language statements in the database upgrade method according to an embodiment of the present application is schematically shown.

[0055] As shown in Figure 3 , the method 300 includes steps S301-S302.

[0056] In step S301, the n first structured query language statements in the first structured query language request traffic are parsed, and the database instance identifier of the n first structured query language statements in the first structured query language request traffic is acquired.

[0057] For example, the n SQL statements in the SQL request traffic can be parsed by the built-in SQL parser of the recording and playing tool, and the database instance identifier involved in the build connection information is extracted. For a database, the database instance identifier is the unique identifier of itself. For an application, the database instance identifier is the address of the connection target. For operation and maintenance, the database instance identifier is an effective identifier for management and monitoring. For architecture, the database instance identifier is the basis for service discovery and load balancing.

[0058] In step S302, key information of the n first structured query language statements is extracted according to the database instance identifiers of the n first structured query language statements, and the key information of the n first structured query language statements is generated. The key information includes a table name, a field name, and a parameter value.

[0059] For example, the built-in SQL parser of the recording and playing tool parses the key information in the SQL statement. The key information generally includes a table name, a field name, and a parameter value. The table name and the field name can be the table name and the field name involved in the SQL. For example, the A table and the col1 field in SELECT* FROM A WHERE col1 = '123456789012'. The parameter value can be the input parameter value in the SQL statement. For example, '123456789012' in SELECT * FROM A WHERE col1 = '123456789012'.

[0060] In addition, the abstract syntax tree technology can also be used to parse the SQL structure and extract the table name, the field name, and the parameter value.

[0061] By parsing the SQL traffic and obtaining the database instance identifier, the key information can be accurately traced. The accuracy and reliability of the key information acquisition can be effectively improved according to the database instance identifier.

[0062] Referring back Figure 2 In step S202, the key information of the n first structured query language statements is analyzed, and the deformation rules of the n first structured query language statements are generated.

[0063] Figure 4 An embodiment of a flowchart for generating the deformation rules of the n first structured query language statements in the database upgrade method according to the embodiments of the present application is schematically shown.

[0064] As Figure 4 shown, the method 400 includes steps S401-S402.

[0065] Step S401, parse the key information of the n first structured query language statements to determine the table name and field name of the n first structured query language statements.

[0066] For example, the database instance identifier such as prod_db or test_db, the table name such as A or B, and the field name such as col1 or col3 can be obtained by calling the interface provided by the data transformation tool in the database through the recording and playing tool in the database.

[0067] Step S402, based on the database instance identifier, the table name and the field name of the n first structured query language statements, obtain the transformation rule of the n first structured query language statements in the preset transformation rule library, wherein the preset transformation rule library includes: m key-value pairs stored in the form of key-value pairs, wherein m is an integer and m is greater than 1.

[0068] For example, the preset transformation rule library can be stored in the form of key-value pairs, the key is <instance name>: : <field name>, and the value is a transformation algorithm identifier. The preset transformation rule library supports dynamic updating of rules, which can take effect without restarting the recording and playing tool. The data transformation tool queries the corresponding transformation rule from the rule library according to the instance+table+field combination, for example, the card number transformation algorithm is used for the col1 field of the A table.

[0069] By querying and obtaining the transformation rule of each SQL statement from the rule library stored in the form of key-value pairs, the reliability and efficiency of the transformation rule can be effectively improved.

[0070] Figure 5 An embodiment of the method for generating n first structured query language statements in the database upgrade method according to the application is shown schematically.

[0071] As shown in Figure 5 The method 500 includes steps S501-S503.

[0072] Step S501, parse the key information of the n first structured query language statements to determine the parameter value of the n first structured query language statements.

[0073] Step S502, analyze the parameter value of the n first structured query language statements to generate sensitive data of the n first structured query language statements.

[0074] Step S503, input the sensitive data of the n first structured query language statements into the pre-trained local transformation rule matching model to output the transformation rule of the n first structured query language statements.

[0075] The local deformation rule matching model can improve the reliability and acquisition efficiency of deformation rules while ensuring data security.

[0076] Return reference Figure 2 , in step S203, based on the deformation rules of the n first Structured Query Language statements, perform data deformation on the n first Structured Query Language statements to generate n second Structured Query Language statements.

[0077] For example, the recording and playback tool can load the deformation algorithm compression package provided by the data deformation tool to perform deformation processing on the extracted parameter values. For example, if the value of the col1 field is a bank card number of 123456789012, it becomes 1234****9012 after calling the bank card number deformation algorithm. Then replace the sensitive field value in the original SQL with the deformed parameter value to generate a new SQL statement. For example, the original SQL statement is: SELECT * FROM A WHERE col1 = '123456789012'; The deformed SQL statement is: SELECT * FROM A WHERE col1 = '1234****9012'. The original SQL statement is: INSERT INTO A(col1, col2) VALUES ('123456789012','Zhang San'); The deformed SQL statement is: INSERT INTO A(col1, col2) VALUES ('1234****9012', 'Jia Yi').

[0078] Step S204, based on the n second Structured Query Language statements, perform database upgrade on the initial database to generate a target database.

[0079] Figure 6 Schematically shows a flowchart of generating a target database in the database upgrade method according to an embodiment of the present application.

[0080] Such as Figure 6 shown, the method 600 includes steps S601 to S605.

[0081] Step S601, extract traffic characteristics from the first Structured Query Language request traffic to generate traffic characteristics of the first Structured Query Language request traffic.

[0082] For example, the traffic characteristics of the first Structured Query Language request traffic may include the concurrency number and execution interval of the first Structured Query Language request traffic. Extract traffic characteristics from the first SQL request traffic to generate traffic characteristics of the concurrency number and execution interval of the first SQL request traffic.

[0083] Step S602, based on the traffic characteristics of the first structured query language request traffic, request traffic generation is performed on the n second structured query language statements to generate second structured query language request traffic.

[0084] Step S603, database upgrade is performed on the initial database to generate a test database.

[0085] Figure 7 A flowchart illustrating the generation of a test database in the database upgrade method according to an embodiment of the present application is shown.

[0086] As shown in Figure 7 The method 700 includes steps S701-S704.

[0087] Step S701, initial data backup is performed on the data of the initial database to generate a first database after initial data backup.

[0088] Step S702, full data transformation is performed on the data of the first database to generate a second database after full data transformation.

[0089] Step S703, transformed data backup is performed on the data of the second database to generate a third database after transformed data backup.

[0090] Step S704, database upgrade is performed on the third database to generate a test database.

[0091] Through the multiple security isolation technical means of initial data backup on the data of the initial database, full data transformation on the data of the first database, and transformed data backup on the data of the second database, data damage or data loss caused by misoperation, upgrade failure, or transformation algorithm error can be effectively prevented, the fault tolerance is greatly improved, and the reliability and security of database upgrade are significantly improved.

[0092] Referring back to Figure 6 In step S604, the second structured query language request traffic is played back in the test database to obtain the performance index of the data playback.

[0093] For example, after the transformed SQL statements are packaged and transmitted to the test environment, SQL playback is performed in the playback target database according to the traffic characteristics when recording.

[0094] Step S605, if the performance index of the data playback is less than a preset threshold, the test database is determined as a target database.

[0095] For example, the SQL execution time, resource consumption and other indicators in the recording playback process are compared with the production environment to verify the performance after the upgrade or transformation. For example, a threshold value can be preset according to the performance indicators of the initial database in the production environment. If the performance indicators of the data playback are less than the preset threshold value, it indicates that the second SQL request flow is relatively stable when running in the test database, and the test database can be determined as the target database.

[0096] By performing data playback on the second structured query language request flow in the test database and obtaining performance indicators, and determining the test database as the target database according to the expected value of the performance indicators, the efficiency and reliability of upgrading the initial database to generate the target database can be effectively improved. The risk of database upgrade is reduced, and the security of database upgrade is improved.

[0097] According to an embodiment of the present application, the method further comprises: if the performance indicators of the data playback are greater than or equal to the preset threshold value, performing database rollback and generating a performance indicator report for sending.

[0098] For example, if the performance indicators of the data playback are greater than or equal to the preset threshold value, it indicates that the second SQL request flow has a high risk when running in the test database, and database rollback is needed, and a performance indicator report is generated and sent to the relevant technical personnel for timely adjustment of the data.

[0099] Through the database rollback mechanism and the performance indicator report sending mechanism, the database upgrade operation with performance indicators of data playback greater than or equal to the preset threshold value can be effectively avoided, the loss caused by database exception is avoided, and the reliability and credibility of database upgrade are significantly ensured.

[0100] The present application does not need manual marking of sensitive fields, significantly reduces the workload of developers, reduces manual intervention, improves efficiency and accuracy. The sensitive data is deformed in real time during the recording and playback process, avoiding the risk of test environment data and improving data security. By simulating the data distribution and flow characteristics of the production environment, the accuracy of performance testing is improved, and the testing accuracy is enhanced. Dynamic call of deformation rules is supported according to instances, tables and fields, which is suitable for complex business scenarios and supports dynamic rule adaptation. Manual intervention and data synchronization cost are reduced, and the verification efficiency of database upgrade and transformation is improved.

[0101] Figure 8 The structure block diagram of the database upgrade device according to an embodiment of the present application is schematically shown.

[0102] As shown in Figure 8 The device 800 includes a first generation module 801, a second generation module 802, a third generation module 803 and a fourth generation module 804.

[0103] The first generation module 801 is configured to acquire first structured query language request traffic of an initial database, extract key information of n first structured query language statements in the first structured query language request traffic, and generate the key information of the n first structured query language statements, where n is an integer and n is greater than 1. In an embodiment, the first generation module 801 can be configured to perform the step S201 described above.

[0104] The first generation module 801 includes a first acquisition module and a fifth generation module.

[0105] The first acquisition module is configured to parse the n first structured query language statements in the first structured query language request traffic, and acquire database instance identifiers of the n first structured query language statements in the first structured query language request traffic. In an embodiment, the first acquisition module can be configured to perform the step S301 described above, and details are not repeated here.

[0106] The fifth generation module is configured to extract key information of the n first structured query language statements according to the database instance identifiers of the n first structured query language statements, and generate the key information of the n first structured query language statements, where the key information includes a table name, a field name, and a parameter value. In an embodiment, the fifth generation module can be configured to perform the step S302 described above, and details are not repeated here.

[0107] The second generation module 802 is configured to analyze the key information of the n first structured query language statements, and generate n first structured query language statement transformation rules. In an embodiment, the second generation module 802 can be configured to perform the step S202 described above.

[0108] The second generation module 802 includes a first determination module and a second acquisition module.

[0109] The first determination module is configured to parse the key information of the n first structured query language statements, and determine table names and field names of the n first structured query language statements. In an embodiment, the first determination module can be configured to perform the step S401 described above, and details are not repeated here.

[0110] The second obtaining module is configured to obtain, based on the database instance identifiers, the table names and the field names of the n first structured query language statements, the deformation rules of the n first structured query language statements from a preset deformation rule library, where the preset deformation rule library comprises m rule libraries stored in a key-value pair form with database instance identifiers, table names and field names as keys and deformation rules as values, where m is an integer and m is greater than 1. In an embodiment, the second obtaining module can be configured to perform the step S402 described above, and details are not repeated here.

[0111] The second generating module 802 further comprises a second determining module, a sixth generating module and a seventh generating module.

[0112] The second determining module is configured to analyze the key information of the n first structured query language statements and determine the parameter values of the n first structured query language statements. In an embodiment, the second determining module can be configured to perform the step S501 described above, and details are not repeated here.

[0113] The sixth generating module is configured to analyze the parameter values of the n first structured query language statements and generate the sensitive data of the n first structured query language statements. In an embodiment, the sixth generating module can be configured to perform the step S502 described above, and details are not repeated here.

[0114] The seventh generating module is configured to input the sensitive data of the n first structured query language statements into a pre-trained local deformation rule matching model and output the deformation rules of the n first structured query language statements. In an embodiment, the seventh generating module can be configured to perform the step S503 described above, and details are not repeated here.

[0115] The third generating module 803 is configured to perform data deformation on the n first structured query language statements based on the deformation rules of the n first structured query language statements and generate n second structured query language statements. In an embodiment, the third generating module 803 can be configured to perform the step S203 described above, and details are not repeated here.

[0116] The fourth generating module 804 is configured to perform database upgrade on the initial database based on the n second structured query language statements and generate a target database. In an embodiment, the fourth generating module 804 can be configured to perform the step S204 described above.

[0117] The fourth generating module 804 comprises an eighth generating module, a ninth generating module, a tenth generating module, an eleventh generating module and a third determining module.

[0118] An eighth generating module is configured to perform traffic feature extraction on the first structured query language request traffic, and generate a traffic feature of the first structured query language request traffic. In an embodiment, the eighth generating module can be configured to perform the step S601 described above, and details are not repeated here.

[0119] A ninth generating module is configured to generate the n second structured query language statements based on the traffic feature of the first structured query language request traffic, and generate second structured query language request traffic. In an embodiment, the ninth generating module can be configured to perform the step S602 described above, and details are not repeated here.

[0120] A tenth generating module is configured to perform database upgrade on the initial database, and generate a test database. In an embodiment, the tenth generating module can be configured to perform the step S603 described above.

[0121] The tenth generating module includes a twelfth generating module, a thirteenth generating module, a fourteenth generating module, and a fifteenth generating module.

[0122] The twelfth generating module is configured to perform initial data backup on data of the initial database, and generate a first database after initial data backup. In an embodiment, the twelfth generating module can be configured to perform the step S701 described above, and details are not repeated here.

[0123] The thirteenth generating module is configured to perform full-amount data deformation on data of the first database, and generate a second database after full-amount data deformation. In an embodiment, the thirteenth generating module can be configured to perform the step S702 described above, and details are not repeated here.

[0124] The fourteenth generating module is configured to perform deformation data backup on data of the second database, and generate a third database after deformation data backup. In an embodiment, the fourteenth generating module can be configured to perform the step S703 described above, and details are not repeated here.

[0125] The fifteenth generating module is configured to perform database upgrade on the third database, and generate a test database. In an embodiment, the fifteenth generating module can be configured to perform the step S704 described above, and details are not repeated here.

[0126] The eleventh generating module is configured to perform data playback on the second structured query language request traffic in the test database, and obtain a performance index of the data playback. In an embodiment, the eleventh generating module can be configured to perform the step S604 described above, and details are not repeated here.

[0127] The third determining module is configured to determine the test database as the target database if the performance index of the data playback is less than a preset threshold. In an embodiment, the third determining module can be configured to perform the step S605 described above, and thus no further description is given here.

[0128] According to an embodiment of the present application, the apparatus further comprises a sixteenth generating module configured to, if the performance index of the data playback is greater than or equal to a preset threshold, perform database rollback and generate a performance index report for sending.

[0129] According to an embodiment of the present application, any one or more of the first generating module 801, the second generating module 802, the third generating module 803 and the fourth generating module 804 can be combined in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the first generating module 801, the second generating module 802, the third generating module 803 and the fourth generating module 804 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware that can be integrated or packaged, or any one of software, hardware and firmware or any appropriate combination of any of them. Alternatively, at least one of the first generating module 801, the second generating module 802, the third generating module 803 and the fourth generating module 804 can be at least partially implemented as a computer program module which can perform corresponding functions when the computer program module is run.

[0130] Figure 9 The block diagram of an electronic device suitable for implementing the database upgrade method according to an embodiment of the present application is schematically shown.

[0131] As Figure 9As shown, the electronic device 900 according to an embodiment of the present application includes a processor 901 which can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The processor 901 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 901 can also include an on-board memory for cache use. The processor 901 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present application.

[0132] In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method processes according to embodiments of the present application by executing the programs in the ROM 902 and / or the RAM 903. Note that the programs can also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 can also perform various operations of the method processes according to embodiments of the present application by executing the programs stored in the one or more memories.

[0133] According to embodiments of the present application, the electronic device 900 can also include an input / output (I / O) interface 905 which is also connected to the bus 904. The electronic device 900 can also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as necessary. A removable medium 911 such as a magnetic disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 910 as necessary, so that a computer program read out from the removable medium 911 is installed in the storage section 908 as necessary.

[0134] The application further provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist independently without being assembled into the device / apparatus / system. The computer readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the application.

[0135] According to the embodiments of the application, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In this application, a computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to the embodiments of the application, the computer readable storage medium can include one or more of the above-described ROM 902 and / or RAM 903 and / or one or more memories other than the ROM 902 and the RAM 903.

[0136] The embodiments of the application also include a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the database upgrading method provided by the embodiments of the application.

[0137] The above-described functions defined in the system / apparatus of the embodiments of the application are performed when the computer program is executed by the processor 901. According to the embodiments of the application, the above-described system, apparatus, module, unit, etc. can be implemented by computer program modules.

[0138] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium, and be downloaded and installed through the communication part 909, and / or installed from the detachable medium 911. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0139] In such embodiments, the computer program can be downloaded and installed from the network via the communication section 909, and / or installed from the removable media 911. When the computer program is executed by the processor 901, the above-described functions defined in the system of the embodiments of the present application are performed. According to the embodiments of the present application, the system, device, apparatus, module, unit, and the like described above can be implemented by the computer program modules.

[0140] According to the embodiments of the present application, the program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. The programming language includes, but is not limited to, such as Java, C++, python, "C" language, or similar programming language. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0141] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0142] Those skilled in the art can understand that the features described in various embodiments of the present application can be combined and / or integrated in various combinations and / or integrations, even if such combinations or integrations are not explicitly described in the present application. In particular, the features described in various embodiments of the present application can be combined and / or integrated in various combinations and / or integrations without departing from the spirit and teachings of the present application. All such combinations and / or integrations fall within the scope of the present application.

Claims

1. A database upgrade method, characterized in that, The method includes: Obtain the first structured query language request traffic of the initial database, extract key information from n first structured query language statements in the first structured query language request traffic, and generate key information of the n first structured query language statements, where n is an integer and n is greater than 1; Analyze the key information of the n first structured query language statements and generate transformation rules for the n first structured query language statements; Based on the transformation rules of n first structured query language statements, the n first structured query language statements are transformed to generate n second structured query language statements; and Based on the n second structured query language statements, the initial database is upgraded to generate the target database.

2. The method according to claim 1, characterized in that, Extract key information from n first structured query language statements in the first structured query language request traffic to generate key information for the n first structured query language statements, including: Parse the n first structured query language statements in the first structured query language request traffic to obtain the database instance identifiers of the n first structured query language statements in the first structured query language request traffic; and Based on the database instance identifiers of the n first structured query language statements, key information is extracted from the n first structured query language statements to generate key information for the n first structured query language statements. The key information includes: table name, field name, and parameter value.

3. The method according to claim 2, characterized in that, Analyze the key information of the n First Structured Query Language statements to generate n transformation rules for the First Structured Query Language statements, including: Parse the key information of the n First Structured Query Language (SCL) statements to determine the table names and field names of the n SCL statements; and Based on the database instance identifier, table name, and field name of the n first structured query language statements, the transformation rules of the n first structured query language statements are obtained from the preset transformation rule base. The preset transformation rule base includes m rule bases stored in the form of key-value pairs with database instance identifier, table name, and field name as keys and transformation rules as values, where m is an integer and m is greater than 1.

4. The method according to claim 2, characterized in that, Analyze the key information of the n First Structured Query Language statements to generate n transformation rules for the First Structured Query Language statements, including: Analyze the key information of the n first structured query language statements to determine the parameter values ​​of the n first structured query language statements; Analyze the parameter values ​​of the n first structured query language statements to generate sensitive data for the n first structured query language statements; and The sensitive data of the n first structured query language statements are input into a pre-trained local variation rule matching model, and the variation rules of the n first structured query language statements are output.

5. The method according to any one of claims 1 to 4, characterized in that, Based on the n second structured query language statements, the initial database is upgraded to generate the target database, including: Traffic features are extracted from the first structured query language request traffic to generate traffic features of the first structured query language request traffic; Based on the traffic characteristics of the first structured query language request traffic, request traffic is generated for the n second structured query language statements to generate second structured query language request traffic. Upgrade the initial database to generate a test database; The second structured query language request traffic is replayed in the test database, and the performance metrics of the data replay are obtained; and If the performance metrics of the data playback are less than a preset threshold, then the test database is determined as the target database.

6. The method according to claim 5, characterized in that, Upgrade the initial database to generate a test database, including: Perform an initial data backup on the initial database to generate a first database after the initial data backup; Perform full data transformation on the data in the first database to generate a second database after full data transformation; Perform a modified data backup on the second database to generate a third database with the modified data backup; and The third database is upgraded to generate a test database.

7. The method according to claim 5, characterized in that, The method also includes: If the performance metrics of the data playback are greater than or equal to a preset threshold, then database rollback is performed and a performance metric report is generated and sent.

8. A database upgrade device, characterized in that, The device includes: The first generation module is used to obtain the first structured query language request traffic of the initial database, extract key information from n first structured query language statements in the first structured query language request traffic, and generate key information of the n first structured query language statements, where n is an integer and n is greater than 1; The second generation module is used to analyze the key information of the n first structured query language statements and generate transformation rules for the n first structured query language statements. The third generation module is used to perform data transformation on the n first structured query language statements based on the transformation rules of the n first structured query language statements, to generate n second structured query language statements; and The fourth generation module is used to upgrade the initial database based on the n second structured query language statements to generate the target database.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images