Database configuration method, apparatus, terminal device, and computer program product

Abstract

The application relates to a database configuration method and device, a terminal equipment and a computer program product. The method comprises the following steps: obtaining a configuration entity, the configuration entity comprising at least a plurality of SQL instructions and a plurality of row-level fingerprints, each row-level fingerprint corresponding to an SQL instruction; generating a first configuration sequence comprising a plurality of configuration tasks based on the data version difference between the configuration entity and a database to be configured, each configuration task corresponding to a row-level fingerprint; inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into an audit table when each configuration task is executed; and determining that the placeholder record is inserted successfully and executing the SQL instruction corresponding to the row-level fingerprint in the case that the audit table does not comprise an index corresponding to the row-level fingerprint. Through the application, the problem of poor breakpoint continuation performance of the database configuration method taking a script file as an execution unit in the related art is solved, and the technical effect of improving the configuration efficiency and reliability of the database is achieved.

Description

Technical Field

[0001] This application belongs to the field of database technology, and in particular relates to a database configuration method, apparatus, terminal equipment, and computer program product. Background Technology

[0002] In microservices and hybrid cloud architectures, database schemas (schema or architecture) change very frequently. Traditional tools often configure the database using script files as the execution unit. When some instructions in the script file fail to configure, the entire script file needs to be reconfigured. These tools have poor resuming performance and are prone to data duplication or structural conflicts during resuming, which reduces the efficiency and reliability of database configuration.

[0003] There is no effective solution yet to address the poor performance of breakpoint resume in database configuration methods that use script files as the execution unit. Summary of the Invention

[0004] This application provides a database configuration method, apparatus, terminal device, and computer program product to at least solve the problem of poor resume performance of database configuration methods that use script files as the execution unit in the related art.

[0005] In a first aspect, embodiments of this application provide a database configuration method, the method comprising: obtaining a configuration entity, the configuration entity including at least a plurality of SQL instructions and a plurality of row-level fingerprints, each row-level fingerprint corresponding to one SQL instruction; generating a first configuration sequence including a plurality of configuration tasks based on data version differences between the configuration entity and the database to be configured, each configuration task corresponding to one row-level fingerprint; when executing each configuration task, inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into an audit table; if the audit table does not include an index corresponding to the row-level fingerprint, determining that the placeholder record was successfully inserted, and executing the SQL instruction corresponding to the row-level fingerprint.

[0006] In some embodiments, inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table includes: inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table, and configuring the status of the placeholder record as pending execution; and after determining that the placeholder record has been successfully inserted and executing the SQL instruction corresponding to the row-level fingerprint, the method further includes: if the SQL instruction is executed successfully, configuring the status of the placeholder record as execution completed; if the SQL instruction fails to execute, configuring the status of the placeholder record as execution exception.

[0007] In some embodiments, after determining that the placeholder record was successfully inserted and executing the SQL instruction corresponding to the row-level fingerprint, the method further includes: in the event that the configuration process of the database to be configured is interrupted and restarted, querying from each of the indexes of the audit table for placeholder records with a status of pending execution or execution error; based on the position of the configuration task corresponding to the placeholder record with a status of pending execution or execution error in the first configuration sequence, reconstructing the first configuration sequence to obtain a second configuration sequence, the second configuration sequence including the configuration task corresponding to the placeholder record with a status of pending execution or execution error, and the configuration tasks that have not been executed among the multiple configuration tasks; and continuing to configure the database to be configured according to the second configuration sequence.

[0008] In some embodiments, the configuration entity further includes instruction information corresponding to each SQL instruction; inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table when executing each configuration task includes: parsing the row-level fingerprint corresponding to the configuration task to obtain the SQL instruction corresponding to the configuration task when executing each configuration task; obtaining the instruction information of the SQL instruction from the configuration entity, and creating a distributed lock key based on the instruction information, wherein the distributed lock corresponding to the distributed lock key is released when the SQL instruction is successfully executed; and, if the distributed lock key is successfully created, inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table, and creating an index uniquely associated with the configuration task in the audit table based on the row-level fingerprint and the instruction information.

[0009] In some embodiments, obtaining the configuration entity includes: obtaining metadata for configuring the database to be configured; calling a dialect parser corresponding to the type of the database to be configured to parse the metadata to detect whether there are any abnormal instructions in the metadata; if there are no abnormal instructions in the metadata, performing a hash operation on each SQL instruction in the metadata to obtain a row-level fingerprint corresponding to each SQL instruction; and encapsulating the metadata, each SQL instruction, and each row-level fingerprint into the configuration entity.

[0010] In some embodiments, the step of hashing each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction includes: for each SQL instruction, if the SQL instruction includes a persistent change instruction, hashing the SQL instruction to obtain the row-level fingerprint corresponding to the SQL instruction; if the SQL instruction includes a session context instruction, injecting a random offset into the SQL instruction, and hashing the SQL instruction with the injected random offset to obtain the row-level fingerprint corresponding to the SQL instruction.

[0011] In some embodiments, generating a first configuration sequence comprising multiple configuration tasks based on the data version difference between the configuration entity and the database to be configured includes: obtaining a first version corresponding to the database to be configured and a second version corresponding to the configuration entity; extracting all version nodes from the configuration entity and the dependencies between the version nodes, wherein each version node includes a unique identifier for each version; constructing a version dependency directed acyclic graph (DAG) based on all version nodes and the dependencies between them; if the version dependency DAG does not contain cycles and the dependency path from the first version to the second version is reachable, enumerating all configuration paths starting from the first version and ending at the second version; and generating the first configuration sequence based on the shortest configuration path among all configuration paths.

[0012] Secondly, embodiments of this application provide a database configuration apparatus, the apparatus comprising: an acquisition module, configured to acquire a configuration entity, the configuration entity including at least a plurality of SQL instructions and a plurality of row-level fingerprints, each row-level fingerprint corresponding to one of the SQL instructions; a generation module, configured to generate a first configuration sequence including a plurality of configuration tasks based on the data version differences between the configuration entity and the database to be configured, each configuration task corresponding to one of the row-level fingerprints; an insertion module, configured to insert the row-level fingerprint corresponding to the configuration task as a placeholder record into an audit table when executing each configuration task; and an execution module, configured to determine that the placeholder record has been successfully inserted and execute the SQL instruction corresponding to the row-level fingerprint if the audit table does not include the index corresponding to the row-level fingerprint.

[0013] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the database configuration method described in any one of the first aspects above.

[0014] Fourthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the database configuration method described in any one of the first aspects to be executed.

[0015] Compared to related technologies, the database configuration method, apparatus, terminal device, and computer program product provided in this application can obtain a configuration entity, which includes at least multiple SQL instructions and multiple row-level fingerprints. Each row-level fingerprint corresponds to one SQL instruction. Based on the data version differences between the configuration entity and the database to be configured, a first configuration sequence including multiple configuration tasks can be generated, with each configuration task corresponding to one row-level fingerprint. Then, when executing each configuration task, the row-level fingerprint corresponding to the configuration task can be inserted as a placeholder record into the audit table. Finally, if the audit table does not include the index corresponding to the row-level fingerprint, it can be determined that the placeholder record was successfully inserted, and the SQL instruction corresponding to the row-level fingerprint can be executed. By refining the execution unit of the configuration database from "script file" to "SQL instruction" and generating an independent row-level fingerprint for each SQL instruction, and performing uniqueness verification of the audit table using the row-level fingerprint as a placeholder record before execution, idempotent control at the instruction level can be achieved. This solves the problems of poor breakpoint resumption performance and easy data duplication or structural conflicts caused by using script files as the execution unit in related technologies, significantly improving the configuration efficiency and reliability of the database. This application solves the problem of poor resume performance of database configuration methods that use script files as the execution unit in related technologies, and achieves the technical effect of improving the configuration efficiency and reliability of the database.

[0016] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a database configuration method according to an embodiment of this application; Figure 2 This is a flowchart of a database configuration method according to another embodiment of this application; Figure 3 This is a flowchart of a database configuration method according to another embodiment of this application; Figure 4This is a flowchart of a database configuration method according to another embodiment of this application; Figure 5 This is a flowchart of a database configuration method according to another embodiment of this application; Figure 6 This is a flowchart of a database configuration method according to another embodiment of this application; Figure 7 This is a schematic diagram of the structure of a database configuration device according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a terminal device according to an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] In microservices and hybrid cloud architectures, database schemas (schema or architecture) change very frequently. Traditional tools often configure the database using script files as the execution unit. When facing heterogeneous databases and large-scale database sharding scenarios, they suffer from bottlenecks such as difficulty in resuming interrupted downloads, inability to proactively intercept risks during the build phase, and weak cross-database orchestration capabilities.

[0026] For example, when some instructions in a script file fail to be configured, the entire script file needs to be reconfigured. These tools have poor resuming performance and are prone to data duplication or structural conflicts during resuming, which reduces the efficiency and reliability of database configuration. At the same time, database syntax errors are usually only discovered during actual runtime, and these risks cannot be prevented in advance during the code building phase.

[0027] There is no effective solution yet to address the poor performance of breakpoint resume in database configuration methods that use script files as the execution unit.

[0028] In view of this, embodiments of this application provide a database configuration method that can obtain a configuration entity, which includes at least multiple SQL commands and multiple row-level fingerprints. Each row-level fingerprint corresponds to one SQL command. Based on the data version differences between the configuration entity and the database to be configured, a first configuration sequence including multiple configuration tasks can be generated, with each configuration task corresponding to one row-level fingerprint. Then, when executing each configuration task, the row-level fingerprint corresponding to the configuration task can be inserted as a placeholder record into an audit table. Finally, if the audit table does not include the index corresponding to the row-level fingerprint, it can be determined that the placeholder record was successfully inserted, and the SQL command corresponding to the row-level fingerprint can be executed. By refining the execution unit of the configuration database from "script file" to "SQL command" and generating an independent row-level fingerprint for each SQL command, and performing uniqueness verification of the audit table using the row-level fingerprint as a placeholder record before execution, idempotent control at the command level can be achieved. This solves the problems of poor breakpoint resumption performance and easy data duplication or structural conflicts caused by using script files as the execution unit in related technologies, significantly improving the configuration efficiency and reliability of the database. This application solves the problem of poor resume performance of database configuration methods that use script files as the execution unit in related technologies, and achieves the technical effect of improving the configuration efficiency and reliability of the database.

[0029] The following will combine Figure 1 This application describes a database configuration method according to one embodiment. Please refer to [link to documentation]. Figure 1 , Figure 1 This is a flowchart of a database configuration method according to an embodiment of this application, such as... Figure 1 As shown, the method includes: Step S101: Obtain the configuration entity. The configuration entity includes at least multiple SQL commands and multiple row-level fingerprints, with each row-level fingerprint corresponding to one SQL command.

[0030] In this embodiment, row-level fingerprints can serve as unique identifiers for SQL commands. That is, each SQL command in the configuration entity corresponds uniquely to a row-level fingerprint. Subsequently, when configuring the database to be configured, the row-level fingerprint of the SQL command and the audit table can be used to determine whether the SQL command has been executed.

[0031] In this embodiment, the configuration entity is generated by the build-time plugin during the code compilation phase and uploaded to a centralized object store after generation. This configuration entity can serve as the sole basis for changes executed in the database production environment, ensuring strict consistency of execution across different environments.

[0032] Specifically, developers can submit metadata (e.g., including SQL scripts) used to configure the database to be configured (e.g., upgrade or update) to a Git repository, triggering a pipeline in the Continuous Integration / Continuous Delivery (CI / CD) system. The pipeline's execution flow can be referenced. Figure 2 .

[0033] Figure 2 This is a flowchart of a database configuration method according to another embodiment of this application, such as... Figure 2 As shown, in one embodiment, the "obtaining configuration entity" step S101 above includes: Step S201: Obtain metadata for configuring the database to be configured.

[0034] In this embodiment, metadata can refer to all data used to configure the database to be configured (e.g., including SQL scripts). As an example, metadata may include multiple SQL commands, as well as the context and constraint data corresponding to each SQL command (e.g., stage identifier, type of database to be configured, version information, shard identifier, etc.).

[0035] Step S202: Call the dialect parser corresponding to the type of the database to be configured to parse the metadata in order to detect whether there are any abnormal instructions in the metadata.

[0036] In this embodiment, when developers package and build the code, the SQL script can be streamed by integrating a metadata parsing plugin. This plugin can call the corresponding dialect parser for the type of the database to be configured and detect whether there are any abnormal instructions in the metadata. As an example, abnormal instructions may include: statements that do not conform to the syntax of the database to be configured, unsupported heterogeneous operations, and high-risk operations (e.g., DROP DATABASE or DELETE statements without conditions).

[0037] If any of the above-mentioned abnormal instructions are detected in the metadata, the pipeline will be terminated immediately, and corresponding alarms or error messages will be issued.

[0038] Step S203: If there are no abnormal instructions in the metadata, perform a hash operation on each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction.

[0039] In this embodiment, if there are no abnormal instructions in the metadata, a hash operation can be performed on each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction.

[0040] In one embodiment, the step S203 above, "performing a hash operation on each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction", includes: for each SQL instruction, if the SQL instruction includes a persistent change instruction, performing a hash operation on the SQL instruction to obtain the row-level fingerprint corresponding to the SQL instruction; if the SQL instruction includes a session context instruction, injecting a random offset into the SQL instruction, and performing a hash operation on the SQL instruction with the injected random offset to obtain the row-level fingerprint corresponding to the SQL instruction.

[0041] In this embodiment, SQL commands in metadata are typically divided into two types: persistent change commands and session context commands. Persistent change commands include Data Definition Language (DDL) and Data Manipulation Language (DML) commands, which, upon execution, have a permanent impact on the database structure or data (i.e., affect database operations) and require strict idempotency control. Session context commands, on the other hand, only affect the environment variables or connection context of the current session and do not produce persistent changes (i.e., do not affect database operations), such as the USE and SET commands.

[0042] For persistent change instructions, a hash operation can be performed using the SHA-256 hash algorithm, and the generated Universally Unique Identifier (UUID) that uniquely corresponds to the persistent change instruction can be used as the row-level fingerprint of the persistent change instruction. For session context instructions, a random offset (e.g., a nanosecond-level timestamp or a random sequence) can be injected into them first, and then the session context instruction with the injected random offset can be hashed using the SHA-256 hash algorithm to obtain the row-level fingerprint of the session context instruction.

[0043] In this way, it can be ensured that session context instructions (usually as auxiliary statements) can have independent placeholders (also known as placeholder records) in the audit table. This avoids some session context instructions being skipped incorrectly due to placeholder record conflicts when the same session context instructions exist in different tasks.

[0044] Step S204: Encapsulate the metadata, each SQL command, and each row-level fingerprint into a configuration entity.

[0045] In this embodiment, the plugin can also parse the comments or metadata of the SQL script to determine the stage identifier (e.g., pre corresponds to before the service shutdown, mid corresponds to during the service shutdown, and post corresponds to after the service shutdown).

[0046] Then, the parsed metadata, row-level fingerprints, stage identifiers, etc., can be encapsulated into configuration entities and uploaded to a centralized object storage.

[0047] Step S102: Based on the data version difference between the configuration entity and the database to be configured, a first configuration sequence including multiple configuration tasks is generated, with each configuration task corresponding to a row-level fingerprint.

[0048] In this embodiment, version topology dependency calculation and configuration path planning can be performed based on the data version difference between the configuration entity and the database to be configured, to obtain a first configuration sequence including multiple configuration tasks, and each configuration task corresponds to a row-level fingerprint, that is, each configuration task corresponds to an SQL instruction.

[0049] Figure 3 This is a flowchart of a database configuration method according to another embodiment of this application, such as... Figure 3 As shown, in one embodiment, step S102 above, "generating a first configuration sequence including multiple configuration tasks based on the data version difference between the configuration entity and the database to be configured," includes: Step S301: Obtain the first version corresponding to the database to be configured and the second version corresponding to the configuration entity.

[0050] In this embodiment, the execution engine can obtain the configuration entity from the centralized object storage, obtain the audit table corresponding to the database to be configured, read the first version of the database to be configured from the audit table (the first version is, for example, the position or version number of the last successful configuration of the database to be configured recorded in the audit table), and take the latest version represented by the configuration entity as the second version.

[0051] Step S302: Extract all version nodes and the dependencies between version nodes from the configuration entity. Each version node includes a unique identifier for each version.

[0052] In this embodiment, all version nodes (which can be version association identifiers in the configuration entity) and the dependencies between each version node can be obtained from the configuration entity. Each version node includes a unique identifier for each version.

[0053] Step S303: Based on all version nodes and the dependencies between each version node, construct a version dependency directed acyclic graph (DAG).

[0054] In this embodiment, a version dependency DAG can be constructed using version nodes as vertices and dependencies as directed edges.

[0055] Step S304: If there is no loop in the version dependency DAG and the dependency path from the first version to the second version is reachable, enumerate all configuration paths starting from the first version and ending with the second version.

[0056] In this embodiment, loop detection and dependency path detection can be performed on the version dependency DAG. Loop detection determines whether there is a loop in the version dependency DAG (e.g., V1 version → V2 version → V2 version, where "→" indicates a dependency relationship). If a loop exists, the configuration process is terminated, and corresponding alarms and error handling are performed. Dependency path detection determines whether the predecessor versions that each version depends on exist in the graph. If the dependency chain is incomplete, the configuration process is terminated, and corresponding alarms and error handling are performed.

[0057] In this embodiment, if there are no loops in the version dependency DAG and the dependency path from the first version to the second version is reachable, all configuration paths can be enumerated starting from the first version and ending at the second version.

[0058] Step S305: Generate the first configuration sequence based on the shortest configuration path among all configuration paths.

[0059] In this embodiment, a shortest path algorithm (e.g., combining topological sorting and dynamic programming or calculating the path length according to the edge weights) can be used to enumerate all configuration paths; then, the configuration path with the shortest path (i.e. the fewest version nodes passed through) can be selected from all configuration paths, and this configuration path can support cross-version skip upgrades.

[0060] Then, the version nodes along the shortest configuration path can be output sequentially to form an ordered first configuration sequence. This first configuration sequence can be returned to the execution engine for subsequent database configuration operations.

[0061] Step S103: When executing each configuration task, insert the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table.

[0062] In this embodiment, the execution engine can generate a first configuration sequence according to step S102 and execute each configuration task sequentially. When executing a single configuration task, atomic placeholder operations can be performed. Specifically, a placeholder record can be constructed based on the row-level fingerprint corresponding to the current configuration task (i.e., the configuration task being executed) and inserted into the audit table.

[0063] If the placeholder record does not exist in the audit table, the insertion is successful. In this case, the placeholder record indicates that the current configuration task has not yet been executed, and the execution engine obtains the permission to continue execution. Then, it proceeds to step S104 to execute the corresponding SQL instruction. If the placeholder record already exists in the audit table, the insertion operation fails due to violation of the unique constraint. In this case, the execution engine determines that the current configuration task has been processed by other nodes, skips the current configuration task, and continues to process the next configuration task in the first configuration sequence.

[0064] Figure 4 This is a flowchart of a database configuration method according to another embodiment of this application, such as... Figure 4 As shown, in one embodiment, step S103 includes: Step S401: When executing each configuration task, the row-level fingerprint corresponding to the configuration task is parsed to obtain the SQL command corresponding to the configuration task.

[0065] In this embodiment, when the execution engine traverses the first configuration sequence, it can obtain the row-level fingerprint corresponding to the current configuration task, and use this fingerprint as the key to retrieve the corresponding SQL instruction and its instruction information from the configuration entity. This instruction information may include the SQL instruction's task type, business identifier, and storage type, etc.

[0066] Step S402: Obtain the instruction information of the SQL instruction from the configuration entity, and create a distributed lock key based on the instruction information. The distributed lock corresponding to the distributed lock key is released when the SQL instruction is executed successfully.

[0067] In this embodiment, a distributed lock key can be constructed based on the instruction information of the SQL command (e.g., task type, business identifier, and storage type). Then, the execution engine can send a corresponding command to the Redis cluster to attempt to create the distributed lock key and set a reasonable expiration time to prevent the distributed lock from being permanently occupied due to node crashes.

[0068] Step S403: If the distributed lock key is successfully created, insert the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table, and create an index in the audit table that is uniquely associated with the configuration task based on the row-level fingerprint and instruction information.

[0069] In this embodiment, if the distributed lock key is successfully created, the execution engine can insert the row-level fingerprint corresponding to the current configuration task as a placeholder record into the audit table. If the placeholder record insertion fails, it indicates that the current configuration task has been executed by another node. In this case, the current configuration task is skipped, and the distributed lock obtained in step S402 is released.

[0070] In this embodiment, a multidimensional composite unique index can be constructed in the audit table based on the row-level fingerprint, task type, business identifier, and storage type corresponding to the SQL command.

[0071] In addition, if the placeholder record is successfully inserted, the SQL instruction corresponding to the current configuration task will be executed successfully, and the distributed lock corresponding to the distributed lock key will also be released; or, if the SQL instruction corresponding to the current configuration task fails to execute, the execution engine will also release the distributed lock after catching the exception, leaving it for subsequent retries.

[0072] Through the steps S401 to S403 above, creating a distributed lock key can prevent multiple nodes from concurrently executing the same configuration task. At the same time, the unique index design of row-level fingerprints in the audit table can also prevent the same node or cross-nodes from repeatedly executing the same configuration task. This can jointly improve the configuration efficiency and reliability of the database.

[0073] Step S104: If the audit table does not include the index corresponding to the row-level fingerprint, determine that the placeholder record was successfully inserted and execute the SQL command corresponding to the row-level fingerprint.

[0074] In this embodiment, if there is no index in the audit table that is completely consistent with the row-level fingerprint corresponding to the current configuration task, it is determined that the placeholder record insertion was successful, the current configuration task has not yet been executed, and the execution engine successfully obtains execution permission; if there is an index in the audit table that is completely consistent with the row-level fingerprint corresponding to the current configuration task, it is determined that the insertion operation of the placeholder record failed due to violation of the unique constraint principle, the execution engine determines that the current configuration task has been executed by other nodes, and directly skips the current configuration task and continues to process the next configuration task in the first configuration sequence.

[0075] Figure 5 This is a flowchart of a database configuration method according to another embodiment of this application, such as... Figure 5 As shown, in one embodiment, after step S102 described above, the method further includes: Step S501: When executing each configuration task, insert the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table, and configure the status of the placeholder record as pending execution.

[0076] Step S502: If the audit table does not include the index corresponding to the row-level fingerprint, determine that the placeholder record was successfully inserted and execute the SQL command corresponding to the row-level fingerprint.

[0077] Step S503: If the SQL command is executed successfully, the status of the placeholder record is configured as execution completed; if the SQL command fails to execute, the status of the placeholder record is configured as execution exception.

[0078] In this embodiment, in the event that an SQL instruction fails to execute, in addition to configuring the status of the placeholder record as execution exception, the exception stack summary of the currently configured task (e.g., error code and error message) can also be recorded for subsequent analysis and retry.

[0079] Through the above steps S501 to S503, the audit table can record the complete execution lifecycle of each configuration task. For example, if the status of the placeholder record corresponding to the current configuration task is PENDING, it means that the current configuration task has been placed but has not yet been executed; if the status of the placeholder record corresponding to the current configuration task is SUCCESS, it means that the current configuration task has been successfully executed; if the status of the placeholder record corresponding to the current configuration task is FAILED, it means that the current configuration task has failed to execute and needs to be retried.

[0080] In this way, when the configuration process of the database to be configured is interrupted due to node failure, network interruption, or manual termination, the execution engine can accurately locate incomplete or failed configuration tasks by scanning placeholder records in the audit table that are in a pending or abnormal execution state after restarting. It can then reconstruct the second configuration sequence according to the order of the first configuration sequence, thereby achieving SQL command-level breakpoint resumption. Simultaneously, due to the existence of indexes in the audit table, successfully executed configuration tasks (with a SUCCESS status) will not be executed repeatedly, avoiding the risk of data duplication or structural conflicts.

[0081] Figure 6 This is a flowchart of a database configuration method according to another embodiment of this application, such as... Figure 6 As shown, in one embodiment, after step S104 above, the method further includes: Step S601: In the event that the configuration process of the database to be configured is interrupted and restarted, query the placeholder records with a status of pending execution or execution error from each index of the audit table.

[0082] Step S602: Based on the position of the configuration task corresponding to the placeholder record with the status of pending execution or execution error in the first configuration sequence, the first configuration sequence is reconstructed to obtain the second configuration sequence. The second configuration sequence includes the configuration task corresponding to the placeholder record with the status of pending execution or execution error, as well as the configuration task that has not been executed among multiple configuration tasks.

[0083] Step S603: Continue configuring the database to be configured according to the second configuration sequence.

[0084] In this embodiment, when the configuration process of the database to be configured is interrupted and restarted due to node failure or crash, network failure, master-slave switching or manual termination, the status statistics of each placeholder record can be statistically analyzed through the audit table to achieve accurate breakpoint resume without manual intervention.

[0085] After the configuration process of the database to be configured restarts, the execution engine can start and scan the audit table, construct query conditions, and filter out placeholder records with a status of PENDING or FAILED. Then, based on the first configuration sequence, the configuration task corresponding to each placeholder record with a status of PENDING or FAILED can be determined, as well as the original order of the configuration tasks that have not yet been executed in the first configuration sequence (i.e., configuration tasks for which placeholder records have not been inserted into the audit table) in the first configuration sequence. The second configuration sequence is then reconstructed by rearranging these original orders.

[0086] Then, the execution engine can execute each configured task in the second configuration sequence in turn.

[0087] In this way, without manual intervention, the configuration process can be automatically resumed from the point of interruption after the database configuration process is interrupted. Based on the status statistics of each placeholder record in the audit table, successfully executed configuration tasks can be eliminated, thereby avoiding problems such as data duplication or structural conflicts caused by repeated execution of configuration tasks. In addition, even if the original node in the execution engine permanently fails (e.g., due to a fault or crash), a new node can rebuild the execution queue by scanning the audit table and take over to complete the task.

[0088] In one embodiment, when executing the SQL command in step S104 above, the heterogeneous storage adaptation and physical topology unfolding process can be further executed according to the type of the database to be configured and whether the SQL command involves horizontal sharding, thereby supporting accurate execution and breakpoint resume in complex environments.

[0089] For complex sharded environments, the execution engine can obtain the target database type of the SQL command from the configuration entity. It can pre-configure dialect adapters for different database types, each responsible for converting standardized execution requests into a syntax format recognizable by the target database. Through this adaptation mechanism, multiple heterogeneous database types can be handled with a unified execution flow, eliminating the need to develop separate upgrade tools for each database.

[0090] When SQL commands involve horizontal sharding, the execution engine can perform a physical topology unpacking process, breaking down logical commands into sub-commands targeting multiple physical shards. Specifically, it can identify the horizontal splitting identifier of the SQL command and automatically parse the physical sharding topology, generating commands with independent sub-sequence fingerprints for each physical shard, ensuring that the sub-sequence fingerprints of different physical shards are unique.

[0091] Then, the execution engine can create a parent task node for SQL commands involving horizontal sharding and a child task node for each physical shard, forming a "parent task-child task" state tree. Based on this state tree, the status of each child task (e.g., PENDING, SUCCESS, or FAILED) can be recorded independently in the audit table, and a relationship can be established between the child sequence fingerprint and the row-level fingerprint corresponding to the parent task.

[0092] During the execution phase of the SQL commands involving horizontal sharding mentioned above, the execution engine can traverse each subtask in the state tree. For each subtask, it queries the audit table for the status corresponding to the subsequence fingerprint of the task. If the status is SUCCESS, the subtask is skipped and the next subtask is executed. If the status is PENDING or FAILED, the subtask is added to the execution queue.

[0093] If the configuration process of the database to be configured is interrupted during the execution phase of the SQL commands involving horizontal sharding, the following process will be used to achieve precise breakpoint resumption after restarting: Scan the status corresponding to all subsequence fingerprints in the audit table and filter out the subsequence fingerprints with the status of PENDING or FAILED; rebuild the execution queue according to the position of the subtasks corresponding to the subsequence fingerprints with the status of PENDING or FAILED in the status tree and the sharding order recorded in the status tree.

[0094] In this way, the execution status of each subtask can be independently tracked through subsequence fingerprints and state trees; and after the execution of SQL instructions involving horizontal sharding is interrupted, the status statistics of the audit table can be used to filter out the subtasks with the status of PENDING or FAILED, and these subtasks can be retried to avoid secondary disturbance to the subtasks that have been successfully executed, thereby improving the configuration efficiency and reliability of the database.

[0095] By refining the execution unit of the database configuration process to the instruction level using the above methods, the problems of poor performance in resuming interrupted downloads and easy data duplication or structural conflicts caused by using script files as the execution unit in related technologies can be solved. In addition, by performing anomaly detection on the code during the construction phase of the configuration entity, the syntax risks in the database configuration process can be brought forward to the development phase, which can reduce the runtime anomaly rate after the database is actually put into use. Furthermore, by unifying the multimodal upgrade process and combining it with subsequence fingerprinting, the problem of resuming interrupted downloads in large-scale table partitioning environments can also be solved. Moreover, the end-to-end automation of this database configuration process can eliminate the need for manual synchronization of script files and ensure strict consistency of change logic across different environments.

[0096] Through the steps S101 to S104 described above, firstly, a configuration entity can be obtained. This configuration entity includes at least multiple SQL commands and multiple row-level fingerprints, with each row-level fingerprint corresponding to one SQL command. Based on the data version differences between the configuration entity and the database to be configured, a first configuration sequence including multiple configuration tasks can be generated, with each configuration task corresponding to one row-level fingerprint. Then, when executing each configuration task, the row-level fingerprint corresponding to the configuration task can be inserted as a placeholder record into the audit table. Finally, if the audit table does not include the index corresponding to the row-level fingerprint, it can be determined that the placeholder record was successfully inserted, and the SQL command corresponding to the row-level fingerprint can be executed. By refining the execution unit of the configuration database from "script files" to "SQL commands" and generating an independent row-level fingerprint for each SQL command, and performing uniqueness verification of the audit table using row-level fingerprints as placeholder records before execution, idempotent control at the command level can be achieved. This solves the problems of poor breakpoint resumption performance and easy data duplication or structural conflicts caused by using script files as the execution unit in related technologies, significantly improving the configuration efficiency and reliability of the database. This application solves the problem of poor resume performance of database configuration methods that use script files as the execution unit in related technologies, and achieves the technical effect of improving the configuration efficiency and reliability of the database.

[0097] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0098] Corresponding to the database configuration method described in the above embodiments, Figure 7 A schematic diagram of a database configuration apparatus according to an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0099] Please see Figure 7 The database configuration device 7 includes: an acquisition module 70, used to acquire a configuration entity, the configuration entity including at least multiple SQL commands and multiple row-level fingerprints, each row-level fingerprint corresponding to one SQL command; a generation module 71, used to generate a first configuration sequence including multiple configuration tasks based on the data version differences between the configuration entity and the database to be configured, each configuration task corresponding to one row-level fingerprint; an insertion module 72, used to insert the row-level fingerprint corresponding to the configuration task as a placeholder record into an audit table when executing each configuration task; and an execution module 73, used to determine that the placeholder record was successfully inserted and execute the SQL command corresponding to the row-level fingerprint if the audit table does not include the index corresponding to the row-level fingerprint.

[0100] In one embodiment, the database configuration device 7 further includes a configuration module, configured to set the status of the placeholder record to pending execution after inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table; the configuration module is also configured to set the status of the placeholder record to execution completed when the SQL instruction is executed successfully; and to set the status of the placeholder record to execution exception when the SQL instruction fails to execute.

[0101] In one embodiment, the database configuration device 7 further includes a restart module, used to query placeholder records with a status of pending execution or execution error from each index of the audit table when the configuration process of the database to be configured is interrupted and restarted; based on the position of the configuration task corresponding to the placeholder record with a status of pending execution or execution error in the first configuration sequence, the first configuration sequence is reconstructed to obtain a second configuration sequence, the second configuration sequence including the configuration task corresponding to the placeholder record with a status of pending execution or execution error, and the configuration task that has not been executed among multiple configuration tasks; and the database to be configured is continued according to the second configuration sequence.

[0102] In one embodiment, the configuration entity further includes instruction information corresponding to each SQL instruction; the insertion module 72 is further configured to parse the row-level fingerprint corresponding to the configuration task when executing each configuration task to obtain the SQL instruction corresponding to the configuration task; obtain the instruction information of the SQL instruction from the configuration entity, and create a distributed lock key based on the instruction information, the distributed lock corresponding to the distributed lock key is released when the SQL instruction is successfully executed; if the distributed lock key is successfully created, insert the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table, and create an index in the audit table that is uniquely associated with the configuration task based on the row-level fingerprint and instruction information.

[0103] In one embodiment, the acquisition module 70 is further configured to acquire metadata for configuring the database to be configured; call a dialect parser corresponding to the type of the database to be configured to parse the metadata in order to detect whether there are any abnormal instructions in the metadata; if there are no abnormal instructions in the metadata, perform a hash operation on each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction; and encapsulate the metadata, each SQL instruction and each row-level fingerprint into a configuration entity.

[0104] In one embodiment, the acquisition module 70 is further configured to, for each SQL instruction, perform a hash operation on the SQL instruction to obtain the row-level fingerprint corresponding to the SQL instruction if the SQL instruction includes a persistent change instruction; and inject a random offset into the SQL instruction and perform a hash operation on the SQL instruction with the injected random offset to obtain the row-level fingerprint corresponding to the SQL instruction if the SQL instruction includes a session context instruction.

[0105] In one embodiment, the generation module 71 is further configured to obtain the first version corresponding to the database to be configured and the second version corresponding to the configuration entity; extract all version nodes from the configuration entity, as well as the dependencies between version nodes, wherein each version node includes a unique identifier for each version; construct a version dependency directed acyclic graph (DAG) based on all version nodes and the dependencies between them; if there are no cycles in the version dependency DAG and the dependency path from the first version to the second version is reachable, enumerate all configuration paths starting from the first version and ending at the second version; and generate a first configuration sequence based on the shortest configuration path among all configuration paths.

[0106] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0108] Figure 8 This is a schematic diagram of the structure of a terminal device according to an embodiment of this application. Figure 8 As shown, the terminal device 8 includes: at least one processor 80 ( Figure 8 (Only one is shown in the image) a processor, a memory 81, and a computer program 82 stored in the memory 81 and executable on at least one processor 80, wherein the processor 80 executes the computer program 82 to implement the steps in any of the above-described database configuration method embodiments.

[0109] Terminal device 8 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. Terminal device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8This is merely an example of terminal device 8 and does not constitute a limitation on terminal device 8. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0110] The processor 80 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0111] In some embodiments, memory 81 may be an internal storage unit of the terminal device 8, such as a hard disk or memory of the terminal device 8. In other embodiments, memory 81 may be an external storage device of the terminal device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the terminal device 8. In other embodiments, memory 81 may include both internal and external storage units of the terminal device 8. Memory 81 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of computer program 82. Memory 81 may also be used to temporarily store data that has been output or will be output.

[0112] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various database configuration method embodiments above.

[0113] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the various database configuration method embodiments.

[0114] This application implements all or part of the processes in the methods of the above embodiments, which can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a database configuration device or terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0115] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0116] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A database configuration method, characterized in that, The method includes: Obtain a configuration entity, the configuration entity including at least a plurality of SQL instructions and a plurality of row-level fingerprints, each row-level fingerprint corresponding to one of the SQL instructions; Based on the data version difference between the configuration entity and the database to be configured, a first configuration sequence including multiple configuration tasks is generated, and each configuration task corresponds to a row-level fingerprint. When executing each configuration task, the row-level fingerprint corresponding to the configuration task is inserted into the audit table as a placeholder record; If the audit table does not contain the index corresponding to the row-level fingerprint, the placeholder record is determined to have been successfully inserted, and the SQL instruction corresponding to the row-level fingerprint is executed.

2. The method according to claim 1, characterized in that, The step of inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table includes: Insert the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table, and configure the status of the placeholder record to be pending execution; and After confirming that the placeholder record has been successfully inserted and executing the SQL instruction corresponding to the row-level fingerprint, the method further includes: If the SQL command is executed successfully, the status of the placeholder record will be configured as execution completed; In the event that the SQL instruction fails to execute, the status of the placeholder record will be configured as execution exception.

3. The method according to claim 2, characterized in that, After confirming that the placeholder record has been successfully inserted and executing the SQL instruction corresponding to the row-level fingerprint, the method further includes: In the event that the configuration process of the database to be configured is interrupted and restarted, the placeholder records with a status of pending execution or execution error are queried from each of the indexes of the audit table; Based on the position of the configuration task corresponding to the placeholder record whose status is pending execution or execution error in the first configuration sequence, the first configuration sequence is reconstructed to obtain a second configuration sequence. The second configuration sequence includes the configuration task corresponding to the placeholder record whose status is pending execution or execution error, as well as the configuration tasks that have not been executed among the multiple configuration tasks. Continue configuring the database to be configured according to the second configuration sequence.

4. The method according to any one of claims 1 to 3, characterized in that, The configuration entity also includes instruction information corresponding to each SQL instruction; The step of inserting the row-level fingerprint corresponding to the configuration task as a placeholder record into the audit table when executing each configuration task includes: When executing each configuration task, the row-level fingerprint corresponding to the configuration task is parsed to obtain the SQL instruction corresponding to the configuration task; The instruction information of the SQL instruction is obtained from the configuration entity, and a distributed lock key is created based on the instruction information. The distributed lock corresponding to the distributed lock key is released when the SQL instruction is executed successfully. If the distributed lock key is successfully created, the row-level fingerprint corresponding to the configuration task is inserted into the audit table as a placeholder record, and an index uniquely associated with the configuration task is created in the audit table based on the row-level fingerprint and the instruction information.

5. The method according to any one of claims 1 to 3, characterized in that, The entity for obtaining configuration includes: Obtain metadata used to configure the database to be configured; The dialect parser corresponding to the type of the database to be configured is invoked to parse the metadata in order to detect whether there are any abnormal instructions in the metadata; If the abnormal instruction is not found in the metadata, perform a hash operation on each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction; The metadata, each of the SQL commands, and each of the row-level fingerprints are encapsulated into the configuration entity.

6. The method according to claim 5, characterized in that, The step of performing a hash operation on each SQL instruction in the metadata to obtain the row-level fingerprint corresponding to each SQL instruction includes: For each SQL instruction, if the SQL instruction includes a persistent change instruction, a hash operation is performed on the SQL instruction to obtain the row-level fingerprint corresponding to the SQL instruction; If the SQL instruction includes a session context instruction, a random offset is injected into the SQL instruction, and a hash operation is performed on the SQL instruction with the injected random offset to obtain the row-level fingerprint corresponding to the SQL instruction.

7. The method according to any one of claims 1 to 3, characterized in that, The step of generating a first configuration sequence comprising multiple configuration tasks based on the data version difference between the configuration entity and the database to be configured includes: Obtain the first version corresponding to the database to be configured and the second version corresponding to the configuration entity; Extract all version nodes and the dependencies between the version nodes from the configuration entity, where each version node includes a unique identifier for each version; Based on all version nodes and the dependencies between each version node, a version dependency directed acyclic graph (DAG) is constructed. If there are no loops in the version dependency DAG and the dependency path from the first version to the second version is reachable, enumerate all configuration paths starting from the first version and ending with the second version. The first configuration sequence is generated based on the shortest configuration path among all configuration paths.

8. A database configuration device, characterized in that, The device includes: The acquisition module is used to acquire a configuration entity, wherein the configuration entity includes at least a plurality of SQL instructions and a plurality of row-level fingerprints, and each row-level fingerprint corresponds to one of the SQL instructions; The generation module is used to generate a first configuration sequence including multiple configuration tasks based on the data version difference between the configuration entity and the database to be configured, wherein each configuration task corresponds to a row-level fingerprint. An insertion module is used to insert the row-level fingerprint corresponding to each configuration task as a placeholder record into the audit table when each configuration task is executed. The execution module is used to determine that the placeholder record was successfully inserted and to execute the SQL instruction corresponding to the row-level fingerprint if the audit table does not contain the index corresponding to the row-level fingerprint.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the database configuration method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, It includes a computer program, which, when run, causes the database configuration method as described in any one of claims 1 to 7 to be executed.

Images