Construction method and device of operation anomaly prediction model, equipment, medium and product

Abstract

The embodiment of the invention provides a construction method and device of an operation anomaly prediction model, equipment, a medium and a product. According to the method, a mode template is generated according to historical DDL operation data and a DDL operation mode of a source database, and simulated DDL operation data similar to the historical DDL operation data is generated by utilizing the mode template to expand training data, so that the problem of insufficient training data in related technologies is solved; according to the method, historical DDL operation data and simulated DDL operation data are preprocessed to obtain training data, the training data are used for training an initial DDL operation anomaly prediction model, a trained target DDL operation anomaly prediction model is finally output, and the target DDL operation anomaly prediction model can accurately predict real-time DDL operation possibly causing anomaly. And a reliable risk assessment result is provided for heterogeneous database synchronization.

Description

Technical Field

[0001] This application belongs to the field of database technology, and in particular relates to a method, apparatus, device, medium and product for constructing an operational anomaly prediction model. Background Technology

[0002] Heterogeneous database synchronization refers to the migration and synchronization of data between databases of different types or versions. Heterogeneous database synchronization plays a crucial role in the process of database localization. This process is essentially a large-scale heterogeneous database synchronization process. In this environment, the synchronization of Data Definition Language (DDL) operations is particularly important and challenging.

[0003] In heterogeneous database environments, the synchronization of some DDL operations can lead to unpredictable exceptions. These operations may cause data inconsistencies in certain situations, but not every execution will result in problems. The specific impact depends on factors such as the synchronization status at the time of the operation and the content of the operation. These operations include modifying table structures (such as adding, deleting, or modifying columns), creating indexes, and modifying constraints. For example, when modifying a table structure, if the target database has views or stored procedures that depend on that table, these objects may become invalid, leading to exceptions.

[0004] In related technologies, static rule matching, manual intervention, and statistical analysis are commonly used to address anomalies caused by DDL operations. However, static rule matching suffers from limited coverage and lack of flexibility, requiring frequent changes to the rule base to adapt to new database characteristics and changes. Statistical analysis heavily relies on large amounts of high-quality historical data, proving ineffective when data is insufficient or of poor quality. Manual intervention can only address issues retrospectively and cannot prevent anomalies from occurring in advance. None of these three methods can accurately determine whether such DDL operations have caused anomalies. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for constructing an operation anomaly prediction model. Based on historical DDL operation data and DDL operation patterns from a source database, a pattern template is generated. This template is then used to generate simulated DDL operation data similar to the historical data, expanding the training data and addressing the problem of insufficient training data in related technologies. The preprocessed simulated and historical DDL operation data are used to train an initial DDL operation anomaly prediction model, ultimately outputting a trained target DDL operation anomaly prediction model. This target model can accurately predict real-time DDL operations that may cause anomalies, providing reliable risk assessment results for heterogeneous database synchronization.

[0006] In a first aspect, embodiments of this application provide a method for constructing a DDL operation anomaly prediction model, the method comprising: Retrieve historical DDL operation data from the source database. The historical DDL operation data includes DDL operation patterns. Generate a pattern template based on historical DDL operation data and DDL operation patterns; Generate simulated DDL operation data based on the pattern template; The historical DDL operation data and simulated DDL operation data are preprocessed to obtain training data. The preprocessing includes removing duplicate data, handling missing values, data format standardization, and data type conversion. The training data is input into the initial DDL operation anomaly prediction model for training, and the target DDL operation anomaly prediction model is obtained.

[0007] In one feasible implementation, the historical DDL operation data also includes operation type and operation statement; based on the historical DDL operation data and the DDL operation pattern, a pattern template is generated, including: The target operation data is defined as the DDL operation data in the historical DDL operation data whose operation type is the same as the target operation type. The target operation type is any one of the multiple operation types in the historical DDL operation data. Perform structural parsing on the operation statements of the target operation data, and extract the first and second fields of the DDL operation mode; Based on the preset relationship between the first field and the preset placeholder, determine the placeholder corresponding to the first field, and replace the first field with the corresponding placeholder; Based on the syntactic order of the operation statements, the replaced first field and second field are recombined to generate a pattern template corresponding to the target operation type.

[0008] In one feasible implementation, historical DDL operation data also includes database object names, and simulated DDL operation data includes simulated operation statements; based on the schema template, simulated DDL operation data is generated, including: Perform structural parsing on the pattern template and extract the third and fourth fields of the pattern template; Replace the third field with the target database object name, where the target database object name is any one of the database object names; Based on the syntactic order of the pattern template, the replaced third and fourth fields are recombined to generate the simulated operation statement corresponding to the pattern template.

[0009] In one feasible implementation, historical DDL operation data also includes database object names, and simulated DDL operation data includes simulated operation statements and target information; based on the schema template, simulated DDL operation data is generated, including: Perform structural parsing on the pattern template and extract the third and fourth fields of the pattern template; Replace the third field with the target database object name, where the target database object name is any one of the database object names; Based on the syntactic order of the pattern template, the replaced third and fourth fields are recombined to generate the simulated operation statement corresponding to the pattern template; Execute each simulated operation statement in the test database and remove the simulated operation statements that meet the preset conditions; the environment information of the test database is the same as that of the source database. According to preset rules, target information is added to each retained simulated operation statement to obtain simulated DDL operation data.

[0010] In one feasible implementation, the database object name includes the table name, column name, and data type; the third field includes the table name field, column name field, and data type field; and the target database object name includes the target table name, target column name, and target data type. Replace the third field with the name of the target database object, including: Replace the table name field with the target table name; Replace the column name field with the target column name; Replace the data type field with the target data type.

[0011] In one feasible implementation, the database object name also includes object type, object name and operation content, the third field includes object type field, object name field and operation content field, and the target database object name includes target object type, target object name and target operation content; Replace the third field with the name of the target database object, including: Replace the object type field with the target object type; Replace the object name field with the target object name; Replace the operation content field with the target operation content.

[0012] In one feasible implementation, training data is input into an initial DDL operation anomaly prediction model for training, and a target DDL operation anomaly prediction model is output, including: Calculate the information gain value of each feature in the training data, and determine the target feature based on the information gain value; Based on the gain information value of the target feature, determine the feature weight and sample weight; Input the feature weights into the initialized weighted Naive Bayes sub-model to obtain the initial DDL operation anomaly prediction model. Input the training data and sample weights into the initial DDL operation anomaly prediction model to train the model; Cross-validation was performed on the initial DDL operation anomaly prediction model after training to obtain the cross-validation accuracy. If the accuracy is greater than or equal to the set threshold, output the target DDL operation anomaly prediction model.

[0013] Secondly, embodiments of this application also provide an apparatus for constructing a DDL operation anomaly prediction model, the apparatus comprising: The acquisition module is used to acquire historical DDL operation data from the source database. The historical DDL operation data includes DDL operation patterns. The generation module is used to generate pattern templates based on historical DDL operation data and DDL operation patterns. The generation module is also used to generate simulated DDL operation data based on the pattern template; The preprocessing module is used to preprocess historical DDL operation data and simulated DDL operation data to obtain training data. Preprocessing includes removing duplicate data, handling missing values, data format standardization, and data type conversion. The training module is used to input training data into the initial DDL operation anomaly prediction model for training, and obtain the target DDL operation anomaly prediction model.

[0014] Thirdly, embodiments of this application also provide an electronic device, which includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement any of the above-described methods for constructing an anomaly prediction model for DDL operations.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the construction method of any of the above-described DDL operation anomaly prediction models.

[0016] Fifthly, embodiments of this application also provide a computer program product, wherein the instructions in the computer program product, when executed by the processor of an electronic device, enable the electronic device to execute any of the above-described methods for constructing a DDL operation anomaly prediction model.

[0017] The method, apparatus, device, medium, and product for constructing an operation anomaly prediction model provided in this application's embodiments generate a pattern template based on historical DDL operation data and DDL operation patterns from a source database. Using the pattern template, simulated DDL operation data similar to the historical DDL operation data is generated to expand the training data, solving the problem of insufficient training data in related technologies. The historical and simulated DDL operation data are preprocessed to obtain training data, which is then used to train an initial DDL operation anomaly prediction model. Finally, a trained target DDL operation anomaly prediction model is output. This target DDL operation anomaly prediction model can accurately predict real-time DDL operations that may cause anomalies, providing reliable risk assessment results for heterogeneous database synchronization. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a method for constructing a DDL operation anomaly prediction model provided in an embodiment of this application. Figure 2 This is provided by the embodiments of this application. Figure 1 A detailed process diagram of S120 in China; Figure 3 This is provided by the embodiments of this application. Figure 1 A detailed process diagram of S130 in China; Figure 4 This is provided by the embodiments of this application. Figure 1 Another detailed process diagram of S130; Figure 5 This is provided by the embodiments of this application. Figure 1 A detailed process diagram of the S150; Figure 6 This is a schematic diagram of the structure of a device for constructing a DDL operation anomaly prediction model provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0022] Data Definition Language (DDL) is a type of Structured Query Language (SQL) statement used in the database field to define, modify, and delete database structures. DDL operations refer to performing structured operations on objects in the database (such as databases, tables, views, indexes, and constraints) using Data Definition Language, rather than processing specific data records.

[0023] DDL operation statements are specific instructions written in a data definition language, used to define, modify, and delete the structure of a database.

[0024] DDL operation types mainly include CREATE, ALTER, DROP, REMOD, and TRUNCATE.

[0025] DDL operation mode: refers to the structural pattern of DDL operation statements, which usually follows the pattern of "operation type + object type / object name + specific configuration".

[0026] A schema template is generated based on historical DDL operation data and DDL operation patterns. It is obtained by replacing the changed fields (i.e., the first field) in the DDL operation pattern with preset placeholders, and its structure is similar to the DDL operation pattern. This application divides schema templates into two categories based on object types: database table objects and non-database table objects (such as views, triggers, functions, and sequences). The first category of schema templates is used to generate simulated DDL operation statements for objects of the database table type, and the second category is used to generate simulated DDL operation statements for objects of the non-database table type.

[0027] Feature engineering refers to the key steps in machine learning and data modeling involving the processing, transformation, extraction, and selection of raw data to construct more effective features. Its core objective is to transform raw data into a form that better fits the model's learning mechanism, thereby improving the model's prediction accuracy, stability, and generalization ability. Feature engineering can include data cleaning, feature transformation, feature selection, and feature construction.

[0028] Weighted Naïve Bayes Model (WNBM): Based on the classic Naïve Bayes algorithm, it assigns different weights to different features to alleviate the limitations of the feature independence assumption, improve the model's adaptability to real data, and improve the accuracy of prediction to a certain extent.

[0029] The core goal of database localization is to replace the previously relied-upon foreign databases with domestically produced ones. Since this process is typically not a "one-size-fits-all" approach but rather a gradual migration, enterprises will simultaneously have both foreign and domestic databases, even domestic databases from different brands. This coexistence of multiple databases creates a heterogeneous environment. Business continuity requires data consistency across these heterogeneous databases (i.e., heterogeneous database synchronization). Therefore, heterogeneous database synchronization becomes a necessary supporting technology in the database localization migration process, and the localization of databases has spurred an urgent need for heterogeneous database synchronization.

[0030] In related technologies, for DDL operations where it's uncertain whether they will trigger anomalies, static rule matching predefines a set of processing rules for each DDL operation, performs syntactic analysis on each operation, and then matches the analysis results with the predefined rules to identify potential problematic operations. However, the coverage of predefined rules is limited and cannot handle complex or newly emerging DDL anomalies, requiring frequent updates to the rule base to adapt to new database features and changes.

[0031] Statistical analysis based on historical data requires collecting a large amount of historical DDL operation data and its execution results. Statistical methods are then used to analyze the correlation between DDL types and anomaly occurrences, and these statistical results are used to conduct risk assessments for new DDL operations. The accuracy of this method heavily relies on a large amount of high-quality historical data; its accuracy is low when data is insufficient or of poor quality.

[0032] Manual intervention involves deploying monitoring scripts or systems to patch discovered DDL anomalies after the fact. However, it cannot prevent problems from occurring, is inefficient and prone to oversights, and cannot guarantee the stability and consistency of the synchronization process. Therefore, it is not suitable for handling large-scale or high-frequency DDL operations.

[0033] None of the three methods mentioned above can accurately determine whether such DDL operations have caused an anomaly.

[0034] To address the problems in existing technologies, this application provides a method, apparatus, device, medium, and product for constructing an operation anomaly prediction model. Based on historical DDL operation data and DDL operation patterns from a source database, this application generates a pattern template and uses the pattern template to generate simulated DDL operation data similar to the historical DDL operation data, thereby expanding the training data and solving the problem of insufficient training data in related technologies. The historical and simulated DDL operation data are preprocessed to obtain training data, which is then used to train an initial DDL operation anomaly prediction model. Finally, a trained target DDL operation anomaly prediction model is output. This target DDL operation anomaly prediction model can accurately predict real-time DDL operations that may cause anomalies, providing reliable risk assessment results for heterogeneous database synchronization.

[0035] The following section first introduces the method for constructing the operation anomaly prediction model provided in the embodiments of this application.

[0036] Figure 1 This illustration shows a flowchart of a method for constructing a DDL operation anomaly prediction model according to an embodiment of this application. Figure 1 As shown, the method for constructing the DDL operation anomaly prediction model may include the following steps: S110~S150.

[0037] S110. Obtain historical DDL operation data from the source database. The historical DDL operation data includes DDL operation patterns.

[0038] The historical DDL operation data consists of DDL operation records in the source database, which include multiple DDL operation records.

[0039] This step can use monitoring tools or custom scripts to collect historical DDL operation data from the source database. Specific methods include extracting data from database logs, audit logs, and custom triggers.

[0040] In one example, historical DDL operation data may include at least one of the following: the database user executing the DDL operation, the operation type, the timestamp, the operation statement, the database object name, the execution status, the number of rows affected, and error messages. The operation type, timestamp, operation statement, and database object name belong to DDL operation information, used to identify the type and target of the operation, and are fundamental features of the predictive model. The database user executing the DDL operation belongs to the source database's environmental information, describing the circumstances under which the DDL operation occurred. Its role is to provide the context in which the DDL operation occurred, helping the model understand the potential impact of the operation under different environments. The execution status, the number of rows affected, and error messages belong to the synchronization result information, describing the actual impact and result of the DDL operation on the heterogeneous database synchronization process after execution in the source database. For example, the execution status can be used as a training label, and / or the number of rows affected as an auxiliary feature, to help the predictive model learn the association between DDL operations and anomalies.

[0041] DDL operation mode refers to the structural pattern of DDL operation statements, which usually follows the pattern of "operation type + object type / object name + specific configuration".

[0042] For example, taking the operation statement "ALTER TABLE X1 ADD COLUMN Y1 Z1" as an example, "ALTERTABLE (modify table structure)" and "ADD COLUMN (add column)" represent the operation type, "TABLE" indicates that the object type is a table object; "X1" is the specific table name, "Y1" is the specific column name, both of which represent object names; "Z1" represents the specific configuration, such as DATE_TYPE (data type).

[0043] For example, taking the operation statement "ALTER VIEW X2 RENAME VIEW TO Y2" as an example, "ALTERVIEW" and "RENAME VIEW" represent the operation type, "VIEW" indicates that the object type is a view; "X2" is the original view name, representing the object name; "Y2" represents the specific configuration, such as the updated view name.

[0044] The historical DDL operation data obtained in this step refers to DDL operation data where it is uncertain whether an exception will occur after execution. These operations can include, but are not limited to, modifying table structure (ALTER_TABLE), adding columns (ADD_COLUMN), deleting columns (DROP_COLUMN), modifying column attributes (MODIFY_COLUMN), modifying column default values ​​(MODIFY_COLUMN_DEFAULT), renaming columns (RENAME_COLUMN), adding partitions to a table (ADD_PARTITION), deleting table partitions (DROP_PARTITION), creating indexes (CREATE_INDEX), adding constraints (ADD_CONSTRAINT), modifying constraints (MODIFY_CONSTRAINT), and rebuilding indexes (REBUILD_IN). The following functions are available: DEX, RENAME_INDEX, ALTER_VIEW, RENAME_VIEW, ALTER_TRIGGER, ALTER_PROCEDURE, ALTER_FUNCTION, ALTER_SEQUENCE, ALTER_SCHEMA, ALTER_DATABASE, ENABLE_TRIGGER, DISABLE_TRIGGER, CREATE_OR_REPLACE_VIEW, and ALTER_MATERIALIZED_VIEW.

[0045] S120. Generate a pattern template based on historical DDL operation data and DDL operation patterns.

[0046] The pattern template is generated based on the DDL operation patterns from historical DDL operation data and has a similar structure to the DDL operation patterns. The pattern template is used to generate simulated operation statements from the simulated DDL operation data.

[0047] In this step, based on the operation statements of historical DDL operation data, the corresponding DDL operation pattern is determined. This DDL operation pattern is analyzed and broken down into two parts: a relatively fixed command field (referred to as the second field later) and a variable parameter field (referred to as the first field later). Preset placeholders are used to replace the parameter fields. Following the correct syntactic order of the operation statements, the command fields and the replaced parameter fields are recombine to generate a pattern template. Each pattern template corresponds one-to-one with the operation type. This step generates pattern templates corresponding to each operation type.

[0048] In one example, historical DDL operation data also includes operation type and operation statement. Historical DDL operation data consists of DDL operation records from the source database, which include multiple DDL operation records. Each DDL operation record may include an operation type and an operation statement.

[0049] Accordingly, Figure 1 S120 may include the following steps: S121~S124, such as Figure 2 As shown.

[0050] S121. Determine the target operation data as the DDL operation data in the historical DDL operation data whose operation type is the same as the target operation type.

[0051] The target operation type can be any one of the multiple operation types in the historical DDL operation data. The historical DDL operation data includes multiple DDL operation data, each corresponding to one operation type. The operation types of different DDL operation data can be the same or different.

[0052] In this step, select an operation type as the target operation type, and identify the DDL operation data in the historical DDL operation data that has the same operation type as the target operation type as the target operation data.

[0053] S122. Perform structural parsing on the operation statement of the target operation data and extract the first and second fields of the DDL operation mode.

[0054] In this step, the operation statements of the target operation data are structurally parsed. Since the operation types of the target operation data are all the same, their DDL operation modes are basically the same. The variable parameter fields are extracted as the first field, and the fixed command fields are extracted as the second field.

[0055] As discussed above, the first field can include fields corresponding to object names such as table name, column name, and view name, as well as fields corresponding to specific configurations. The first field will differ for different target operation data. The second field can include fields corresponding to the operation type, such as modifying the table structure (ALTER_TABLE) or adding a column (ADD_COLUMN). Since the operation types are the same for different target operation data, the second field is the same for different target operation data.

[0056] S123. Based on the preset relationship between the first field and the preset placeholder, determine the placeholder corresponding to the first field, and replace the first field with the corresponding placeholder.

[0057] In this step, a preset relationship between the first field and the preset placeholder is set in advance. Based on the first field extracted in S122, the placeholder corresponding to the first field can be determined, and the first field can be replaced with the corresponding placeholder.

[0058] For example, when the first field is a specific table name, its corresponding placeholder is determined to be: {table_name}; when the first field is a specific column name, its corresponding placeholder is determined to be: {column_name}; when the first field is a specific data type, its corresponding placeholder is determined to be: {data_type}; when the first field is a specific object type, its corresponding placeholder is determined to be: {object_type}; when the first field is a specific object name, its corresponding placeholder is determined to be: {object_name}; when the first field is a specific operation content, its corresponding placeholder is determined to be: {operation_body}.

[0059] S124. According to the syntactic order of the operation statement, recombine the replaced first field and the second field to generate the pattern template corresponding to the target operation type.

[0060] This step, following the syntactic order of the operation statements, recombines the first and second fields after replacing the placeholders to generate a pattern template corresponding to the target operation type.

[0061] For example, the target operation type is adding a column (ADD_COLUMN), and the corresponding operation statement for the target operation data is "ALTER TABLE orders ADD COLUMN order_date DATE". The structure of this operation statement is parsed, and the first field extracted includes "orders", "order_date", and "DATE". The second field includes "ALTER TABLE" and "ADD COLUMN". "orders" is replaced with the placeholder "{table_name}", "order_date" is replaced with the placeholder "{column_name}", and "DATE" is replaced with the placeholder "{data_type}". Then, according to the syntactic order of the operation statement, the replaced "{table_name}", "{column_name}", "{data_type}", "ALTER TABLE", and "ADDCOLUMN" are recombine to generate the pattern template corresponding to this target operation type (adding a column), specifically "ALTER TABLE{table_name} ADD COLUMN {column_name} {data_type}".

[0062] For example, the target operation type is to delete a column (DROP_COLUMN), and the corresponding operation statement for the target data is "ALTER TABLE customers DROP COLUMN age". The structure of this operation statement is parsed, and the first field extracted includes "customers" and "age", while the second field includes "ALTER TABLE" and "DROP COLUMN". "customers" is replaced with the placeholder "{table_name}", and "age" is replaced with the placeholder "{column_name}". Then, following the syntactic order of the operation statement, the replaced "{table_name}", "{column_name}", "ALTER TABLE", and "DROP COLUMN" are recombine to generate the pattern template corresponding to this target operation type (deleting a column), specifically "ALTER TABLE {table_name} DROP COLUMN {column_name}".

[0063] For example, the target operation type is to rename a view (RENAME_VIEW), and the corresponding operation statement for the target operation data is "ALTER VIEW active_users RENAME TO current_active_users". The structure of this operation statement is parsed, and the first field extracted includes "active_users" and "current_active_users", while the second field includes "ALTER VIEW" and "RENAME TO". "active_users" is replaced with the placeholder "{view_name}", and "current_active_users" is replaced with the placeholder "{new_view_name}". Then, according to the syntactic order of the operation statement, the replaced "{view_name}", "{new_view_name}", "ALTER VIEW", and "RENAME TO" are recombine to generate the pattern template corresponding to this target operation type (renamed view), specifically "ALTER VIEW {view_name} RENAME TO {new_view_name}".

[0064] The method for constructing a DDL operation anomaly prediction model provided in this embodiment generates a standardized and reusable pattern template based on historical DDL operation data and DDL operation patterns. This template is applicable to various DDL operation types and is used to generate operation statements in simulated operation data, providing a flexible and universal operation statement generation mechanism.

[0065] S130. Generate simulated DDL operation data based on the pattern template.

[0066] In this step, the pattern template generated by S120 is used to generate simulated DDL operation data to expand the training data, which solves the problem of insufficient training data in related technologies. At the same time, more training data is beneficial to improving the prediction accuracy of the target DDL operation anomaly prediction model.

[0067] In one example, historical DDL operation data also includes database object names. Database objects refer to various elements and structures in the source database that store or manage data. Database object types can include tables, views, indexes, stored procedures, triggers, functions, constraints, and sequences, etc. The database object name refers to the name of the aforementioned database object type and the name of its contents.

[0068] Simulated DDL operation data includes simulated operation statements. When the operation type is the same, the simulated operation statements have a similar structure to the real operation statements in the historical DDL operation data. Whether there are differences between the two mainly depends on the database object names. If the database object names are also the same, then the simulated operation statements and the real operation statements have the same structure.

[0069] Accordingly, Figure 1 S130 may include the following steps: S131~S133, such as Figure 3 As shown.

[0070] S131. Perform structural parsing on the pattern template and extract the third and fourth fields of the pattern template.

[0071] The method for extracting the third and fourth fields in this step is similar to that mentioned in S122 above. The third field corresponds to the first field after replacement in S123, i.e., the placeholder field, and the fourth field corresponds to the second field.

[0072] S132. Replace the third field with the name of the target database object.

[0073] The target database object name can be any one of the database object names.

[0074] S133. According to the syntactic order of the pattern template, recombine the replaced third and fourth fields to generate the simulation operation statement corresponding to the pattern template.

[0075] For example, an automated script can be written to iterate through each database object name and replace the third field, and then recombine the replaced third field with the fourth field according to the syntax order of the pattern template to generate specific simulated operation statements.

[0076] For example, the operation type is adding a column (ADD_COLUMN), and the corresponding pattern template is "ALTER TABLE {table_name} ADD COLUMN {column_name} {data_type}". The third field extracted from it includes "{table_name}", "{column_name}", and "{data_type}", and the fourth field is "ALTER TABLE" and "ADDCOLUMN". Replace "{table_name}" with a specific table name, such as "orders", replace "{column_name}" with a specific column name, such as "new_col_1", and replace "{data_type}" with a specific data type, such as "INT". The generated simulated operation statement is "ALTER TABLE orders ADD COLUMN new_col_1 INT".

[0077] For example, the operation type is to delete a column (DROP_COLUMN), and the corresponding pattern template is "ALTER TABLE {table_name} DROP COLUMN {column_name}". The extracted third field includes "{table_name}" and "{column_name}", and the fourth field includes "ALTER TABLE" and "DROP COLUMN". Replace "{table_name}" with the specific table name, such as "orders", and replace "{column_name}" with the specific column name, such as "order_date". Then, according to the syntax order of the pattern template, recombine the replaced "orders", "order_date", "ALTER TABLE", and "DROP COLUMN" to generate the simulated operation statement "ALTER TABLE orders DROPCOLUMN order_date".

[0078] For example, the operation type is to modify sequence attributes (ALTER_SEQUENCE), and the corresponding pattern template is "ALTER SEQUENCE {sequence_name} {sequence_options}". The extracted third field includes "{sequence_name}" and "{sequence_options}", and the fourth field includes "ALTER SEQUENCE". Replace "{sequence_name}" with the specific sequence name, such as "seq_user_id", and replace "{sequence_options}" with the specific attribute conditions, such as "INCREMENT BY 2 MAXVALUE 10000". Then, according to the syntax order of the pattern template, recombine the replaced "seq_user_id", "INCREMENT BY 2 MAXVALUE 10000", and "ALTER SEQUENCE" to generate the simulated operation statement "ALTER SEQUENCE seq_user_idINCREMENT BY 2 MAXVALUE 10000".

[0079] For example, the operation type is modifying a stored procedure (ALTER_PROCEDURE), and the corresponding schema template is "ALTER PROCEDURE {procedure_name} AS {operation_body}". The extracted third field includes "{procedure_name}" and "{operation_body}", and the fourth field includes "ALTER PROCEDURE". "{procedure_name}" is replaced with the specific procedure name, such as "proc_update_order", and "{operation_body}" is replaced with the specific operation content, such as "AS BEGIN UPDATE orders SET status = 'processed' WHERE order_id = 1;EN". Then, following the syntax order of the schema template, the replaced "proc_update_order", "AS BEGIN UPDATE orders SET status = 'processed' WHERE order_id = 1;EN", and "ALTER PROCEDURE" are recombine to generate the simulated operation statement "ALTERPROCEDURE proc_update_order AS BEGIN UPDATE orders SET status = 'processed'WHERE order_id = 1;END".

[0080] The method for constructing a DDL operation anomaly prediction model provided in this embodiment uses pattern templates to generate simulated DDL operation statements (i.e., simulated operation data). The simulated operation data has the same structure as the actual DDL operation data and can be used as training data, effectively expanding the scale of training data and solving the problem of insufficient training data in related technologies. At the same time, more training data is beneficial to improving the prediction accuracy of the target DDL operation anomaly prediction model.

[0081] In one example, database object names can include table-type object names and non-table-type object names. Table-type object names refer to the specific object names when the database object type is a table, including the table name, column names, and data types. Non-table-type object names refer to the specific object names when the database object type is a view, index, stored procedure, trigger, function, constraint, or sequence, including the object type, object name, and operation content.

[0082] In one example, the database object name includes the table name, column name, and data type; the third field includes the table name field, column name field, and data type field; and the target database object name includes the target table name, target column name, and target data type. Correspondingly, Figure 3 S132 may include the following steps: Replace the table name field with the target table name; Replace the column name field with the target column name; Replace the data type field with the target data type.

[0083] The method for constructing a DDL operation anomaly prediction model provided in this embodiment is applicable when the database object is a table. The table name, column name, and data type fields in the third field are replaced with the corresponding target table name, target column name, and target data type, respectively. By traversing all table names, column names, and data types in the database, simulated operation statements with table type are generated. The generated simulated operation data has high data diversity, which helps improve the prediction accuracy of the target DDL operation anomaly prediction model.

[0084] In one example, the database object name also includes the object type, object name, and operation content; the third field includes the object type field, object name field, and operation content field; and the target database object name includes the target object type, target object name, and target operation content. Correspondingly, Figure 3 S132 may also include the following steps: Replace the object type field with the target object type; Replace the object name field with the target object name; Replace the operation content field with the target operation content.

[0085] The method for constructing a DDL operation anomaly prediction model provided in this embodiment is applicable to cases where the database object is not a table (e.g., views, indexes, stored procedures, triggers, functions, constraints, and sequences). The object type, object name, and operation content fields in the third field are replaced with the corresponding target object type, target object name, and target operation content, respectively. By traversing all non-table object types, all non-table object names, and all operation contents within the database, simulated operation statements with non-table object types (e.g., views, indexes, stored procedures, triggers, functions, constraints, and sequences) are generated. The generated simulated operation data has high data diversity, which helps improve the prediction accuracy of the target DDL operation anomaly prediction model.

[0086] In one example, historical DDL operation data also includes database object names, while simulated DDL operation data includes simulated operation statements and target information.

[0087] Correspondingly, Figure 1 S130 may include the following steps: S231~S235, such as Figure 4 As shown.

[0088] S231. Perform structural parsing on the pattern template and extract the third and fourth fields of the pattern template.

[0089] S232. Replace the third field with the name of the target database object.

[0090] The target database object name can be any one of the database object names.

[0091] S233. According to the syntactic order of the pattern template, the replaced third and fourth fields are recombined to generate the simulation operation statement corresponding to the pattern template.

[0092] In this embodiment, S231~D233 are the same as S131~S133 mentioned above. For details, please refer to the explanations at S131~S133, which will not be repeated here.

[0093] S234. Execute each simulated operation statement in the test database and remove the simulated operation statements that meet the preset conditions.

[0094] The test database uses the same environment information as the source database. The test database and the source database have the same database structure and objects.

[0095] Remove the corresponding simulation operation statement if at least one of the following conditions is met: (1) The data type is not in the list of data types supported by the test database. The list of data types can be dynamically generated and updated by querying the supported types of the source database system.

[0096] (2) The length of the database object name is greater than the set length. For example, the set length is 30 characters.

[0097] (3) Database object name is a reserved word. Compare the database object name with the reserved words in the reserved word list. If the database object name is a reserved word, delete the corresponding simulated operation statement. For example, reserved words include "DELETE", "UPDATE", "SELECT", and "INSERT", etc. Reserved words are not allowed to be used as database object names. The reserved word list can be dynamically maintained by periodically checking the update documents of the source database.

[0098] (4) For simulated operation statements with the operation type of deleting a column, the column name is the key column. The key column may include at least one of the primary key, foreign key, and unique constraint columns. The key column list can be dynamically updated by periodically retrieving the above information from the database dictionary or the source database using automated scripts.

[0099] (5) For simulated operation statements with the operation type of deleting a column, the column name is the dependency column. For example, the dependency column includes object name key columns, such as "CUSTOMER_ID", "ORDER_ID", or "PRODUCT_ID". Automated tools / scripts can be used to scan database objects and dynamically update the dependency column information.

[0100] This step verifies the rationality and validity of the simulated operation statements, removes simulated operation statements that meet the preset conditions, i.e., simulated operation statements that do not conform to the database environment, and ensures that the retained simulated operation statements conform to the preset conditions and the database environment.

[0101] The implementation method for this step is as follows: Using a programming language (such as Python), the simulated operation statements are input into the test database, and the test database system parses and verifies the simulated operation statements. If the statement is invalid, the test database will return an error message, and the corresponding simulated operation statement will be deleted based on the error message, while successfully executed simulated operation statements will be retained. This method utilizes the parsing capabilities of the test database system itself, resulting in high accuracy; at the same time, the environment information of the test database is the same as that of the source database, ensuring that the structure of the retained simulated operation statements is consistent with that of the actual operation statements.

[0102] S235. According to the preset rules, supplement the target information for each retained simulation operation statement to obtain the simulation DDL operation data.

[0103] The target information may include at least one of the following: database user, operation type, timestamp, database object name, execution status, number of rows affected, and error message.

[0104] For example, the supplementary rules for each target information are as follows: (1) Database user: The database user corresponding to the operation type and object name.

[0105] (2) Operation type: Use regular expressions to identify the operation type in the simulated operation statement.

[0106] (3) Timestamp: The timestamp of the simulated operation statement executed in the test database.

[0107] (4) Database object name: Use regular expressions to identify the database object name in the simulated DDL statement.

[0108] (5) Execution status: The status of the simulated operation statement in the test database.

[0109] (6) Number of rows affected: The number of rows affected as reported when the simulated operation statement is executed in the test database.

[0110] (7) Error messages: Error messages returned when the simulated operation statement is executed on the test database.

[0111] This step supplements each retained simulated operation statement with target information to obtain complete simulated DDL operation data, ensuring that the simulated DDL operation data is consistent with the historical DDL operation data.

[0112] For example, both simulated DDL operation data and historical DDL operation data include database user, operation type, timestamp, operation statement, database object name, execution status, number of affected rows, and error information. It should be noted that this embodiment only exemplifies that simulated DDL operation data and historical DDL operation data include the above-mentioned database information, but does not constitute a limitation on the construction method of the DDL operation anomaly prediction model provided in this application embodiment. In other embodiments, simulated DDL operation data and historical DDL operation data may also include all data information known to those skilled in the art, and are not limited here.

[0113] The method for constructing a DDL operation anomaly prediction model provided in this embodiment uses the parsing capabilities of the test database system itself to verify the rationality and effectiveness of simulated operation statements, removes simulated operation statements that do not conform to the database environment, and supplements the retained simulated operation statements with target information, so that the data composition of simulated DDL operation data is consistent with that of historical DDL operation data, conforms to the database environment, and can be used as training data, effectively expanding the scale of training data and solving the problem of insufficient training data in related technologies; at the same time, more training data is conducive to improving the prediction accuracy of the target DDL operation anomaly prediction model.

[0114] S140. Preprocess the historical DDL operation data and simulated DDL operation data to obtain training data.

[0115] The preprocessing includes removing duplicate data, handling missing values, standardizing data formats, and converting data types. This step performs these processes on the data and then integrates the processed data to obtain the training data.

[0116] Remove duplicate data: Check and remove duplicate data in the dataset (including historical DDL operation data and simulated DDL operation data) to ensure the uniqueness of each DDL operation data.

[0117] Handling missing values: Check for missing values ​​in the dataset and handle them accordingly. For example, for a small number of missing values, data records containing missing values ​​can be deleted; for a large number of missing values, interpolation or imputation methods can be used, such as filling with the previous valid value.

[0118] Data format standardization includes: (1) Standardize the format of dates and timestamps. For example, convert timestamps to the format “YYYY-MM-DD HH:MM:SS”, such as converting the timestamp in the format “2024-01-01 00:00” to “2024-01-01 00:00:00”.

[0119] (2) Perform uniform processing on the numerical data to ensure that the numerical precision and units are consistent. For example, the numerical precision of the affected row number is determined to be the units digit, and "10.0" is converted to 10.

[0120] (3) Heterogeneous Data Integration Processing. When integrating data from different databases, it is necessary to ensure that the format and encoding of each data source are consistent. Specific methods include: ① Standardizing dates and timestamps, for example, formatting all data timestamps as "YYYY-MM-DD HH:MM:SS". ② Converting characters to the same encoding, for example, converting them to UTF-8 encoding. ③ Standardizing numerical data to ensure consistent numerical precision and units. ④ Handling differences in specific fields between different databases, for example, converting the NUMBER type in Oracle database to the DECIMAL type in MySQL.

[0121] Data type conversion: Feature engineering is performed on historical DDL operation data and simulated DDL operation data to ensure that the processed data accurately reflects the characteristics of the original data. Based on data type, the data can be divided into numerical features, categorical features, timestamp features, and text features. Categorical features, timestamp features, and text features are then converted into numerical features and represented in the form of a feature matrix, maintaining data consistency to meet the needs of the model.

[0122] For numerical features, standardization is performed using the Z-score standardization method, transforming them into a standard normal distribution with a mean of 0 and a standard deviation of 1, thus making different numerical features comparable. The specific formula is: in, This represents the Z-score after standardization. Represents the original value. This represents the mean. It represents the standard deviation.

[0123] mean The calculation formula is: Standard deviation The calculation formula is: in, This indicates the number of values ​​contained in the dataset. Indicates the index of the values ​​in the dataset, from 1 to N. Represents the first in the dataset A specific numerical value.

[0124] For example, performing Z-score standardization on the numerical feature "number of affected rows": [0, 10, 0, 0], the result is: [ 0.577, 1.732, 0.577, 0.577].

[0125] Categorical features are encoded to transform categorical data into numerical data. The categorical features to be encoded include database user, operation type, execution status, and database object name. A unique value is extracted from each categorical feature.

[0126] For example, One-Hot encoding converts categorical features into binary form, defining a one-hot vector for each option, where one option is 1 and the rest are 0. When processing categorical features, One-Hot encoding converts categorical data into binary form, ensuring the independence and interpretability of each category's information. For example, the classification features of database users are ['ADMIN', 'DBUSER', 'MIGRATION_TOOL'], which are converted to ['100', '010', '001']; DDL operation types are ['ALTER_TABLE', 'MODIFY_COLUMN', 'CREATE_INDEX'], which are converted to ['100', '010', '001']; execution status is ['SUCCESS', 'FAILURE'], which are converted to ['010', '001']; database object names are ['USERS', 'ORDERS', 'CUSTOMERS', 'PRODUCTS'], which are converted to ['1000', '0100', '0010', '0001'].

[0127] For timestamp features, information such as year, season, and day of the week can be extracted, or time intervals and time differences can be calculated to convert the data into numerical data. For example, the conversion can be performed from a fixed point in time (e.g., 1970-01-01) to the number of seconds until the timestamp. For instance, an original timestamp of 2024-01-01 00:00:00 is converted to 1704067200 seconds; an original timestamp of 2024-01-01 00:10:00 is converted to 1704067800 seconds.

[0128] TF-IDF transformation is performed on the text features "operation statements" and "error messages" to convert the text features into numerical features.

[0129] For example, using the programming language Python, the specific steps are as follows: (1) Extract all DDL operation statements and error messages to form a text set. (2) Use the TF-IDF Vectorizer in the sklearn library to vectorize the text. (3) Use the TF-IDF vectorizer to convert the operation statements and error messages into numerical features. (4) Each operation statement and error message will be converted into a vector, and each element of the vector represents the TF-IDF weight of a word in the text. For example, the text feature "N / A" is converted to [0.10, 0.20, 0.30, 0.40, 0.50] after TF-IDF conversion.

[0130] S150. Input the training data into the initial DDL operation anomaly prediction model for training to obtain the target DDL operation anomaly prediction model.

[0131] In this step, the preprocessed historical DDL operation data and simulated DDL operation data are used as training data and input into the initial DDL operation anomaly prediction model for training. Finally, the trained target DDL operation anomaly prediction model is output.

[0132] The method for constructing an operation anomaly prediction model provided in this application generates a pattern template based on historical DDL operation data and DDL operation patterns from the source database. Using the pattern template, simulated DDL operation data similar to the historical DDL operation data is generated to expand the training data, thus solving the problem of insufficient training data in related technologies. The historical and simulated DDL operation data are preprocessed to obtain training data, which is then used to train an initial DDL operation anomaly prediction model. Finally, a trained target DDL operation anomaly prediction model is output. This target model can accurately predict real-time DDL operations that may cause anomalies, providing reliable risk assessment results for heterogeneous database synchronization.

[0133] In one example Figure 1 S150 may include the following steps: S151~S156, such as Figure 5 As shown.

[0134] S151. Calculate the information gain value of each feature in the training data, and determine the target feature based on the information gain value.

[0135] This step calculates the information gain of each feature to identify the features that have a significant impact on whether the target variable "causes an anomaly" and provides an important basis for subsequent model training.

[0136] For each feature in the training data, an information gain is calculated based on whether it "causes an anomaly," resulting in the information gain value of each feature with respect to the target variable. The formula for calculating information gain is: in, feature Information gain Represent the target variable entropy, Indicates in features Under the conditions The conditional entropy.

[0137] (1) Calculate the target variable according to formula (5). entropy .

[0138] in, This represents the probability that the target variable "whether an anomaly is triggered" takes the value 0. This represents the probability that the target variable "whether it causes an anomaly" takes the value 1. For example, in the training data, if the probability of the target variable "whether it causes an anomaly" taking the value 0 is 0.5, and the probability of it taking the value 1 is also 0.5, then... In other embodiments, and Other values ​​can also be taken, for example, and The values ​​are 0.3 and 0.7 respectively, and are not specified here.

[0139] (2) Calculation characteristics Under the conditions conditional entropy .

[0140] For example, with features Taking "database user_dbuser" as an example, when the value of "database user_dbuser" is 1: If the value of "database user_dbuser" is 1, the probability of the target variable "whether to raise an exception" being 0 and 1 is 0.5 respectively. .

[0141] When the value of "database user_dbuser" is 0: If the value of "database user_dbuser" is 0, the probability that the target variable "whether to raise an exception" is 0 or 1 is 0.5 respectively. .

[0142] (3) Combining features Calculate the target variable for different values. The overall conditional entropy.

[0143] For example, with features Taking "database user_dbuser" as an example, its target variable is... The overall conditional entropy is: Target variable The overall conditional entropy represents the probability of the target variable taking the values ​​1 and 0 for the database user _dbuser. The uncertainty. If the probability of "database user_dbuser" taking the value 1 and 0 is 0.5 respectively, then... .

[0144] (4) Calculate information gain That is, when considering features Then, the target variable How much has the uncertainty been reduced? The greater the information gain, the better the characteristic... For target variable The stronger the explanatory power, the better.

[0145] For example, if ,but This information gain This indicates that after considering the feature "database user_dbuser", the target variable is... The uncertainty has not decreased, therefore this feature is not effective for the target variable. The explanatory power is 0.

[0146] (5) Perform the above (1)~(4) calculations on all features in the training dataset to obtain the information gain value of each feature, sort these information gain values, and select the features whose information gain values ​​are in the top preset proportion (e.g., 30%) as the features that have a greater impact on the target variable "whether it causes an anomaly".

[0147] For example, the target features are as follows: Feature 1: Number of rows affected, information gain: 0.4; Feature 2: Execution status _SUCCESS, information gain value: 0.35; Feature 3: DDL operation type _MODIFY_COLUMN, information gain value: 0.3; Feature 4: DDL operation type _ALTER_TABLE, information gain value 0.25; Feature 5: Database user _admin, information gain value: 0.2; Feature 6: Execution status _FAILURE, information gain value: 0.2; Feature 7: Database user _dbuser, information gain value: 0.15; Feature 8: DDL operation type _CREATE_INDEX, information gain value: 0.1; Feature 9: Database user _migration_tool, information gain value: 0.05.

[0148] For example, the preset ratio is 30%. This ratio is determined based on experience and preliminary experimental results, which can effectively balance the breadth of feature selection and the efficiency of model training. It can effectively retain features that have strong explanatory power for the target variable, while avoiding the increase in model complexity and excessive computational cost caused by too many features.

[0149] S152. Determine the feature weights and sample weights based on the gain information values ​​of the target features.

[0150] In this step, a linear proportional allocation method can be chosen as the weight allocation strategy, that is, weights are allocated based on the information gain value of the target feature. The weight normalization formula is: in: Representing target features Feature weights, Target features Information gain value, Indicates all The sum of the information gain values ​​of each target feature.

[0151] Calculate the initial sample weights based on the feature values ​​and feature weights of each sample. The formula for calculating the initial sample weights is: in, Indicates the initial sample weights. Indicates that the sample has the target feature The value on, Representing target features Feature weights.

[0152] The calculated initial sample weights are normalized so that the sum of all sample weights is 1. The calculation method is as follows: in, This represents the total number of samples. This represents the normalized sample weights. This represents the calculated initial sample weights. It is the sum of the weights of all initial samples.

[0153] For example, the target weights include: Feature 1: Database user _admin, information gain: 0.2; Feature 2: DDL operation type _ALTER_TABLE, information gain: 0.25; Feature 3: Execution status _SUCCESS, information gain: 0.35; Feature 4: Number of rows affected, information gain: 0.4.

[0154] The feature weights of the target features are normalized respectively: For feature 1: =0.2 / (0.2+0.25+0.35+0.4)=0.2 / 1.2≈0.167; For feature 2: =0.25 / 1.2≈0.208; For feature 3: ; For feature 4: =0.4 / 1.2≈0.333.

[0155] For a sample S, its features are: database user _admin=1; DDL operation type _ALTER_TABLE =1; execution status _SUCCESS=0; number of rows affected =10. Calculate the initial sample weights for this sample: =1×0.167+1×0.208+0×0.292+10×0.333=0.167+0.208+0+3.33=3.705.

[0156] If the sum of all sample weights is 10, normalize the initial sample weights to obtain the final sample weights: =3.705 / 10=0.3705.

[0157] S153. Input the feature weights into the initialized weighted Naive Bayes sub-model to obtain the initial DDL operation anomaly prediction model.

[0158] The training model selected is a weighted Naive Bayes model. The weighted Naive Bayes model is initialized by introducing the feature weights obtained in S152 into the model, resulting in an initial DDL operation anomaly prediction model, preparing for model training. Feature weights are introduced into the weighted Naive Bayes model to adjust the influence of each target feature on the final prediction result. The feature weights are used to adjust the importance of each target feature in the model, and the formula is: in, Representing the posterior probability, it refers to the probability under known features. and weight Under the condition, the target variable The probability of "whether an anomaly occurs"; It is the prior probability, without considering other conditional target variables. The probability of "whether an anomaly occurs"; Represents conditional probability, in Under the conditions that it occurs, the first Target features The probability of occurrence; Represents marginal probability, features The overall probability of occurrence, and its normalization effect. Represents the feature set, Represents the first in the feature set The target features, Representing target features The feature weights are used to characterize the target features. The relative importance to the final classification decision.

[0159] For example, the specific method for initializing a weighted Naive Bayes model is as follows: ① Import the GaussianNB class from the Scikit-learn library to build a Naive Bayes classifier.

[0160] ② Define a class named WeightedNaiveBayes, which inherits from GaussianNB, to implement the weighted Naive Bayes classifier.

[0161] ③ Define the class's initialization method, which accepts an optional `feature_weights` parameter and stores it as an instance attribute. Call the initialization method of the parent class `GaussianNB`.

[0162] ④ Define the `fit` method for training the model. If feature weights are provided, the input data X is weighted, and then the parent class's `fit` method is called to perform training.

[0163] ⑤ Initialize the weighted Naive Bayes model and pass the feature weight parameters.

[0164] For example, feature weights and sample weights can be introduced into the model using a programming language (such as Python), as follows: ① Import the NumPy library to support large-dimensional arrays and matrix operations.

[0165] ② Define a NumPy array to store the weight values ​​of each feature. Each element in the array represents a feature weight, which is used to adjust the importance of each feature in the weighted Naive Bayes model. For example, feature_weights = np.array([0.167, 0.208, 0.292, 0.333]).

[0166] ③ Define a NumPy array to store the weight values ​​for each sample. Each element in the array represents a sample weight, which is used to adjust the importance of each sample in the weighted Naive Bayes classifier model. For example, sample_weights = np.array([0.3705, 0.2941, 0.3354]).

[0167] S154. Input the training data and sample weights into the initial DDL operation anomaly prediction model to train the model.

[0168] This step inputs the preprocessed training data, target variable, and sample weights into the model. Internally, the model adjusts the input training data according to the defined feature weights, ensuring that the influence of each feature in the model is positively correlated with its weight. The weighted training data and target variable are then used to fit the model, i.e., the model is trained. During training, sample weights are used to adjust the impact of each sample on the model's learning.

[0169] For example, the specific method for inputting training data and sample weights into the model is as follows: ① Create a two-dimensional array containing multiple data points, each with multiple feature values.

[0170] ② Create a target variable array containing the same number of data entries as the two-dimensional array, where each value represents the category label of the corresponding data. The data comes from the execution status of the collected DDL operation data, indicating whether an exception was triggered.

[0171] ③ Create a sample weight array containing the same number of data points as the two-dimensional array, where each value represents the weight of the corresponding data. For example, sample_weights = np.array([0.3705, 0.294, 0.3354]).

[0172] S155. Perform cross-validation on the initial DDL operation anomaly prediction model after training to obtain the cross-validation accuracy.

[0173] Accuracy can include average accuracy.

[0174] This step uses cross-validation to evaluate the performance of the initial DDL operation anomaly prediction model, ensuring that the model can effectively predict on unseen data, meaning its performance meets expectations. The specific method is as follows: (1) The training data is further divided into K subsets (called “folds”), where K is chosen to be between 5 and 10. For example, based on the data size and number of features in this scheme, 5 folds are chosen as the number of folds for cross-validation. 5-fold cross-validation provides a good balance between training time and model evaluation. Fewer folds may lead to unstable model evaluation results, while more folds will significantly increase computational overhead. 5-fold cross-validation can provide stable and reliable model evaluation without significantly increasing the computational burden.

[0175] (2) Select one of the K subsets in turn as the validation subset, and combine it with the K-1 subsets to form a new training subset; train the model using the new training subset, and then evaluate the performance (i.e., obtain the accuracy) using the validation subset; repeat K times, with each fold being used as a validation subset once, and finally obtain K accuracies. For example, perform 5-fold cross-validation, divide the training data into 5 subsets, select 1 subset as the validation subset in each iteration, and combine the remaining 4 subsets into the training subset, repeat 5 times to generate 5 accuracies.

[0176] (3) Calculate the average accuracy of the K accuracy rates and use the average accuracy as the evaluation result of the model.

[0177] S156. When the accuracy is greater than or equal to the set threshold, output the target DDL operation anomaly prediction model.

[0178] The threshold value is set to a range of [0.8, 0.9]. Using [0.8, 0.9] as the threshold value during preliminary experiments and model evaluation ensures the effectiveness and reliability of the model in practical applications while avoiding excessive computational burden and model complexity due to overly high requirements.

[0179] If the average accuracy is greater than or equal to the set threshold, it indicates that the cross-validation accuracy has met the expected requirements, the model evaluation result is passed, the model training is complete, and the trained target DDL operation anomaly prediction model is output.

[0180] If the average accuracy is less than the set threshold, it indicates that the cross-validation accuracy has not met the expected requirements, and the model evaluation result is unsuccessful. The hyperparameters of the initial DDL operation anomaly prediction model are adjusted to optimize the model performance. Then, the initial DDL operation anomaly prediction model with adjusted hyperparameters is trained using training data, and cross-validation is performed on the trained model to obtain the accuracy of poor validation. Finally, when the average accuracy is greater than or equal to the set threshold, the trained target DDL operation anomaly prediction model is output.

[0181] For example, hyperparameters can include the value of variance smoothing (var_smoothing). The range of variance smoothing values ​​is [1e-9, 1e-8, 1e-7, 1e-6], with an initial default value of 1e-9. Smaller variance smoothing values ​​generally provide higher accuracy, while larger values ​​provide better stability when data noise is high or variance is extremely small; at the same time, it also ensures that a suitable variance smoothing value is found without increasing the computational burden. Taking the implementation using the programming language Python as an example, the specific steps are as follows: (1) Import the following necessary libraries, which may include: GaussianNB, GridSearchCV class, train_test_split function and NumPy library in Scikit-learn.

[0182] (2) Define the range of hyperparameters so that the optimal parameters can be found through grid search.

[0183] (3) The preprocessed training data (feature matrix), target variable, and sample weights are divided into training and validation sets. 20% of the data is used for validation, and 80% of the data is used for training. The random seed is set to 42 to ensure the repeatability of the results.

[0184] (4) Use GridSearchCV to perform grid search and find the hyperparameters that optimize model performance by traversing all possible parameter combinations.

[0185] (5) Output the optimal hyperparameters. Set the optimal variance smoothing parameter during the initialization of the weighted Naive Bayes model, retrain the model, and evaluate the model again using S155 until the model performance (i.e. accuracy) meets the requirements. Output the trained target DDL operation anomaly prediction model.

[0186] The method for constructing an operational anomaly prediction model provided in this application uses an initialized weighted Naive Bayes model as the initial model, which maintains high efficiency and real-time response capability of the system while ensuring high accuracy; by assigning high weights to target features, it solves the problem of balancing accuracy and real-time performance in pipeline technology.

[0187] In one example, the target DDL operation anomaly prediction model is used to predict real-time DDL operation data (unexecuted operation data) and to provide early warnings based on the output.

[0188] In this embodiment, by integrating a trained target DDL operation anomaly prediction model into the database synchronization system, the system predicts real-time DDL operation data (unexecuted operation data), overcoming the limitations of post-event analysis and the inability to prevent problems in a timely manner through manual intervention. This enables real-time risk assessment before DDL operations are executed, thus improving the ability to prevent problems.

[0189] For example, a method for predicting real-time DDL operation data using a target DDL operation anomaly prediction model includes: (1) Perform the same preprocessing on the real-time DDL operation data as on the training data. Preprocessing may include removing duplicate data, handling missing values, data format standardization, and data type conversion. For details, please refer to the explanation in S140, which will not be repeated here.

[0190] (2) Use the trained target DDL operation anomaly prediction model to predict real-time DDL operation data and output the prediction result corresponding to each data point, that is, determine whether the DDL operation will cause anomalies.

[0191] (3) If the prediction result is abnormal, the prediction result will be sent to the early warning system for early warning processing.

[0192] Based on the method for constructing a DDL operation anomaly prediction model, this application also provides a device for constructing a DDL operation anomaly prediction model, which will be described below.

[0193] Figure 6 This is a schematic diagram of the structure of a device for constructing a DDL operation anomaly prediction model provided in an embodiment of this application. Figure 6As shown, the construction device 300 for the DDL operation anomaly prediction model may include: an acquisition module 301, a generation module 302, a preprocessing module 303, and a training module 304.

[0194] The acquisition module 301 is used to acquire historical DDL operation data from the source database. The historical DDL operation data includes DDL operation modes.

[0195] The generation module 302 is used to generate a pattern template based on historical DDL operation data and DDL operation patterns.

[0196] The generation module 302 is also used to generate simulated DDL operation data based on the pattern template.

[0197] The preprocessing module 303 is used to preprocess historical DDL operation data and simulated DDL operation data to obtain training data. The preprocessing includes removing duplicate data, handling missing values, data format standardization, and data type conversion.

[0198] The training module 304 is used to input training data into the initial DDL operation anomaly prediction model for training, and obtain the target DDL operation anomaly prediction model.

[0199] The device for constructing an operation anomaly prediction model provided in this application generates a pattern template based on historical DDL operation data and DDL operation patterns from a source database. Using the pattern template, simulated DDL operation data similar to the historical DDL operation data is generated to expand the training data, thus solving the problem of insufficient training data in related technologies. The historical and simulated DDL operation data are preprocessed to obtain training data, which is then used to train an initial DDL operation anomaly prediction model. Finally, a trained target DDL operation anomaly prediction model is output. This target DDL operation anomaly prediction model can accurately predict real-time DDL operations that may cause anomalies, providing reliable risk assessment results for heterogeneous database synchronization.

[0200] In one example, historical DDL operation data also includes operation types and operation statements. The generation module is used to generate a pattern template based on the historical DDL operation data and the DDL operation pattern. This can include: identifying DDL operation data in the historical DDL operation data whose operation type is the same as the target operation type as the target operation data, wherein the target operation type is any one of multiple operation types in the historical DDL operation data; performing structural parsing on the operation statement of the target operation data to extract the first field and the second field of the DDL operation pattern; determining the placeholder corresponding to the first field based on the preset relationship between the first field and the preset placeholder, and replacing the first field with the corresponding placeholder; and recombine the replaced first field and the second field according to the syntactic order of the operation statement to generate a pattern template corresponding to the target operation type.

[0201] In one example, historical DDL operation data also includes database object names, and simulated DDL operation data includes simulated operation statements. The generation module is used to generate simulated DDL operation data based on the schema template, and may include: performing structural parsing on the schema template to extract the third and fourth fields of the schema template; replacing the third field with the target database object name, where the target database object name is any one of the database object names; and recombine the replaced third and fourth fields according to the syntax order of the schema template to generate the simulated operation statements corresponding to the schema template.

[0202] In one example, historical DDL operation data also includes database object names, and simulated DDL operation data includes simulated operation statements and target information. The generation module is used to generate simulated DDL operation data based on the schema template, which may include: performing structural parsing on the schema template and extracting the third and fourth fields of the schema template; replacing the third field with the target database object name, where the target database object name is any one of the database object names; recombining the replaced third and fourth fields according to the syntax order of the schema template to generate the simulated operation statement corresponding to the schema template; executing each simulated operation statement in the test database and removing simulated operation statements that meet preset conditions; the environment information of the test database is the same as that of the source database; and supplementing the target information for each retained simulated operation statement according to preset rules to obtain the simulated DDL operation data.

[0203] In one example, the database object name includes the table name, column name, and data type; the third field includes the table name field, column name field, and data type field; and the target database object name includes the target table name, target column name, and target data type. The generation module is also used to replace the third field with the target database object name, which may include: replacing the table name field with the target table name; replacing the column name field with the target column name; and replacing the data type field with the target data type.

[0204] In one example, the database object name also includes object type, object name, and operation content; the third field includes object type field, object name field, and operation content field; and the target database object name includes target object type, target object name, and target operation content. The generation module is also used to replace the third field with the target database object name, and may also include: replacing the object type field with the target object type; replacing the object name field with the target object name; and replacing the operation content field with the target operation content.

[0205] In one example, the training module is used to train an initial DDL operation anomaly prediction model by inputting training data into the initial model and outputting a target DDL operation anomaly prediction model. This includes: calculating the information gain value of each feature in the training data and determining the target feature based on the information gain value; determining the feature weight and sample weight based on the information gain value of the target feature; inputting the feature weight into an initialized weighted Naive Bayes sub-model to obtain the initial DDL operation anomaly prediction model; inputting the training data and sample weight into the initial DDL operation anomaly prediction model for model training; performing cross-validation on the trained initial DDL operation anomaly prediction model to obtain the cross-validation accuracy; and outputting the target DDL operation anomaly prediction model if the accuracy is greater than or equal to a set threshold.

[0206] Figure 7 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0207] The electronic device may include a processor 401 and a memory 402 storing computer program instructions.

[0208] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0209] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 402 may include removable or non-removable (or fixed) media, or memory 402 may be a non-volatile solid-state memory. Memory 402 may be internal or external to an electronic device.

[0210] In one example, memory 402 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0211] The processor 401 reads and executes computer program instructions stored in the memory 402 to achieve... Figure 1 The method for constructing the DDL operation anomaly prediction model in the illustrated embodiment.

[0212] In one example, the electronic device may also include a communication interface 403 and a bus 404. For example, Figure 7 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 404 and complete communication with each other.

[0213] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0214] Bus 404 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VESA Local Bus, VLB), or other suitable buses, or a combination of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0215] Furthermore, in conjunction with the method for constructing the DDL operation anomaly prediction model in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the methods for constructing the DDL operation anomaly prediction model in the above embodiments.

[0216] This application also provides a computer program product, including a computer program, which, when executed, implements a method for constructing any of the DDL operation anomaly prediction models described in the above embodiments.

[0217] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0218] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0219] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0220] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0221] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for constructing a DDL operation anomaly prediction model, characterized in that, The method includes: Obtain historical DDL operation data from the source database, including DDL operation patterns. Generate a pattern template based on the historical DDL operation data and the DDL operation pattern; Based on the pattern template, simulated DDL operation data is generated; The historical DDL operation data and the simulated DDL operation data are preprocessed to obtain training data. The preprocessing includes removing duplicate data, handling missing values, data format standardization, and data type conversion. The training data is input into the initial DDL operation anomaly prediction model for training to obtain the target DDL operation anomaly prediction model.

2. The method for constructing a DDL operation anomaly prediction model according to claim 1, characterized in that, The historical DDL operation data also includes operation types and operation statements; the step of generating a pattern template based on the historical DDL operation data and the DDL operation mode includes: The target operation data is determined from the historical DDL operation data whose operation type is the same as the target operation type. The target operation type is any one of the multiple operation types in the historical DDL operation data. The operation statements of the target operation data are structurally parsed to extract the first and second fields of the DDL operation mode. Based on the preset relationship between the first field and the preset placeholder, determine the placeholder corresponding to the first field, and replace the first field with the corresponding placeholder; Based on the syntactic order of the operation statements, the replaced first field and the second field are recombined to generate a pattern template corresponding to the target operation type.

3. The method for constructing a DDL operation anomaly prediction model according to claim 1, characterized in that, The historical DDL operation data also includes database object names, and the simulated DDL operation data includes simulated operation statements; generating simulated DDL operation data according to the pattern template includes: Perform structural parsing on the pattern template to extract the third and fourth fields of the pattern template; Replace the third field with the target database object name, wherein the target database object name is any one of the database object names; According to the grammatical order of the pattern template, the replaced third field and the fourth field are recombined to generate the simulated operation statement corresponding to the pattern template.

4. The method for constructing a DDL operation anomaly prediction model according to claim 1, characterized in that, The historical DDL operation data also includes database object names, and the simulated DDL operation data includes simulated operation statements and target information; generating simulated DDL operation data according to the pattern template includes: Perform structural parsing on the pattern template to extract the third and fourth fields of the pattern template; Replace the third field with the target database object name, wherein the target database object name is any one of the database object names; According to the grammatical order of the pattern template, the replaced third field and the fourth field are recombined to generate the simulated operation statement corresponding to the pattern template; Each of the simulated operation statements is executed in the test database, and the simulated operation statements that meet the preset conditions are removed; the environment information of the test database is the same as the environment information of the source database. According to preset rules, the target information is supplemented for each retained simulated operation statement to obtain the simulated DDL operation data.

5. The method for constructing a DDL operation anomaly prediction model according to claim 3 or 4, characterized in that, The database object name includes table name, column name, and data type; the third field includes table name field, column name field, and data type field; and the target database object name includes target table name, target column name, and target data type. The step of replacing the third field with the target database object name includes: Replace the table name field with the target table name; Replace the column name field with the target column name; Replace the data type field with the target data type.

6. The method for constructing a DDL operation anomaly prediction model according to claim 3 or 4, characterized in that, The database object name also includes object type, object name and operation content; the third field includes object type field, object name field and operation content field; the target database object name includes target object type, target object name and target operation content. The step of replacing the third field with the target database object name includes: Replace the object type field with the target object type; Replace the object name field with the target object name; Replace the operation content field with the target operation content.

7. The method for constructing a DDL operation anomaly prediction model according to any one of claims 1-4, characterized in that, The step of inputting the training data into the initial DDL operation anomaly prediction model for training and outputting the target DDL operation anomaly prediction model includes: Calculate the information gain value of each feature in the training data, and determine the target feature based on the information gain value; Based on the gain information value of the target feature, determine the feature weight and sample weight; The feature weights are input into the initialized weighted Naive Bayes sub-model to obtain the initial DDL operation anomaly prediction model. The training data and the sample weights are input into the initial DDL operation anomaly prediction model to train the model. Cross-validation was performed on the trained initial DDL operation anomaly prediction model to obtain the cross-validation accuracy. If the accuracy is greater than or equal to a set threshold, the target DDL operation anomaly prediction model is output.

8. A device for constructing a DDL operation anomaly prediction model, characterized in that, The device includes: The acquisition module is used to acquire historical DDL operation data from the source database, including DDL operation modes. The generation module is used to generate a pattern template based on the historical DDL operation data and the DDL operation pattern. The generation module is also used to generate simulated DDL operation data based on the pattern template; The preprocessing module is used to preprocess the historical DDL operation data and the simulated DDL operation data to obtain training data. The preprocessing includes removing duplicate data, handling missing values, data format standardization, and data type conversion. The training module is used to input the training data into the initial DDL operation anomaly prediction model for training, so as to obtain the target DDL operation anomaly prediction model.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the method for constructing a DDL operation anomaly prediction model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for constructing a DDL operation anomaly prediction model as described in any one of claims 1-7.

11. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is able to perform the method for constructing the DDL operation anomaly prediction model as described in any one of claims 1-7.

Images