Grayscale release verification method, system, medium, equipment and program

Abstract

The present application provides a grayscale release verification method, system, medium, device and program. In this method, when a subsystem is updated, a first grayscale release step is executed, and a first execution result after executing the first grayscale release step is determined, and then the first execution result is verified. Only when the first execution result passes the verification process, the next grayscale release step in the grayscale release step sequence is executed. When all grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process, it is determined that the grayscale release verification corresponding to the subsystem version update is successful, so that in the process of grayscale release of the version, automatic verification can be carried out in time after each grayscale step is completed, so as to find problems in time.

Description

Technical Field

[0001] The present application relates to the field of financial technology (Fintech), and in particular to a grayscale release verification method, system, medium, device and program. Background Art

[0002] With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming to Fintech (Finteh), and interface testing technology is no exception. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] At present, with the development of Internet technology, the application of microservice technology, and the increase in the frequency of version updates, the manpower investment in version release within enterprises is increasing, which poses a greater challenge to enterprise operation and maintenance personnel. Currently, after the enterprise operation and maintenance personnel release the version, most of them rely on manual version verification, which is inefficient and time-consuming. In addition, the verification plan is mainly compiled by the release personnel based on experience, and there is a problem of imperfect plan.

[0004] Especially during the grayscale release process, it is difficult to verify in time after each grayscale step is completed. If there is a problem with the version, it is difficult to discover the problem in time. Summary of the invention

[0005] The embodiments of the present application provide a grayscale release verification method, system, medium, device and program to solve the technical problems in the prior art that it is difficult to verify in a timely manner after each grayscale step is completed during the grayscale release of the version, and it is difficult to discover problems with the version in a timely manner.

[0006] In a first aspect, an embodiment of the present application provides a grayscale release verification method, which is applied to a grayscale release verification system, wherein the grayscale release verification system includes multiple subsystems, and the version update of each subsystem includes a grayscale release step sequence, wherein the grayscale release step sequence includes multiple grayscale release steps arranged in a preset order, and the method includes:

[0007] When performing a version update on a subsystem, executing a first grayscale release step, and determining a first execution result after executing the first grayscale release step;

[0008] Performing verification processing on the first execution result, wherein the verification processing includes at least one of application verification, database verification, business function verification, and monitoring performance indicator checking, the application verification is used to verify the application configuration result in the first execution result, the database verification is used to verify the database operation statement in the first execution result, the business function verification is used to verify the structured query language SQL statement execution result in the first execution result, and the monitoring performance indicator checking is used to check the instance level indicator data in the first execution result;

[0009] If the first execution result passes the verification process, a second grayscale release step is executed, where the second grayscale release step is the next grayscale release step of the first grayscale release step;

[0010] When all the grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process, it is determined that the grayscale release verification corresponding to the version update of the subsystem is successful.

[0011] In a possible design, when the verification process includes the database verification, the verification process on the first execution result includes:

[0012] Read the set of SQL statements to be verified in the database DB material package, and use a preset lexical analyzer to parse each SQL statement in the set of SQL statements to be verified to generate a corresponding syntax tree;

[0013] Determine the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition;

[0014] Acquire a first table name involved according to a statement type corresponding to a first SQL statement, and connect to a corresponding first database according to the first table name, wherein the first SQL statement is any one statement in the set of SQL statements to be verified;

[0015] Acquire a first operation statement corresponding to the first database, and parse the first operation statement according to the preset lexical analyzer to generate a first syntax tree;

[0016] The syntax tree to be verified and the first syntax tree are verified according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database, and the syntax tree to be verified is a syntax tree generated by parsing the first SQL statement using the preset lexical analyzer.

[0017] In a possible design, after determining the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition, the method further includes:

[0018] Determine that the statement whose statement type is data definition language DDL is the first SQL statement;

[0019] A first operation type corresponding to the first SQL statement is determined according to a preset classification rule, where the first operation type is used to determine the first operation statement corresponding to the first SQL statement in the first database.

[0020] In a possible design, before acquiring the name of the first table involved according to the statement type corresponding to the first SQL statement, the method further includes:

[0021] Sorting the SQL statements in the set of SQL statements to be verified according to the reading order of each SQL statement in the set of SQL statements to be verified and the corresponding table name classification to generate a first table structure object;

[0022] Starting from the last SQL statement of the first table structure object, the database verification is performed in reverse order; if the verification result is successful, the SQL statement currently being verified is deleted from the first table structure object, and the database verification is rolled back to the previous SQL statement to continue, and after the database verification is completed in the first table structure object, an inspection report is output; if the verification result is a failure, an inspection report is output.

[0023] In a possible design, when the verification process includes the monitoring performance indicator check, the verification process of the first execution result includes:

[0024] Acquire the instance-level multi-operation and maintenance indicator data in the first execution result, and determine the indicator data type of the instance-level multi-operation and maintenance indicator data according to data characteristics;

[0025] A corresponding anomaly detection algorithm is determined according to the indicator data type, and verification processing is performed on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm.

[0026] In a possible design, if the determined indicator data type is the stable data, determining the corresponding anomaly detection algorithm according to the indicator data type, and verifying the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm includes:

[0027] Synchronously using multiple sliding windows to traverse the instance-level multi-operation and maintenance indicator data, and determining feature statistics of the multiple sliding windows, wherein the feature statistics are used to characterize the central tendency characteristics of the instance-level multi-operation and maintenance indicator data;

[0028] Subtracting the instance-level multi-operation and maintenance indicator data from the feature statistics of the multiple sliding windows respectively, and generating a feature sequence;

[0029] The feature sequence is tested using a box plot outlier detection algorithm, and when it is determined that the statistical data deviation between the instance-level multi-operation and maintenance indicator data and the multiple sliding windows is greater than a preset threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0030] In a possible design, the anomalies of the instance-level multi-operation and maintenance indicator data include: burr anomalies, overall rise anomalies, and overall fall anomalies;

[0031] Among them, if the mean of the front window and the rear window of the abnormal point meets the preset normal condition and the difference of the mean of the front window and the rear window is within the preset range, and the increase of the abnormal point indicator data is greater than the preset increase and greater than the mean of the front and rear windows, then it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr anomaly, and the multiple sliding windows include the front window and the rear window;

[0032] If the mean of the window before the abnormal point meets the preset normal condition and is less than the window after the mean, and the increase of the indicator data of the abnormal point is greater than the preset increase and greater than the mean of the window before, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall lifting abnormality;

[0033] If the mean of the window before the abnormal point meets the preset normal conditions and is greater than the mean of the rear window, and the increase in the indicator data of the abnormal point is less than the preset increase and greater than the mean of the front window, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall decline anomaly.

[0034] In a possible design, the testing of the feature sequence using a box plot outlier detection algorithm includes:

[0035] The feature sequence is reconstructed by subtracting the data at the judgment moment from the statistics of the front window and the rear window to determine the normal range of the feature sequence. If the difference between the current data and the statistics of the front window and the rear window is not within the normal range, the feature sequence is determined to be abnormal.

[0036] In a possible design, if it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr abnormality, the length of the left window is set to be greater than the length of the right window to increase the amount of data collected before the abnormal point occurs, and the multiple sliding windows include the left window and the right window;

[0037] If the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall rising abnormality or the overall falling abnormality, the length of the left window and the right window are set to meet the length requirement to capture stable data within the preset time length.

[0038] In a possible design, if the determined indicator data type is the trend data, determining the corresponding anomaly detection algorithm according to the indicator data type, and verifying the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm includes:

[0039] Select K initialized indicator data as initial cluster centers, where K is a positive integer;

[0040] Allocating each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the initial cluster center with the smallest distance according to the distance from the initial cluster center, wherein the cumulative distance is calculated as the measure of the distance;

[0041] Re-determine the cluster center for the class corresponding to the initial cluster center, so as to cluster the new cluster center as a subsequent step;

[0042] Repeatedly assigning each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the cluster center with the smallest distance according to the distance from the current cluster center, and updating the cluster center of the corresponding class until the number of iterations is completed or the iteration result meets the preset conditions, so as to determine the cluster center sequence;

[0043] Calculating the cumulative distance between the current cluster center of the cluster center sequence and the farthest data in the corresponding class as the abnormal indicator threshold of the instance-level multi-operation and maintenance indicator data;

[0044] If the distance between the instance-level multi-operation and maintenance indicator data and the corresponding cluster center is greater than the abnormal indicator threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0045] In a possible design, if the determined indicator data type is the periodic data, determining the corresponding anomaly detection algorithm according to the indicator data type, and verifying the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm includes:

[0046] Generate a prediction model for periodic data memory prediction based on LSTM algorithm and historical multi-operation and maintenance indicator data learning;

[0047] Determine the forecast value at each moment according to the forecast model, and determine the growth range for the same period according to the forecast value and the average growth rate for the same period;

[0048] If the instance-level multi-operation and maintenance indicator data exceeds the growth range over the same period, it is determined that the instance-level multi-operation and maintenance indicator data is abnormal.

[0049] In a second aspect, the embodiment of the present application further provides a grayscale release verification system, including:

[0050] A publishing module, used to execute a first grayscale publishing step when a subsystem is updated, and determine a first execution result after executing the first grayscale publishing step;

[0051] a processing module, configured to perform verification processing on the first execution result, wherein the verification processing includes at least one of application verification, database verification, business function verification, and monitoring performance indicator checking, wherein the application verification is used to verify the application configuration result in the first execution result, the database verification is used to verify the database operation statement in the first execution result, the business function verification is used to verify the structured query language SQL statement execution result in the first execution result, and the monitoring performance indicator checking is used to check the instance-level indicator data in the first execution result;

[0052] The processing module is further configured to execute a second grayscale publishing step when the first execution result passes the verification process, wherein the second grayscale publishing step is a next grayscale publishing step of the first grayscale publishing step;

[0053] The processing module is further configured to determine that the grayscale release verification corresponding to the version update of the subsystem is successful when all grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process.

[0054] In a possible design, the processing module is specifically used to:

[0055] Read the set of SQL statements to be verified in the database DB material package, and use a preset lexical analyzer to parse each SQL statement in the set of SQL statements to be verified to generate a corresponding syntax tree;

[0056] Determine the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition;

[0057] Acquire a first table name involved according to a statement type corresponding to a first SQL statement, and connect to a corresponding first database according to the first table name, wherein the first SQL statement is any one statement in the set of SQL statements to be verified;

[0058] Acquire a first operation statement corresponding to the first database, and parse the first operation statement according to the preset lexical analyzer to generate a first syntax tree;

[0059] The syntax tree to be verified and the first syntax tree are verified according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database, and the syntax tree to be verified is a syntax tree generated by parsing the first SQL statement using the preset lexical analyzer.

[0060] In a possible design, the processing module is specifically used to:

[0061] Determine that the statement whose statement type is data definition language DDL is the first SQL statement;

[0062] A first operation type corresponding to the first SQL statement is determined according to a preset classification rule, where the first operation type is used to determine the first operation statement corresponding to the first SQL statement in the first database.

[0063] In a possible design, before acquiring the name of the first table involved according to the statement type corresponding to the first SQL statement, the method further includes:

[0064] Sorting the SQL statements in the set of SQL statements to be verified according to the reading order of each SQL statement in the set of SQL statements to be verified and the corresponding table name classification to generate a first table structure object;

[0065] Starting from the last SQL statement of the first table structure object, the database verification is performed in reverse order; if the verification result is successful, the SQL statement currently being verified is deleted from the first table structure object, and the database verification is rolled back to the previous SQL statement to continue, and after the database verification is completed in the first table structure object, an inspection report is output; if the verification result is a failure, an inspection report is output.

[0066] In a possible design, the processing module is specifically used to:

[0067] Acquire the instance-level multi-operation and maintenance indicator data in the first execution result, and determine the indicator data type of the instance-level multi-operation and maintenance indicator data according to data characteristics;

[0068] A corresponding anomaly detection algorithm is determined according to the indicator data type, and verification processing is performed on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm.

[0069] In a possible design, the processing module is specifically used to:

[0070] Synchronously using multiple sliding windows to traverse the instance-level multi-operation and maintenance indicator data, and determining feature statistics of the multiple sliding windows, wherein the feature statistics are used to characterize the central tendency characteristics of the instance-level multi-operation and maintenance indicator data;

[0071] Subtracting the instance-level multi-operation and maintenance indicator data from the feature statistics of the multiple sliding windows respectively, and generating a feature sequence;

[0072] The feature sequence is tested using a box plot outlier detection algorithm, and when it is determined that the statistical data deviation between the instance-level multi-operation and maintenance indicator data and the multiple sliding windows is greater than a preset threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0073] In a possible design, the anomalies of the instance-level multi-operation and maintenance indicator data include: burr anomalies, overall rise anomalies, and overall fall anomalies;

[0074] Among them, if the mean of the front window and the rear window of the abnormal point meets the preset normal condition and the difference of the mean of the front window and the rear window is within the preset range, and the increase of the abnormal point indicator data is greater than the preset increase and greater than the mean of the front and rear windows, then it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr anomaly, and the multiple sliding windows include the front window and the rear window;

[0075] If the mean of the window before the abnormal point meets the preset normal condition and is less than the window after the mean, and the increase of the indicator data of the abnormal point is greater than the preset increase and greater than the mean of the window before, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall lifting abnormality;

[0076] If the mean of the window before the abnormal point meets the preset normal conditions and is greater than the mean of the rear window, and the increase in the indicator data of the abnormal point is less than the preset increase and greater than the mean of the front window, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall decline anomaly.

[0077] In a possible design, the processing module is specifically used to:

[0078] The feature sequence is reconstructed by subtracting the data at the judgment moment from the statistics of the front window and the rear window to determine the normal range of the feature sequence. If the difference between the current data and the statistics of the front window and the rear window is not within the normal range, the feature sequence is determined to be abnormal.

[0079] In a possible design, if it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr abnormality, the length of the left window is set to be greater than the length of the right window to increase the amount of data collected before the abnormal point occurs, and the multiple sliding windows include the left window and the right window;

[0080] If the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall rising abnormality or the overall falling abnormality, the length of the left window and the right window are set to meet the length requirement to capture stable data within the preset time length.

[0081] In a possible design, the processing module is specifically used to:

[0082] Select K initialized indicator data as initial cluster centers, where K is a positive integer;

[0083] Allocating each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the initial cluster center with the smallest distance according to the distance from the initial cluster center, wherein the cumulative distance is calculated as the measure of the distance;

[0084] Re-determine the cluster center for the class corresponding to the initial cluster center, so as to cluster the new cluster center as a subsequent step;

[0085] Repeatedly assigning each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the cluster center with the smallest distance according to the distance from the current cluster center, and updating the cluster center of the corresponding class until the number of iterations is completed or the iteration result meets the preset conditions, so as to determine the cluster center sequence;

[0086] Calculating the cumulative distance between the current cluster center of the cluster center sequence and the farthest data in the corresponding class as the abnormal indicator threshold of the instance-level multi-operation and maintenance indicator data;

[0087] If the distance between the instance-level multi-operation and maintenance indicator data and the corresponding cluster center is greater than the abnormal indicator threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0088] In a possible design, the processing module is specifically used to:

[0089] Generate a prediction model for periodic data memory prediction based on LSTM algorithm and historical multi-operation and maintenance indicator data learning;

[0090] Determine the forecast value at each moment according to the forecast model, and determine the growth range for the same period according to the forecast value and the average growth rate for the same period;

[0091] If the instance-level multi-operation and maintenance indicator data exceeds the growth range over the same period, it is determined that the instance-level multi-operation and maintenance indicator data is abnormal.

[0092] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0093] processor; and,

[0094] A memory, configured to store executable instructions of the processor;

[0095] Wherein, the processor is configured to execute any one of the grayscale release verification methods in the first aspect by executing the executable instructions.

[0096] In a fourth aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, any one of the grayscale release verification methods in the first aspect is implemented.

[0097] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements any one of the grayscale release verification methods in the first aspect.

[0098] The embodiments of the present application provide a grayscale release verification method, system, medium, device and program. When a subsystem is updated, a first grayscale release step is executed, and a first execution result after executing the first grayscale release step is determined. Then, the first execution result is verified. Only when the first execution result passes the verification process, the next grayscale release step in the grayscale release step sequence is executed. When all grayscale release steps in the grayscale release step sequence are completed and the execution results pass the verification process, it is determined that the grayscale release verification corresponding to the version update of the subsystem is successful. Therefore, in the process of grayscale release of the version, automatic verification can be performed in time after each grayscale step is completed to detect problems in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0100] Figure 1 It is a flowchart of the grayscale release verification method shown in Example 1 of the present application;

[0101] Figure 2 It is a flowchart of another grayscale release verification method shown in Example 1 of the present application;

[0102] Figure 3 It is a flowchart of the database verification processing steps shown in the second embodiment of the present application;

[0103] Figure 4 It is a flowchart of another database verification processing step shown in the second embodiment of the present application;

[0104] Figure 5 It is a flowchart of the monitoring performance indicator checking steps shown in the third embodiment of the present application;

[0105] Figure 6 is a flowchart of another monitoring performance indicator checking step shown in Embodiment 3 of the present application;

[0106] Figure 7 This is a schematic diagram of abnormal data in the first algorithm shown in the fourth embodiment of the present application;

[0107] Figure 8 is a schematic diagram of a flow chart based on the third algorithm shown in the fourth embodiment of the present application;

[0108] Fig. 9 This is a schematic diagram of the host indicator time series shown in Example 4 of the present application;

[0109] Fig.10 is a flowchart based on the fourth algorithm shown in Example 5 of the present application;

[0110] Fig.11 It is a flowchart of the LSTM-based prediction algorithm shown in Example 6 of the present application;

[0111] Fig.12 Schematic diagram of the LSTM network structure shown in Example 6 of the present application;

[0112] Fig.13 It is a flowchart of the grayscale release verification system shown in Example 7 of the present application;

[0113] Fig.14 It is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0114] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0115] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0116] With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming into Fintech, and interface testing technology is no exception. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology. At present, with the development of Internet technology, the application of microservice technology, and the increase in the frequency of version updates, the manpower investment in version release within enterprises is increasing, which poses greater challenges to enterprise operation and maintenance personnel. At present, after the enterprise operation and maintenance personnel release the version, most of them rely on manual version verification, which is inefficient and time-consuming. In addition, the verification scheme is mainly obtained by the release personnel based on experience, and there is a problem of imperfect scheme. Especially in the process of grayscale release of the version, it is difficult to verify in time after each grayscale step is completed. If there is a problem with the version, it is difficult to find the problem in time.

[0117] Specifically, since the system operation and health status are often determined by a series of indicators, the indicators are not independent of each other. As the scale of the subsystem increases and the complexity of system calls increases, the operation and maintenance personnel need to verify each operation and maintenance indicator one by one after the version is released. The verification efficiency is low and time-consuming.

[0118] In addition, the operation and maintenance personnel configure expert rules for version verification based on their long-term operation and maintenance experience, and need to build different rules for different operation and maintenance indicators and business scenarios, which has high development and maintenance costs. In addition, in version verification, there are problems such as incomplete verification points and inaccurate verification methods, and it is impossible to form a unified and complete verification solution.

[0119] More importantly, manual verification makes it difficult to ensure that each grayscale step in the version release process is verified in a timely manner after the release is completed, and it is impossible to discover version quality problems in a timely manner from massive monitoring data.

[0120] In response to the above-mentioned technical problems, the embodiments of the present application aim to provide a grayscale release verification method to automate the grayscale release verification, so as to improve the accuracy and timeliness of the grayscale release verification. In addition, based on the structured query language (SQL) inverse parsing technology, the database model definition language (DDL) verification SQL and the data manipulation language (DML) verification SQL are automatically generated in reverse, thereby reducing the manpower input of manually generating verification schemes and improving the accuracy of the schemes. In addition, based on a multi-variable operation and maintenance indicator anomaly detection method with a multi-algorithm hybrid, abnormal fluctuation detection and outlier detection are performed on the key monitoring indicators at the instance level after release.

[0121] Figure 1 1 is a flow chart of the grayscale release verification method shown in Example 1 of the present application. Figure 1-Figure 2 As shown, the grayscale release verification method provided in this embodiment includes:

[0122] Step 101: When performing a version update on a subsystem, execute a first grayscale release step, and determine a first execution result after executing the first grayscale release step.

[0123] This embodiment can be applied to a grayscale release verification system, which includes multiple subsystems. The version update of each subsystem includes a grayscale release step sequence, and the grayscale release step sequence includes multiple grayscale release steps arranged in a preset order. Specifically, when a subsystem is updated through grayscale release, the version update of all instances of a subsystem can be divided into multiple grayscale release steps for release, and each grayscale release step contains one or more instances. After each release step is executed, the version verification module is automatically called up for verification. If the verification passes, the next grayscale release step is executed. If the verification fails, the execution of subsequent grayscale release steps is stopped.

[0124] Step 102: Verify the first execution result.

[0125] Specifically, the first execution result is verified, wherein the verification process includes at least one of application verification, database verification, business function verification, and monitoring performance indicator check, wherein the application verification is used to verify the application configuration result in the first execution result, the database verification is used to verify the database operation statement in the first execution result, the business function verification is used to verify the structured query language SQL statement execution result in the first execution result, and the monitoring performance indicator check is used to check the instance level indicator data in the first execution result.

[0126] Continue to refer to Figure 2 For application (APP) verification, the following items can be checked:

[0127] a. Check whether the app process status is started;

[0128] b. Check whether the process startup time is started after the version update;

[0129] c. Check whether the MD5 value of the process material package is consistent with the MD5 value of the process material package in the user acceptance test (UAT) environment;

[0130] d. Check whether the process version number is equal to the version number that needs to be released this time;

[0131] e. Check whether the last update time of the log file is greater than the version update time;

[0132] f. Check if there are any variables in the configuration file that are not replaced.

[0133] For database (DB) verification, the following items can be checked:

[0134] a. Check whether the DDL table change statement in the DB version has been executed;

[0135] b. Check whether the DML statements in the DB version have been executed.

[0136] For business function verification, the following items can be checked:

[0137] a. Execute the verification SQL statement compiled before the version is released to check whether the execution result is equal to the expected result value.

[0138] For monitoring performance indicator inspection, the following items can be checked:

[0139] a. Based on the first algorithm, detect whether there are regular spike anomalies, overall rise or fall anomalies in stable indicators such as CPU, IO and business success rate of the grayed instance, where the first algorithm can be a python toolkit for unsupervised anomaly detection;

[0140] b. Based on the second algorithm, cluster analysis is performed on the time consumption indicators of the business interfaces of the grayscaled instances and the unreleased instances, so as to detect whether there are outliers in the time consumption of key business indicators and the unreleased instances.

[0141] c. Based on the Long Short-Term Memory (LSTM) prediction algorithm, the instance-level business interface transaction volume is predicted and the historical year-on-year increment rate is calculated to generate a normal transaction volume range. This is compared with the business transaction volume after the version to detect whether the key business transaction volume has dropped or increased abnormally.

[0142] Step 103: If the first execution result passes the verification process, the second grayscale release step is executed.

[0143] If the first execution result passes the verification process, the second grayscale release step is executed, wherein the second grayscale release step is the next grayscale release step of the first grayscale release step. That is, the multiple grayscale release steps arranged in a preset order included in the grayscale release step sequence are executed and verified in turn, and the next grayscale release step is started only when the execution result of the previous grayscale release step passes the verification process.

[0144] Step 104: When all the grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process, it is determined that the grayscale release verification corresponding to the subsystem version update is successful.

[0145] When all the grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process, it is determined that the grayscale release verification corresponding to the subsystem version update is successful. If the execution result of the grayscale release step fails to pass the verification process before all the grayscale release steps in the grayscale release step sequence are executed, it is determined that the grayscale release verification corresponding to the subsystem version update has failed.

[0146] In this embodiment, when a subsystem is versioned, a first grayscale release step is executed, and a first execution result after executing the first grayscale release step is determined. Then, the first execution result is verified. Only when the first execution result passes the verification process, the next grayscale release step in the grayscale release step sequence is executed. When all grayscale release steps in the grayscale release step sequence are completed and the execution result passes the verification process, it is determined that the grayscale release verification corresponding to the version update of the subsystem is successful. Therefore, in the process of grayscale release of the version, automatic verification can be performed in time after each grayscale step is completed, so as to detect problems in time.

[0147] Figure 3 is a flowchart of the database verification processing steps shown in the second embodiment of the present application, Figure 4 FIG. 2 is a flowchart of another database verification process step shown in the second embodiment of the present application. Figure 3-Figure 4 As shown, the database verification processing steps provided in this embodiment include:

[0148] Step 201 : read a set of SQL statements to be verified in a database DB material package, and use a preset lexical analyzer to parse each SQL statement in the set of SQL statements to be verified.

[0149] Specifically, the Antlr4MySQL grammar and lexical files can be obtained, and optionally, some grammars that are not adapted in the new version can be added, so as to generate parsing monitoring and interpretation files for the corresponding files.

[0150] In this step, the set of SQL statements to be verified in the database DB material package can be read, and each SQL statement in the set of SQL statements to be verified can be parsed using a preset lexical analyzer, namely the Antlr4 lexical analyzer, to generate a corresponding syntax tree. Then, the first-level structure object type of the syntax tree is parsed, and the statement containing the DdlStatementContext object is identified as a DDL statement. The second-level structure of the syntax tree is parsed to distinguish between create type statements and alter statements, and the third-level syntax tree is parsed for the alter statement, and the specific operation type of the target can also be distinguished.

[0151] Step 202: Determine the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition.

[0152] In this step, DML statements can be parsed into five types of statements: setvalue statements, update statements, replace statements, insert statements, and delete statements. Each type of statement can be verified using the corresponding method.

[0153] DDL statements can be parsed into 11 types of statements, namely: create table statement, delete table statement, alter define field statement, alter add field statement, alter delete field statement, alter modify field statement, alter delete index statement, alter add index statement, alter add primary key statement, alter delete primary key statement, alter add unique key statement. Each type of statement can be verified using the corresponding method.

[0154] Step 203: Obtain the name of the first table involved according to the statement type corresponding to the first SQL statement, and connect to the corresponding first database according to the first table name.

[0155] The first table name involved is obtained according to the statement type corresponding to the first SQL statement, and the corresponding first database is connected according to the first table name, wherein the first SQL statement is any one of the SQL statement set to be verified. If it is determined that the statement whose statement type is the data definition language DDL is the first SQL statement, the first operation type corresponding to the first SQL statement is determined according to the preset classification rule, and the first operation type is used to determine the first operation statement corresponding to the first SQL statement in the first database.

[0156] Specifically, for the new table statement in the DDL statement, it can be verified by checking whether the table structure object has been reviewed. For the delete table statement, it can be verified by checking whether the table structure object has been reviewed. For the alter define field statement, it can be verified by checking whether the field in the table structure object exists and whether the attribute meets the requirements. For the alter add field statement, it can be verified by checking whether the field in the table structure object exists and whether the attribute meets the requirements. For the alter delete field statement, it can be verified by checking whether the field in the table structure object exists. For the alter modify field statement, it can be verified by checking whether the field in the table structure object exists and whether the attribute meets the requirements. For the alter delete index statement, it can be verified by checking whether the index in the table structure object exists. For the alter add index statement, it can be verified by checking whether the index in the table structure object exists and whether the index field is correct. For the alter add primary key statement, it can be verified by checking whether the index in the table structure object exists and whether the index field is correct. For the alter delete primary key statement, it can be verified by checking whether the primary key index in the table structure object exists. For the alter add unique key statement, it can be verified by checking whether the unique index in the table structure object exists. Finally, a check report can be generated based on the verification results.

[0157] Step 204: Obtain a first operation statement corresponding to the first database, and parse the first operation statement according to a preset lexical analyzer to generate a first syntax tree.

[0158] In this step, the first operation statement corresponding to the first database is obtained, and the first operation statement is parsed according to a preset lexical analyzer to generate a first syntax tree. Specifically, the first operation statement can be parsed using an Antlr4 lexical analyzer to generate a corresponding first syntax tree.

[0159] Step 205: Verify the syntax tree to be verified and the first syntax tree according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database.

[0160] In this step, the syntax tree to be verified and the first syntax tree can be verified according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database, wherein the syntax tree to be verified is the syntax tree generated by parsing the first SQL statement using the preset lexical analyzer.

[0161] Continue to refer to Figure 4 , before obtaining the name of the first table involved according to the statement type corresponding to the first SQL statement, the SQL statements in the SQL statement set can also be sorted according to the reading order of each SQL statement in the SQL statement set to be verified and the corresponding table name classification to generate a first table structure object. Then, starting from the last SQL statement of the first table structure object, the database verification is performed in reverse order; wherein, if the verification result is successful, the currently verified SQL statement is deleted from the first table structure object, and the database verification is continued by rolling back to the previous SQL statement, and after the database verification of the first table structure object is completed, the inspection report is output; if the verification result is a failure, the inspection report is output.

[0162] In one possible implementation, after obtaining the DDL syntax tree, different table name acquisition methods are executed for different types of DDL statements to obtain the table names involved in the DDL, then connect to the DB database, and obtain the table creation statement for the table, use the DDL parsing function to generate a syntax tree for the table creation statement, and then parse to generate a Python object, where the Python object is used to record the parsing result of the syntax tree.

[0163] Then, according to the DDL statement reading order and the table names involved in the DDL statements, the DDL statements are classified according to the table names and stored in the order of materials. Then, the DDL statements are checked in batches, database by database and table by table. The production DB table structure object of a single table and the list of DDL syntax trees involved are read, and the DDL statements are verified one by one in reverse order. Different checking rules can be used according to different DDL syntax tree types. Specifically, the DDL syntax tree can be parsed first to parse the modified field names and field attributes, and then the real table structure objects are compared to check whether these field names exist and whether the field attributes are consistent. If it is effective, a rollback operation is performed on the real table production DB table structure object to delete the field. If it does not exist, a record is made. Then the next DDL statement of the table is checked. Finally, the inspection records are sorted out and an inspection report is generated for submission.

[0164] In this embodiment, by reverse parsing the DB material package based on a preset lexical analyzer, DDL verification SQL and DML verification SQL are automatically reversely generated, thereby reducing the manpower investment in manually generating verification solutions and improving the accuracy of the solutions.

[0165] In addition, with the in-depth development of distributed architecture, the rise of microservice applications, and the increase in subsystem scale and system call complexity, the application operation and maintenance costs and application version release verification costs have continued to surge. Operation and maintenance personnel are unable to timely and accurately discover quality problems and locate the root causes of failures from the massive monitoring data after the version change. Therefore, the embodiment of the present application proposes a multivariate operation and maintenance indicator anomaly detection module based on a hybrid algorithm, which is used to pay attention to whether the various indicators of the grayed-out instance are abnormal.

[0166] Among them, instance-level operation and maintenance indicator monitoring data is a typical time series data, which has the characteristics of periodicity, regular spikes, overall rise and fall, and low peak period in the indicator data form. For version release verification, operation and maintenance personnel need to pay attention not only to the host performance indicators of the grayscale instance itself, but also to the impact of version release on business transaction volume, interface time consumption and success rate.

[0167] For multivariate operation and maintenance indicator prediction, the indicator monitoring data is diverse, the data scale is large, and the data flow rate is fast. The anomaly detection model is required to process real-time indicator data to accurately locate the abnormal moment. In daily operation and maintenance, for instance-level operation and maintenance indicator monitoring data, it is unreasonable to judge the entire multidimensional data as abnormal if only one dimension is abnormal. Among them, the existing anomaly detection algorithms are mainly:

[0168] 1. Separate multivariate time series data vertically, and study and discover abnormal sequence patterns through relevant algorithms. Take the multivariate sequence as the overall input to the deep learning algorithm such as differential autoencoder or generative adversarial network, and mine the abnormal time series model. The detection model only outputs abnormal results, and the explanatory power of the excavated abnormal pattern is weak, and it is impossible to explicitly locate the root cause of the fault based on the abnormality.

[0169] 2. Split the multivariate time series data horizontally and convert it into multiple single-dimensional time series, and use the algorithms in the field of single-dimensional time series to detect anomalies.

[0170] Figure 5 is a flowchart of the monitoring performance indicator checking steps shown in Example 3 of the present application, Figure 6 FIG. 1 is a flow chart of another monitoring performance indicator checking step shown in the third embodiment of the present application. Figure 5-Figure 6 As shown, the monitoring performance indicator checking steps provided in this embodiment include:

[0171] Step 301: Obtain instance-level multi-operation and maintenance indicator data in the first execution result, and determine the indicator data type of the instance-level multi-operation and maintenance indicator data according to data characteristics.

[0172] Step 302: Determine a corresponding anomaly detection algorithm according to the indicator data type, and verify the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm.

[0173] The variable time series data can be separated horizontally, and the corresponding anomaly detection algorithm can be matched according to the indicator type. The multivariable time series indicators can be detected in parallel, and the abnormal results can be fed back. The operation and maintenance personnel can quickly locate the abnormal indicator type and time period. Specifically, for multi-dimensional operation and maintenance indicator data with different representation types, a horizontally separated detection algorithm can be used, and a single-dimensional anomaly detector can be used for stable, trend-type, and periodic indicator data, allowing operation and maintenance personnel to quickly determine abnormal indicators. Therefore, during the grayscale release process, operation and maintenance personnel need to discover system anomalies in a timely manner. The hybrid of multiple algorithms avoids the use of multi-dimensional deep learning algorithm modeling and speeds up the anomaly detection time. Then, the corresponding anomaly detection algorithm is determined according to the indicator data type, and the instance-level multi-operation and maintenance indicator data is verified and processed according to the anomaly detection algorithm.

[0174] Specifically, if the determined indicator data type is stable data, the determined anomaly detection algorithm is the first algorithm. If the determined indicator data type is trend data, the determined anomaly detection algorithm is based on the second algorithm. If the determined indicator data type is periodic data, the determined anomaly detection algorithm is a prediction algorithm based on LSTM.

[0175] Step 3031: If the determined indicator data type is stable data, the determined anomaly detection algorithm is the first algorithm.

[0176] For stable operation and maintenance indicators, their values ​​are stable and the fluctuation range of the indicators is not large. Common abnormality types are burr abnormalities and overall rise and fall abnormalities. That is, whether the current moment is normal depends on whether its value is aligned with its recent past. Figure 7 This is a schematic diagram of abnormal data based on the first algorithm in Example 4 of the present application. As shown in Figure 7, if the value surges or decreases temporarily, it is called a glitch. If the value change is permanent, it becomes an overall increase. After the version is released, the operation and maintenance personnel need to pay attention to whether the new version of the application will have a high CPU usage, or excessive IO usage and abnormal success rate jitter.

[0177] Based on the high availability and high performance architecture of the system, threshold monitoring is generally set based on empirical values. This type of traditional monitoring cannot be perceived by operation and maintenance personnel before the indicator reaches the critical alarm value, and potential anomalies cannot be known in advance. Therefore, in this step, common anomalies of stable indicators can be detected based on the first algorithm.

[0178] Step 3032: If the determined indicator data type is trend data, the determined anomaly detection algorithm is the second algorithm.

[0179] In this step, since the time consumption indicators of business interfaces are often aggregated at the data center IDC level, the overall time consumption of application instances in the same database deployment room to process requests is relatively low, showing a multi-cluster center distribution. After the version is released, operation and maintenance personnel need to pay attention to whether there are outliers compared with other instances of the IDC. Traditional monitoring often uses fixed thresholds, but it is necessary to set anomaly detection thresholds for each computer room separately, which has high maintenance costs and requires reconfiguration when switching DBs or replacing interfaces. Therefore, this step uses the second algorithm to calculate the cluster centers of each instance in the IDC, and the distance between the cluster center and the farthest time series in the cluster is used as the anomaly threshold. The distance between the grayscale instance interface time consumption sequence and the cluster center of this computer room is calculated by DTW. If it is greater than the anomaly threshold, the time consumption sequence is determined to be an outlier anomaly.

[0180] Step 3033: If the determined indicator data type is periodic data, the determined anomaly detection algorithm is a LSTM-based prediction algorithm.

[0181] In this step, since the business transaction volume indicator shows a strong periodicity and there are business high and low peaks, the indicator value waveform shape is different according to the actual business situation. If anomaly detection methods such as year-on-year or fixed thresholds are used, the misjudgment and leakage rate are high. Therefore, this step uses the business transaction volume forecast anomaly detection based on the LSTM long-short cycle forecast algorithm, and uses the LSTM algorithm to learn and generate the business transaction volume forecast model to obtain the business transaction volume forecast value R at each judgment time after the version is released. The dynamic threshold setting method is adopted, with days as the period unit, and the current indicator data is compared with the historical data of the same period. The average growth rate Y% of the historical period is calculated, and the data within the growth range (RR*Y%, R+R*Y%) of the same period is determined to be normal data, and the rest are determined to be abnormal values.

[0182] In this embodiment, through a multivariate operation and maintenance indicator anomaly detection method based on a multi-algorithm mixture, abnormal fluctuation detection and outlier detection are performed on the instance-level key monitoring indicators after release.

[0183] Figure 8 is a flow chart based on the third algorithm shown in the fourth embodiment of the present application. Figure 8 As shown, the third algorithm provided in this embodiment includes:

[0184] Step 401: synchronously use two sliding windows to traverse the instance-level multi-operation and maintenance indicator data, and determine characteristic statistics of the two sliding windows.

[0185] In this step, two sliding windows can be used simultaneously to traverse the instance-level multi-operation and maintenance indicator data, and characteristic statistics of the two sliding windows can be determined. The characteristic statistics are used to characterize the central trend characteristics of the instance-level multi-operation and maintenance indicator data.

[0186] Specifically. The third algorithm is an anomaly detection module based on sliding windows and outlier discovery, providing different components that can be combined into anomaly detection models for different scenarios and different types. The third algorithm uses multiple sliding windows to traverse the original data simultaneously, and continuously tracks the difference between the original sequence value and the window average or median. Among them, the characteristic statistics of the above multiple sliding windows can be between the window average or median.

[0187] Step 402: Subtract the instance-level multi-operation and maintenance indicator data from the feature statistics of the two sliding windows respectively, and generate a feature sequence.

[0188] The data at the judgment moment can be subtracted from the statistics of the previous and next reference windows to reconstruct the feature sequence s1. The synchronous sliding window method is used to reconstruct the sequence, and the aggregation operation of the previous and next windows is performed to characterize the abnormal changes of the original sequence through the reconstructed sequence. In order to avoid the insufficient representation of the sequence of a single window sliding, the aggregation operation uses the average value, and the overall statistical window value changes, thereby reducing the impact of a certain outlier.

[0189] Step 403: Use a box plot outlier detection algorithm to test the feature sequence. When it is determined that the deviation of the statistical data of the instance-level multi-operation and maintenance indicator data from the two sliding windows is greater than a preset threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0190] Specifically, the box plot outlier detection method is applied to the feature sequence s1 to test it. When the statistical data in the left and right windows are significantly different, it is determined that an abnormal change occurs at that moment. Fig. 9 This is a schematic diagram of the host indicator time series shown in Example 4 of this application. Fig. 9 As shown in the figure, in this step, in order to consider the integrity of the original sequence, the original window is aggregated and anomaly judged with the average value as the statistical variable. The specific judgment methods for various anomalies are as follows, where μ w1 is the mean of the previous window, μ w2 is the mean of the rear window, v t1 is the indicator value at the time of determination, Q1 is the first quartile, Q3 is the third quartile, and IQR is the interquartile range, that is, the difference between the third quartile and the first quartile. Among them, s1 can be determined according to the following formula:

[0191]

[0192] Continue to refer to Figure 7 and Fig. 9 , for the following types of data anomalies, different anomaly expressions can be constructed:

[0193] Glitch anomaly: If the mean of the front window and the rear window of the anomaly point meets the preset normal conditions and the difference of the mean of the front window and the rear window is within the preset range, and the increase of the indicator data of the anomaly point is greater than the preset increase and greater than the mean of the front and rear windows, then it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the anomaly point belongs to the glitch anomaly. Specifically, the mean of the front and rear windows of the anomaly point is normal and the values ​​are close, and the indicator data of the anomaly point surges and is greater than the mean of the front and rear windows. The abnormal expression is as follows.

[0194]

[0195] Overall increase anomaly: If the mean of the window before the anomaly point meets the preset normal conditions and is smaller than the mean of the window after, and the increase of the indicator data of the anomaly point is greater than the preset increase and greater than the mean of the window before, then the instance-level multi-operation and maintenance indicator data corresponding to the anomaly point is determined to be an overall increase anomaly. That is, the window before the anomaly point is normal and smaller than the window after, and the indicator data of the anomaly point increases sharply and is greater than the mean of the window before.

[0196] If the mean of the window before the abnormal point meets the preset normal conditions and is greater than the mean of the window after, and the increase of the indicator data of the abnormal point is less than the preset increase and greater than the mean of the window before, then it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall decline abnormality. That is, the mean of the window before the abnormal point is normal and greater than the mean of the window after, and the indicator data of the abnormal point surges and is less than the mean of the window before.

[0197] Specifically, the abnormal expression is as follows:

[0198] (μ w1 <μ w2 &&μ w1 <v t1 )||(μ w1 >μ w2 &&μ w2 <v t1 )

[0199] Abnormal determination of reconstructed sequence: subtract the data at the time of determination from the statistics of the front window and the rear window, reconstruct the feature sequence to determine the normal range of the feature sequence. If the difference between the current data and the statistics of the front window and the rear window is not within the normal range, the feature sequence is determined to be abnormal. Specifically, the normal range of the reconstructed sequence s1 is calculated as [Q1-c*IQR, Q3+c*IQR]. If the difference between the current value and the window statistic is not within the normal range, it is determined to be abnormal. According to different types of abnormal manifestations, the parameters of the anomaly detection model, namely the number of windows W and the limit parameter c, are adjusted based on the first algorithm. The normal interval is calculated based on the box plot with quartiles, and different limit parameters c are used.

[0200] The length of the sliding window controls the detection time scale: If it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to a glitch anomaly, the length of the left window will be set to be greater than the length of the right window to increase the amount of data collected before the abnormal point occurs. The multiple sliding windows include the left window and the right window. If the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to an overall lifting anomaly or an overall falling anomaly, the lengths of the left window and the right window are set to meet the length requirements to capture stable data within the preset time length. Specifically, for glitch anomalies, the left window is longer than the right window to capture representative information in the recent past; for overall horizontal lifting or falling anomalies, both the left and right windows should be set to be long enough to capture long-term stable states. Among them, for the front and rear window lengths, the glitch type needs to collect enough values ​​before the abnormal point occurs, and the left window is set to be larger than the right window. The lifting anomaly needs to exclude the interference of the glitch anomaly, and the left and right windows need to be large enough to weaken the interference of the glitch outlier through average aggregation.

[0201] In addition, for the first algorithm implementation, for burr anomalies, the sliding left window can be set to w L is 10, slide right w R The normal range of the historical quartile range is set to 5, and the limit parameter C of the normal range is set to 3, that is, the normal range of the reconstructed sequence s1 is [Q1-3*IQR, Q3+3*IQR]. By setting a larger front window w L and a smaller limit parameter c, to enhance the algorithm's attention to the host indicators before release and its sensitivity to abnormal capture. For the overall rise and fall anomaly, the front and back windows W can be set to 10, and the limit parameter C of the normal range of the historical quartile range can be set to 6, that is, the normal range of the reconstructed sequence s1 is [Q1-6*IQR, Q3+6*IQR]. By setting the same window scale before and after and a larger limit parameter c, we can pay attention to the changes in the indicator values ​​before and after the decision point and enlarge the normal range to avoid the interference of burrs.

[0202] Fig.10 is a flow chart based on the fourth algorithm shown in Example 5 of the present application. Fig.10 As shown, the fourth algorithm provided in this embodiment includes:

[0203] Step 501: Select K initialized indicator data as initial clustering centers.

[0204] In this step, the fourth algorithm may be an iterative density clustering analysis algorithm.

[0205] Step a: Select k initialized samples as the initial cluster centers, a = a1, a2...a k,In this step, the number of IDCs is clustered and the corresponding cluster centers are generated. ,In each IDC, the time consumption indicator sequence of an instance is randomly selected as the ,initial cluster center.

[0206] Step 502: Allocate each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the initial cluster center with the smallest distance according to the distance from the initial cluster center.

[0207] Each data in the instance-level multi-operation and maintenance indicator data is assigned to the class corresponding to the initial cluster center with the smallest distance according to the distance from the initial cluster center, wherein DTW is selected to calculate the cumulative distance as the distance metric.

[0208] Step b: Target the instance time-consuming indicator sequence X for each IDC i Calculate its distance to each cluster center and assign it to the class corresponding to the cluster center with the smallest distance, where the cluster center is the cluster center. In this step, DTW is selected to calculate the cumulative distance as the distance metric to compare the similarity of time series. It is worth noting that the method of calculating cluster distance based on DTW in this step can eliminate the problem of inconsistent scale length that may appear in historical time series data, align the time series indicator sequence, and avoid using Euclidean distance alone as a measure of familiarity.

[0209] Step 503: re-determine the cluster center for the class corresponding to the initial cluster center, so as to cluster a new cluster center as a subsequent step.

[0210] Step c: For each cluster center a j , recalculate its cluster center

[0211] Then, step b and step c are repeatedly executed until the termination condition iteration number or the minimum error change is reached, that is, the completed iteration number or iteration result meets the preset condition.

[0212] Step 504: Repeat assigning each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the cluster center with the smallest distance according to the distance from the current cluster center, and update the cluster center of the corresponding class until the number of iterations is completed or the iteration result meets the preset conditions to determine the cluster center sequence.

[0213] Step 505: Calculate the cumulative distance between the current cluster center of the cluster center sequence and the farthest data in the corresponding class to serve as the abnormal indicator threshold of the instance-level multi-operation and maintenance indicator data.

[0214] The dynamic time warping algorithm DTW used in this step is a common algorithm for time series similarity comparison. Based on the idea of ​​dynamic programming, the optimal warping path is found to stretch or compress the time series, and the cumulative distance is calculated to obtain the familiarity of the two time series. This distance measurement can avoid the inability to calculate time series with inconsistent lengths using only Euclidean distance.

[0215] For a time series T = {q1, q2, ...q n}, R = {c1, c2, ... c m}, the lengths are n and m respectively. The process of applying DTW to calculate the similarity between T and R is as follows:

[0216] Construct a matrix D of size n×m, with matrix element d ij = dist(q i , c j ), where dist is the Euclidean distance calculation function;

[0217] Using dynamic programming search method, search in matrix D from d 11 to d nm All regular search paths are represented by W. According to monotonicity and continuity, it can be seen that the path in the search path W has searched point (i, j), and the next passing point can only be one of the following three cases: (i+1, j), (i+1, j+1), (i, j+1).

[0218] Then through Obtain the path DTW with the minimum regularization cost of the time series T, R, where w k is the kth search path, and the Euclidean distance is used as the metric for each search path; the K in the denominator is mainly used to compensate for regular paths of different lengths. 11 to d nm The shortest path calculation cumulative distance γ, the cumulative distance γ (i, j) is the point q i and c j The Euclidean distance (dist(q i , c j )) and the sum of the minimum cumulative distance to reach the point, that is, γ(i, j) = dist(q i , c j )+min{γ(i-1, j),γ(i-1, j-1),γ(i, j-1)}, and use this as the similarity measure of time series T and R.

[0219] It is worth noting that in this step, the cumulative distance of DTW is used as the distance measure and familiarity measure between the operation and maintenance indicator time series, which can align the scales between the sequences and consider the time continuity of the indicator series, which is more accurate than the Euclidean distance.

[0220] Step 506: If the distance between the instance-level multi-operation and maintenance indicator data and the corresponding cluster center is greater than the abnormal indicator threshold, it is determined that the instance-level multi-operation and maintenance indicator data is abnormal.

[0221] In this step, the cluster core sequence is obtained based on the second algorithm, and each cluster core sequence should be a representation of each IDC time-consuming indicator sequence. The DTW cumulative distance between each cluster core sequence and the farthest sequence in the cluster is calculated as the abnormal threshold distance, and the DTW cumulative distance between the grayed instance business time-consuming indicator sequence and the IDC cluster core sequence is calculated. If it is greater than the abnormal threshold distance, the instance-level multi-operation and maintenance indicator data is judged to be abnormal in time consumption.

[0222] In this embodiment, the cluster center of each IDC time-consuming indicator time series data is calculated, and the distance between the cluster center and the farthest time series sample in the cluster is used as the abnormal threshold. In addition, by setting a dynamic threshold, each IDC time-consuming abnormal threshold is automatically set, which can reduce the maintenance cost of the fixed threshold.

[0223] Fig.11 Schematic diagram of the flow chart of the prediction algorithm based on LSTM shown in Example 6 of the present application. Fig.11 As shown, the LSTM prediction algorithm provided in this embodiment includes:

[0224] Step 601: Generate a prediction model for periodic data memory prediction based on LSTM algorithm and historical multi-operation and maintenance indicator data learning.

[0225] LSTM, or long short-term memory network, is a recurrent neural network that controls the accumulation and transmission of time series information by introducing gate functions, constructs memory units to select and forget some information, and mines the laws of time series changes in time series.

[0226] Fig.12 This is a schematic diagram of the LSTM network structure shown in Example 6 of this application. Fig.12 As shown, the input at time t is the network input value X at the current time t , the LSTM output value h at the previous moment t-1 , the current memory unit state C t C is the core memory cell of the LSTM network, which mainly transmits the cyclic information linearly and outputs the information nonlinearly to the external state h of the hidden layer. t Among them, the forget gate determines the cell state C at the previous moment t-1 How much information needs to be forgotten and passed to C t ; The input gate determines the input X of the network at the current moment t 'How much is saved to the cell state C t ; The output gate determines the control unit state C tHow many outputs are there to the current output value h of the LSTM t .

[0227] In addition, for the construction of LSTM memory unit network, the gate function can be used as a fully connected layer network to output a real vector between 0 and 1, indicating that information transmission is retained at a certain ratio. The gate function can be set as g(x) = σ(wx+b), where σ is the Logistic function, and its output interval is [0,1]; W is the weight vector of the gate function; and b is the bias term. After being processed by the gate function, the outputs of each item in the memory unit are as follows:

[0228] Input gate: i t =σ(W xi x t +W hi h t-1 +b i );

[0229] Forget gate: f t =σ(W xf x t +W hf h t-1 +b f );

[0230] Output gate, o t =σ(W xo x t +W ho h t-1 +b o );

[0231] The current input cell state, C′ t =tanh(W xc x t +W hc h t-1 +b c );

[0232] The current unit status: C t =f t ⊙C t-1 +i t ⊙C′ t ;

[0233] The final output of LSTM: h t =o t ⊙tanh(C t ).

[0234] As for the LSTM prediction algorithm implementation, in this step, the LSTM prediction model is built through the Python commonly used neural network framework keras. It is only necessary to adjust the parameters to activate the function, the number of LSTM layers, and the input and output variable dimensions to optimize the model and shield the underlying model calculation details. The specific construction process is as follows:

[0235] Data normalization and standardization accelerate the loss reduction and convergence speed of the training model. This method uses the Z-score standardization method to make the standardized sequence conform to the standard normal distribution, with a mean of 0 and a standard deviation of 1. The standardization formula is as follows, where μ is the original sequence mean and σ is the original sequence variance:

[0236]

[0237] Activation function selection,determine that the activation function of the LSTM module is tanh, and the activation function of the fully connected artificial neural network receiving and outputting LSTM is linear.

[0238] The model training parameters are determined, and the rejection rate of network nodes is determined to be 0.2 to prevent model overfitting; the Loss calculation method is determined to be the mean square error; the iterative update method of the network weight parameters is determined to be the RMSprop algorithm commonly used in RNN networks.

[0239] The model training hyperparameters epoch and batch size are determined to be 10 and 100, respectively. Batch_size is the size of the data randomly sent to the network for training each time, and is set to 100 to ensure the model convergence speed. Epoch is the number of times a complete data sample is trained in the network, and is set to 10 to ensure the optimization of parameter training.

[0240] Step 602: Determine the predicted value at each moment according to the prediction model, and determine the growth range for the same period according to the predicted value and the average growth rate for the same period.

[0241] Step 603: If the instance-level multi-operation and maintenance indicator data exceeds the growth range of the same period, it is determined that the instance-level multi-operation and maintenance indicator data is abnormal.

[0242] In step 602-step 603, if the instance-level multi-operation and maintenance indicator data is a business transaction volume indicator, it will show strong periodicity, with business high and low peaks. Depending on the actual business situation, the indicator value waveform will be different. Using the business transaction volume prediction anomaly detection based on the LSTM long-short cycle prediction algorithm, the LSTM algorithm is used to learn and generate a prediction model for business transaction volume, and the business transaction volume prediction value R at each judgment time after the version is released is obtained. The dynamic threshold setting method is adopted, with days as the cycle unit, and the current indicator data is compared with the historical data of the same period. The average growth rate Y% of the historical period is calculated, and the data within the growth range (RR*Y%, R+R*Y%) of the same period is determined to be normal data, and the rest are determined to be abnormal values.

[0243] If the year-on-year anomaly detection threshold is used alone for periodic indicator data, the normal range of the indicator cannot be effectively calculated based on the business peaks and low peaks. In this implementation, the normal range of business transaction volume at the instance level at the grayscale moment is calculated by combining the prediction model and statistical year-on-year calculation. Based on the LSTM prediction value and the year-on-year average growth rate as the upper and lower floating range, the upper and lower corridors of the normal range are formed, thereby reducing false alarms for periodic indicators such as business transaction volume.

[0244] Fig.13 Schematic diagram of the process of grayscale release verification system shown in Example 7 of the present application. Fig.13 As shown, the grayscale release verification system 700 provided in this embodiment includes:

[0245] A publishing module 701 is used to execute a first grayscale publishing step when a subsystem is updated, and determine a first execution result after executing the first grayscale publishing step;

[0246] A processing module 702 is used to perform verification processing on the first execution result, wherein the verification processing includes at least one of application verification, database verification, business function verification, and monitoring performance indicator checking, the application verification is used to verify the application configuration result in the first execution result, the database verification is used to verify the database operation statement in the first execution result, the business function verification is used to verify the structured query language SQL statement execution result in the first execution result, and the monitoring performance indicator checking is used to check the instance level indicator data in the first execution result;

[0247] The processing module 702 is further configured to execute a second grayscale publishing step when the first execution result passes the verification process, wherein the second grayscale publishing step is a next grayscale publishing step of the first grayscale publishing step;

[0248] The processing module 702 is further configured to determine that the grayscale release verification corresponding to the version update of the subsystem is successful when all grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process.

[0249] In a possible design, the processing module 702 is specifically configured to:

[0250] Read the set of SQL statements to be verified in the database DB material package, and use a preset lexical analyzer to parse each SQL statement in the set of SQL statements to be verified to generate a corresponding syntax tree;

[0251] Determine the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition;

[0252] Acquire a first table name involved according to a statement type corresponding to a first SQL statement, and connect to a corresponding first database according to the first table name, wherein the first SQL statement is any one statement in the set of SQL statements to be verified;

[0253] Acquire a first operation statement corresponding to the first database, and parse the first operation statement according to the preset lexical analyzer to generate a first syntax tree;

[0254] The syntax tree to be verified and the first syntax tree are verified according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database, and the syntax tree to be verified is a syntax tree generated by parsing the first SQL statement using the preset lexical analyzer.

[0255] In a possible design, the processing module 702 is specifically configured to:

[0256] Determine that the statement whose statement type is data definition language DDL is the first SQL statement;

[0257] A first operation type corresponding to the first SQL statement is determined according to a preset classification rule, where the first operation type is used to determine the first operation statement corresponding to the first SQL statement in the first database.

[0258] In a possible design, before acquiring the name of the first table involved according to the statement type corresponding to the first SQL statement, the method further includes:

[0259] Sorting the SQL statements in the set of SQL statements to be verified according to the reading order of each SQL statement in the set of SQL statements to be verified and the corresponding table name classification to generate a first table structure object;

[0260] Starting from the last SQL statement of the first table structure object, the database verification is performed in reverse order; if the verification result is successful, the SQL statement currently being verified is deleted from the first table structure object, and the database verification is rolled back to the previous SQL statement to continue, and after the database verification is completed in the first table structure object, an inspection report is output; if the verification result is a failure, an inspection report is output.

[0261] In a possible design, the processing module 702 is specifically configured to:

[0262] Acquire the instance-level multi-operation and maintenance indicator data in the first execution result, and determine the indicator data type of the instance-level multi-operation and maintenance indicator data according to data characteristics;

[0263] A corresponding anomaly detection algorithm is determined according to the indicator data type, and verification processing is performed on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm.

[0264] In a possible design, the processing module 702 is specifically configured to:

[0265] Synchronously using multiple sliding windows to traverse the instance-level multi-operation and maintenance indicator data, and determining feature statistics of the multiple sliding windows, wherein the feature statistics are used to characterize the central tendency characteristics of the instance-level multi-operation and maintenance indicator data;

[0266] Subtracting the instance-level multi-operation and maintenance indicator data from the feature statistics of the multiple sliding windows respectively, and generating a feature sequence;

[0267] The feature sequence is tested using a box plot outlier detection algorithm, and when it is determined that the statistical data deviation between the instance-level multi-operation and maintenance indicator data and the multiple sliding windows is greater than a preset threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0268] In a possible design, the anomalies of the instance-level multi-operation and maintenance indicator data include: burr anomalies, overall rise anomalies, and overall fall anomalies;

[0269] Among them, if the mean of the front window and the rear window of the abnormal point meets the preset normal condition and the difference of the mean of the front window and the rear window is within the preset range, and the increase of the abnormal point indicator data is greater than the preset increase and greater than the mean of the front and rear windows, then it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr anomaly, and the multiple sliding windows include the front window and the rear window;

[0270] If the mean of the window before the abnormal point meets the preset normal condition and is less than the window after the mean, and the increase of the indicator data of the abnormal point is greater than the preset increase and greater than the mean of the window before, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall lifting abnormality;

[0271] If the mean of the window before the abnormal point meets the preset normal conditions and is greater than the mean of the rear window, and the increase in the indicator data of the abnormal point is less than the preset increase and greater than the mean of the front window, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall decline anomaly.

[0272] In a possible design, the processing module is specifically used to:

[0273] The feature sequence is reconstructed by subtracting the data at the judgment moment from the statistics of the front window and the rear window to determine the normal range of the feature sequence. If the difference between the current data and the statistics of the front window and the rear window is not within the normal range, the feature sequence is determined to be abnormal.

[0274] In a possible design, if it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr abnormality, the length of the left window is set to be greater than the length of the right window to increase the amount of data collected before the abnormal point occurs, and the multiple sliding windows include the left window and the right window;

[0275] If the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall rising abnormality or the overall falling abnormality, the length of the left window and the right window are set to meet the length requirement to capture stable data within the preset time length.

[0276] In a possible design, the processing module 702 is specifically configured to:

[0277] Select K initialized indicator data as initial cluster centers, where K is a positive integer;

[0278] Allocating each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the initial cluster center with the smallest distance according to the distance from the initial cluster center, wherein the cumulative distance is calculated as the measure of the distance;

[0279] Re-determine the cluster center for the class corresponding to the initial cluster center, so as to cluster the new cluster center as a subsequent step;

[0280] Repeatedly assigning each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the cluster center with the smallest distance according to the distance from the current cluster center, and updating the cluster center of the corresponding class until the number of iterations is completed or the iteration result meets the preset conditions, so as to determine the cluster center sequence;

[0281] Calculating the cumulative distance between the current cluster center of the cluster center sequence and the farthest data in the corresponding class as the abnormal indicator threshold of the instance-level multi-operation and maintenance indicator data;

[0282] If the distance between the instance-level multi-operation and maintenance indicator data and the corresponding cluster center is greater than the abnormal indicator threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

[0283] In a possible design, the processing module 702 is specifically configured to:

[0284] Generate a prediction model for periodic data memory prediction based on LSTM algorithm and historical multi-operation and maintenance indicator data learning;

[0285] Determine the forecast value at each moment according to the forecast model, and determine the growth range for the same period according to the forecast value and the average growth rate for the same period;

[0286] If the instance-level multi-operation and maintenance indicator data exceeds the growth range over the same period, it is determined that the instance-level multi-operation and maintenance indicator data is abnormal.

[0287] The grayscale release verification system provided in this embodiment can be used to execute the steps in the above method embodiment. For details not disclosed in the device embodiment of this application, please refer to the above method embodiment of this application.

[0288] Fig.14 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Fig.14 As shown, this embodiment provides an electronic device 800, including:

[0289] Processor 801; and

[0290] A memory 802, used to store executable instructions of the processor, and the memory may also be a flash memory;

[0291] The processor 801 is configured to execute each step in the above method by executing the executable instructions.

[0292] Optionally, the memory 802 may be independent or integrated with the processor 801 .

[0293] When the memory 802 is a device independent of the processor 801, the electronic device 800 may further include:

[0294] The bus 803 is used to connect the processor 801 and the memory 802 .

[0295] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes each step in the above method.

[0296] This embodiment also provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device implements each step in the above method.

[0297] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0298] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A grayscale release verification method, characterized in that: Applied to a grayscale release verification system, the grayscale release verification system includes multiple subsystems, the version update of each subsystem includes a grayscale release step sequence, the grayscale release step sequence includes multiple grayscale release steps arranged in a preset order, the method includes: When performing a version update on a subsystem, executing a first grayscale release step, and determining a first execution result after executing the first grayscale release step; Performing verification processing on the first execution result, wherein the verification processing includes at least one of application verification, database verification, business function verification, and monitoring performance indicator checking, the application verification is used to verify the application configuration result in the first execution result, the database verification is used to verify the database operation statement in the first execution result, the business function verification is used to verify the structured query language SQL statement execution result in the first execution result, and the monitoring performance indicator checking is used to check the instance level indicator data in the first execution result; If the first execution result passes the verification process, a second grayscale release step is executed, where the second grayscale release step is the next grayscale release step of the first grayscale release step; When all the grayscale release steps in the grayscale release step sequence are executed, and the execution results pass the verification process, it is determined that the grayscale release verification corresponding to the version update of the subsystem is successful; when the verification process includes the database verification, the verification process of the first execution result includes: Read the set of SQL statements to be verified in the database DB material package, and use a preset lexical analyzer to parse each SQL statement in the set of SQL statements to be verified to generate a corresponding syntax tree; Determine the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition; Acquire a first table name involved according to a statement type corresponding to a first SQL statement, and connect to a corresponding first database according to the first table name, wherein the first SQL statement is any one statement in the set of SQL statements to be verified; Acquire a first operation statement corresponding to the first database, and parse the first operation statement according to the preset lexical analyzer to generate a first syntax tree; The syntax tree to be verified and the first syntax tree are verified according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database, and the syntax tree to be verified is a syntax tree generated by parsing the first SQL statement using the preset lexical analyzer.

2. The grayscale release verification method according to claim 1, characterized in that: After determining the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition, the method further includes: Determine that the statement whose statement type is data definition language DDL is the first SQL statement; A first operation type corresponding to the first SQL statement is determined according to a preset classification rule, where the first operation type is used to determine the first operation statement corresponding to the first SQL statement in the first database.

3. The grayscale release verification method according to claim 2, characterized in that: Before acquiring the name of the first table involved according to the statement type corresponding to the first SQL statement, the method further includes: Sorting the SQL statements in the set of SQL statements to be verified according to the reading order of each SQL statement in the set of SQL statements to be verified and the corresponding table name classification to generate a first table structure object; Starting from the last SQL statement of the first table structure object, the database verification is performed in reverse order; if the verification result is successful, the SQL statement currently being verified is deleted from the first table structure object, and the database verification is rolled back to the previous SQL statement to continue, and after the database verification is completed in the first table structure object, an inspection report is output; if the verification result is a failure, an inspection report is output.

4. The grayscale release verification method according to any one of claims 1 to 3, characterized in that: When the verification process includes the monitoring performance indicator check, the verification process of the first execution result includes: Acquire the instance-level multi-operation and maintenance indicator data in the first execution result, and determine the indicator data type of the instance-level multi-operation and maintenance indicator data according to data characteristics; A corresponding anomaly detection algorithm is determined according to the indicator data type, and verification processing is performed on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm.

5. The grayscale release verification method according to claim 4 is characterized in that: If the determined indicator data type is stable data, determining a corresponding anomaly detection algorithm according to the indicator data type, and performing verification processing on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm, includes: Synchronously using multiple sliding windows to traverse the instance-level multi-operation and maintenance indicator data, and determining feature statistics of the multiple sliding windows, wherein the feature statistics are used to characterize the central tendency characteristics of the instance-level multi-operation and maintenance indicator data; Subtracting the instance-level multi-operation and maintenance indicator data from the feature statistics of the multiple sliding windows respectively, and generating a feature sequence; The feature sequence is tested using a box plot outlier detection algorithm, and when it is determined that the statistical data deviation between the instance-level multi-operation and maintenance indicator data and the multiple sliding windows is greater than a preset threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

6. The grayscale release verification method according to claim 5, characterized in that: The anomalies of the instance-level multi-operation and maintenance indicator data include: burr anomalies, overall rise anomalies, and overall fall anomalies; Among them, if the mean of the front window and the rear window of the abnormal point meets the preset normal condition and the difference of the mean of the front window and the rear window is within the preset range, and the increase of the abnormal point indicator data is greater than the preset increase and greater than the mean of the front window and the rear window, then it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr anomaly, and the multiple sliding windows include the front window and the rear window; If the mean of the window before the abnormal point meets the preset normal condition and is less than the window after the mean, and the increase of the indicator data of the abnormal point is greater than the preset increase and greater than the mean of the window before, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall lifting abnormality; If the mean of the window before the abnormal point meets the preset normal conditions and is greater than the mean of the rear window, and the increase in the indicator data of the abnormal point is less than the preset increase and greater than the mean of the front window, it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall decline anomaly.

7. The grayscale release verification method according to claim 6, characterized in that: The testing of the feature sequence using a box plot outlier detection algorithm comprises: The feature sequence is reconstructed by subtracting the data at the judgment moment from the statistics of the front window and the rear window to determine the normal range of the feature sequence. If the difference between the current data and the statistics of the front window and the rear window is not within the normal range, the feature sequence is determined to be abnormal.

8. The grayscale release verification method according to claim 7, characterized in that: If it is determined that the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the burr abnormality, the length of the left window is set to be greater than the length of the right window to increase the amount of data collected before the abnormal point occurs, and the multiple sliding windows include the left window and the right window; If the instance-level multi-operation and maintenance indicator data corresponding to the abnormal point belongs to the overall rising abnormality or the overall falling abnormality, the length of the left window and the right window are set to meet the length requirement to capture stable data within the preset time length.

9. The grayscale release verification method according to claim 4, characterized in that: If the determined indicator data type is trend data, determining a corresponding anomaly detection algorithm according to the indicator data type, and performing verification processing on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm, includes: Select K initialized indicator data as initial cluster centers, where K is a positive integer; Allocating each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the initial cluster center with the smallest distance according to the distance from the initial cluster center, wherein the cumulative distance is calculated as the measure of the distance; Re-determine the cluster center for the class corresponding to the initial cluster center, so as to cluster the new cluster center as a subsequent step; Repeatedly assigning each data in the instance-level multi-operation and maintenance indicator data to the class corresponding to the cluster center with the smallest distance according to the distance from the current cluster center, and updating the cluster center of the corresponding class until the number of iterations is completed or the iteration result meets the preset conditions, so as to determine the cluster center sequence; Calculating the cumulative distance between the current cluster center of the cluster center sequence and the farthest data in the corresponding class as the abnormal indicator threshold of the instance-level multi-operation and maintenance indicator data; If the distance between the instance-level multi-operation and maintenance indicator data and the corresponding cluster center is greater than the abnormal indicator threshold, the instance-level multi-operation and maintenance indicator data is determined to be abnormal.

10. The grayscale release verification method according to claim 4, characterized in that: If the determined indicator data type is periodic data, determining a corresponding anomaly detection algorithm according to the indicator data type, and performing verification processing on the instance-level multi-operation and maintenance indicator data according to the anomaly detection algorithm, includes: Generate a prediction model for periodic data memory prediction based on LSTM algorithm and historical multi-operation and maintenance indicator data learning; Determine the forecast value at each moment according to the forecast model, and determine the growth range for the same period according to the forecast value and the average growth rate for the same period; If the instance-level multi-operation and maintenance indicator data exceeds the growth range over the same period, it is determined that the instance-level multi-operation and maintenance indicator data is abnormal.

11. A grayscale release verification system, characterized in that: include: A publishing module, used to execute a first grayscale publishing step when a subsystem is updated, and determine a first execution result after executing the first grayscale publishing step; a processing module, configured to perform verification processing on the first execution result, wherein the verification processing includes at least one of application verification, database verification, business function verification, and monitoring performance indicator checking, wherein the application verification is used to verify the application configuration result in the first execution result, the database verification is used to verify the database operation statement in the first execution result, the business function verification is used to verify the structured query language SQL statement execution result in the first execution result, and the monitoring performance indicator checking is used to check the instance-level indicator data in the first execution result; The processing module is further configured to execute a second grayscale publishing step when the first execution result passes the verification process, wherein the second grayscale publishing step is a next grayscale publishing step of the first grayscale publishing step; The processing module is further configured to determine that the grayscale release verification corresponding to the version update of the subsystem is successful when all grayscale release steps in the grayscale release step sequence are executed and the execution results pass the verification process; The processing module is specifically used for: Read the set of SQL statements to be verified in the database DB material package, and use a preset lexical analyzer to parse each SQL statement in the set of SQL statements to be verified to generate a corresponding syntax tree; Determine the statement type corresponding to each SQL statement according to the syntax tree corresponding to each SQL statement and the preset object classification condition; Acquire a first table name involved according to a statement type corresponding to a first SQL statement, and connect to a corresponding first database according to the first table name, wherein the first SQL statement is any one statement in the set of SQL statements to be verified; Acquire a first operation statement corresponding to the first database, and parse the first operation statement according to the preset lexical analyzer to generate a first syntax tree; The syntax tree to be verified and the first syntax tree are verified according to the preset syntax tree statement type matching rule to determine the validity status of the table and field attributes in the corresponding database, and the syntax tree to be verified is a syntax tree generated by parsing the first SQL statement using the preset lexical analyzer.

12. An electronic device, characterized in that: include: processor; as well as a memory for storing a computer program for the processor; Wherein, the processor is configured to implement the grayscale release verification method according to any one of claims 1 to 10 by executing the computer program.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the grayscale release verification method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the grayscale release verification method described in any one of claims 1 to 10 is implemented.

Images