A configuration management database audit method, system and related device

Abstract

The present application provides an audit method for a configuration management database, including: obtaining configuration item data in the configuration management database, performing data mapping on the configuration item data, and obtaining various attribute data; determining the corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data; and performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm. The present application analyzes the content of the data stored in the CMDB to abstract an audit scheme for validating the attribute data of configuration items. There is no need to manually set different rules according to each attribute, which simplifies the workload of manual participation in the audit process, expands the scale of the audit, improves the efficiency of data auditing, and helps to improve the data quality of the overall CMDB data. The present application also provides an audit system for a configuration management database, a computer-readable storage medium and an electronic device, a computer-readable storage medium and an electronic device, which have the above-mentioned beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of data auditing, and in particular to an auditing method, system and related devices for a configuration management database. Background Art

[0002] With the development of business scale and the introduction of new IT technologies, the resource volume of enterprise application systems and the complexity of IT architecture are increasing day by day. The Configuration Management Database (CMDB) records the information system configuration items, the attributes of each dimension of configuration items and the relationship between them used in the enterprise IT service process. It is closely linked to all service support and service delivery processes. CMDB is the source of scene interaction data for all operation and maintenance systems in the data center. It is crucial to ensure the accuracy of CMDB data.

[0003] In the process of CMDB construction, data is mainly created and updated by manual input and process-driven methods. As the data scale becomes larger and larger, the integrity, standardization and validity of the data are not verified in time, and there are many problems with the data content carried by CMDB. The traditional management method is to use sampling and manual rule verification audit methods, which can solve the problem of missing data to a certain extent, but cannot find data with abnormal information, which poses a huge challenge to data standardization and data management. Summary of the invention

[0004] The purpose of this application is to provide an audit method, an audit system, a computer-readable storage medium and an electronic device for a configuration management database, which can simplify the workload of manual participation in the audit process, expand the scale of the audit, and improve the efficiency of data auditing.

[0005] In order to solve the above technical problems, this application provides an audit method for a configuration management database, and the specific technical solution is as follows:

[0006] Obtaining configuration item data in the configuration management database;

[0007] Perform data mapping on the configuration item data to obtain the attribute data;

[0008] Determine the corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data;

[0009] The anomaly detection algorithm is used to perform data integrity audit and data validity audit on the configuration item data.

[0010] Optionally, after obtaining the configuration item data in the configuration management database, the method further includes:

[0011] The configuration item data in multi-dimensional states is structured to obtain a configuration item data set that meets the preset standard structure.

[0012] Optionally, data mapping is performed on the configuration item data to obtain attribute data including:

[0013] The attributes of the same configuration item data are sorted in descending order according to the missing values, and the missing data are mapped to globally unique special characters to obtain the attribute data.

[0014] Optionally, a corresponding anomaly detection algorithm is determined according to the distribution characteristics of each attribute data, including:

[0015] Performing feature distribution verification on each of the attribute data;

[0016] If the attribute data passes the feature distribution check and confirms that the attribute data is in a non-identical distribution, a pattern analysis algorithm is used as a corresponding anomaly detection algorithm;

[0017] If the attribute data fails the feature distribution check, it is confirmed that the attribute data is in the same distribution, and an association analysis algorithm is used as the corresponding anomaly detection algorithm.

[0018] Optionally, after performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm, the method further includes:

[0019] The audit results of the configuration item data are displayed using preset charts.

[0020] Optionally, if the attribute data is in a non-identical distribution, performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm includes:

[0021] Determine missing data and non-missing data in the attribute data;

[0022] Performing data annotation on the missing data and classifying them into an abnormal data set;

[0023] For the non-missing data, string splitting and string encoding are performed, anomaly detection is performed on the frequency analysis result after string encoding using an unsupervised first anomaly detection model, and the obtained anomaly detection result is classified into the anomaly data set.

[0024] Optionally, if the attribute data is in a non-identical distribution, performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm includes:

[0025] Determine missing data and non-missing data in the attribute data according to the global unique identifier;

[0026] Performing data annotation on the missing data and classifying them into an abnormal data set;

[0027] For the non-missing data, determining whether there is a common subsequence of the non-missing data;

[0028] If the common subsequence exists in the non-missing data, remove the common subsequence and perform string similarity analysis to obtain a similarity result;

[0029] The second anomaly detection model is used to perform anomaly detection and identification on the similarity results, and the identified anomaly data is classified and annotated and classified into the anomaly data set.

[0030] The present application also provides an audit system for a configuration management database, including:

[0031] A data acquisition module, used to acquire configuration item data in the configuration management database;

[0032] The data mapping module is used to map the configuration item data to obtain the attribute data;

[0033] An algorithm determination module is used to determine the corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data;

[0034] An audit module is used to perform data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm.

[0035] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the method described above when executed by a processor.

[0036] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.

[0037] The present application provides an audit method for a configuration management database, comprising: obtaining configuration item data in the configuration management database, performing data mapping on the configuration item data to obtain various attribute data; determining a corresponding anomaly detection algorithm according to the distribution characteristics of the various attribute data; and performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm.

[0038] This application analyzes the data content stored in the CMDB and abstracts an audit scheme for validating the attribute data of configuration items. It does not require manual setting of different rules based on each attribute, simplifies the workload of manual participation in the audit process, expands the scale of audits, improves data audit efficiency, and helps to improve the data quality of the overall CMDB data.

[0039] The present application also provides an audit system for a configuration management database, a computer-readable storage medium and an electronic device, a computer-readable storage medium and an electronic device, which have the above-mentioned beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0041] Figure 1 A flowchart of an audit method for a configuration management database provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of the audit system structure of a configuration management database provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] Please refer to Figure 1 , Figure 1 A flowchart of an audit method for a configuration management database provided in an embodiment of the present application, the method comprising:

[0045] S101: Acquire configuration item data in the configuration management database;

[0046] This step aims to obtain the data required for the audit process. For example, the configuration item data in the configuration management database can be obtained from the storage medium to provide a basic data source for subsequent data processing and modeling analysis. In the process of obtaining data, the required information can be divided into three steps: configuration item object, configuration item attribute object, and configuration item relationship object. The audit scope targeted by the embodiment of the present application may include the core configuration item data used in various scenarios. After selecting the configuration item object, the configuration item attribute data is obtained, and relationship data information may also be included. For example, in the process of network operation and maintenance, in order to meet the needs of operation and maintenance monitoring, fault handling and other scenarios, it is necessary to obtain configuration item information such as physical device object information such as routers, switches, and firewalls, and logical device object information such as logical routers and logical switches; core attribute information such as configuration item name, name, device status, and usage type of each important configuration item; power supply relationship, storage relationship, virtualization and other association relationships of each important configuration item, etc.

[0047] S102: Perform data mapping on the configuration item data to obtain various attribute data;

[0048] This step aims to map the configuration data to obtain the attribute data it contains. The configuration item data directly obtained usually still has many problems in data content. In order to adapt to the data requirements of the subsequent algorithm model and enrich the data feature information to the greatest extent, this step needs to perform data mapping. Specifically, the attributes of the same configuration item data can be sorted in descending order according to the missing values, and special character data mapping can be performed on the missing data.

[0049] Unified mapping mainly includes two methods: arranging the data of each indicator in reverse order according to the missing value ratio, or mapping the missing data in the indicator data to a globally unique special character. Specifically, it can be arranged in reverse order according to the number of missing values. First, the number of missing values ​​of each indicator in the structure specification data set is counted, and the indicators are arranged according to the number of missing values. The indicators with fewer missing values ​​are placed in front and the indicators with more missing values ​​are placed in the back. The percentage of missing value ratio is set, and only indicators below the missing value ratio threshold are selected. The indicator information is arranged according to the number of missing values, and the indicators with more complete information are placed in the front position of the arrangement. The information completeness decreases with the order of indicator arrangement.

[0050] The purpose of reverse order is to put the data with higher integrity in front, so as to provide more characteristic information for the subsequent model analysis process. In the subsequent analysis of abnormal data, there will be many problems in the processing of missing data. If the entire row of data is deleted, it will affect the accuracy detection of other dimensional data. Therefore, the mapping method of globally unique special characters is used for missing data. Under the premise of not losing data information, the data information of each dimension is used to the greatest extent, ensuring comprehensive accuracy verification of core attribute data.

[0051] In addition, as a preferred execution mode of this step, the configuration item data in a multi-dimensional state can be structured to obtain a configuration item data set that meets the preset standard structure. Due to the different data access methods when obtaining asset configuration data, the data to be audited may be scattered in multiple tables or have data information with multiple structures in one table, which will have a great impact in the subsequent analysis process. The purpose of structured processing is to clean the accessed multi-dimensional heterogeneous data, perform multi-dimensional association and data analysis operations on it, form a standard structured data set, and improve the efficiency of subsequent analysis. Of course, there is no specific limitation on the preset standard structure here, which can be set by those skilled in the art. For example, if multiple tables need to be associated during the data extraction process, the names of each field can be unified, and the data records can be parsed according to the data mapping storage method of the dictionary value, and converted into data records with structural specifications. For configuration item information with a small amount of data, it is possible to directly export the data table object from the front end, perform standardized processing on the data table object in the format of row selection and column name information sorting, and convert it into data records with structural specifications to achieve structured processing.

[0052] After performing data mapping, you can also filter the data to clarify the scope of data audit. Data in different states will be used in different application scenarios. The purpose of filtering data is to split the complex scenario into a single scenario, simplify the analysis algorithm of the subsequent anomaly detection model, and at the same time, it can also facilitate the horizontal expansion of the algorithm model without affecting the overall structure. For the CMDB data of the data center, the configuration item data records the data records of the entire life cycle, but the data in the offline and deregistered states will not be used in the daily operation and maintenance change process. Therefore, in order to ensure the effectiveness of the audit process, the status information of some configuration item attributes is selected to filter out the data in use and accurately filter the required audit scope.

[0053] S103: Determine a corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data;

[0054] This step needs to determine the corresponding anomaly detection algorithm based on the distribution characteristics of the attribute data. Specifically, the attribute data can be subjected to a feature distribution check. If the attribute data passes the feature distribution check, it is confirmed that the attribute data is in a non-identical distribution, and the pattern analysis algorithm is used as the corresponding anomaly detection algorithm; if the attribute data does not pass the feature distribution check, it is confirmed that the attribute data is in an identical distribution, and the association analysis algorithm is used as the corresponding anomaly detection algorithm.

[0055] For a certain configuration item data, it may have multi-dimensional attributes and associations. In this step, it is necessary to perform a characteristic distribution test on each indicator data of the structure specification after output processing, where the null hypothesis is that the distributions of the two indicator data vectors are the same, and the alternative hypothesis is that the distributions of the two indicator data vectors are different. The base point data distribution that needs to be compared is the specific data record of the configuration item indicator.

[0056] Taking router data as an example, its metadata includes configuration item name, name and brand, while the master data information is the specific information of each metadata. The metadata indicator to be compared is the configuration item name, and the metadata indicators that need to be tested are other attributes of the configuration item (name, brand, etc.). The specific content is the indicator name screened out in the unified mapping process. First, each string data is encoded and mapped, and a statistical test is performed on the two mapped vectors. If the test result rejects the original hypothesis, it is considered that the indicators involved in the test are not identically distributed. Otherwise, it is considered that the indicators involved in the test are identically distributed.

[0057] S104: Performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm.

[0058] This step aims to use the anomaly detection algorithm determined in the above steps to perform audits, which specifically include data integrity audits and data validity audits.

[0059] It can be seen that different anomaly detection algorithms can be used for identical distribution and non-identical distribution. More specifically, if the attribute data is in non-identical distribution, this step may include:

[0060] The first step is to determine the missing data and non-missing data in the attribute data;

[0061] The second step is to perform data annotation on the missing data and classify them into abnormal data sets;

[0062] The third step is to perform string splitting and string encoding on non-missing data, use the unsupervised first anomaly detection model to perform anomaly detection on the frequency analysis results after string encoding, and classify the anomaly detection results into the anomaly data set.

[0063] If the attribute data is in non-identical distribution, this step can be divided into the following processes:

[0064] The first step is to determine the missing data and non-missing data in the attribute data based on the global unique identifier;

[0065] The second step is to perform data annotation on the missing data and classify them into abnormal data sets;

[0066] The third step is to determine whether there is a common subsequence in the non-missing data; if there is a common subsequence in the non-missing data, remove the common subsequence and perform string similarity analysis to obtain the similarity result; use the second anomaly detection model to perform anomaly detection and identification on the similarity result, and classify and annotate the identified anomaly data into the anomaly data set.

[0067] Of course, in the above process, the second step and the third step can be processed in parallel during actual execution. The first anomaly detection model and the second anomaly detection model can be trained in advance or directly obtained by a person skilled in the art, and the two can be the same or different models, which are not specifically limited here.

[0068] In this step, integrity assessment refers to the ratio between the number of data marked as missing according to the output of the data modeling module and the total amount of data involved in the overall detection. In traditional detection methods, the audit process can screen out a configuration item data record with missing data, but the selection of specific indicator attributes requires manual setting, and it is impossible to find out whether the manually selected audit indicators are effective and reasonable from a global perspective. The embodiments of the present application can effectively perform algorithm mining and analysis on such problems, and improve the efficiency of the traditional audit analysis process for missing value data retrieval with intelligent analysis methods, and can more effectively perform integrity assessment on the audited configuration item data.

[0069] Effectiveness evaluation refers to the evaluation of abnormal data with different labels based on the classified abnormal data identified by pattern analysis methods and association analysis methods, and the overall evaluation of the effectiveness of the audited configuration item data. In the traditional manual rule-based audit process, it is impossible to find erroneous data in the configuration item records by relying solely on the input rules for judgment. The effectiveness evaluation can locate the specific data records with problems based on the above abnormal data results.

[0070] The embodiment of the present application analyzes the data content stored in the CMDB and abstracts an audit scheme for validating the configuration item attribute data. This eliminates the need to manually set different rules based on each attribute, simplifies the workload of manual participation in the audit process, expands the scale of audits, improves data audit efficiency, and helps to improve the data quality of the overall CMDB data.

[0071] After using the anomaly detection algorithm to perform data integrity audit and data validity audit on the configuration item data, the audit results of the configuration item data can also be displayed using preset charts to achieve audit visualization. Audit visualization is a visual display of the analysis results of the configuration item data. Through the visualization chart, the integrity of the detection data, the correlation between the attribute of each configuration item indicator, and the abnormal data classification results can be intuitively discovered. Audit visualization displays the audit content and audit results in the form of a panel, providing CMDB managers and data maintenance personnel with a convenient channel to quickly discover data problems and quickly solve data problems.

[0072] An audit system for a configuration management database provided in an embodiment of the present application is introduced below. The audit system described below and the audit method for the configuration management database described above can be referenced to each other.

[0073] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an audit system for a configuration management database provided in an embodiment of the present application. The present application also provides an audit system for a configuration management database, including:

[0074] A data acquisition module, used to acquire configuration item data in the configuration management database;

[0075] The data mapping module is used to map the configuration item data to obtain the attribute data;

[0076] An algorithm determination module is used to determine the corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data;

[0077] An audit module is used to perform data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm.

[0078] Based on the above embodiments, as a preferred embodiment, it also includes:

[0079] The data preprocessing module is used to perform structured processing on the configuration item data in the multi-dimensional state to obtain a configuration item data set that meets the preset standard structure.

[0080] Based on the above embodiment, as a preferred embodiment, the data mapping module includes:

[0081] The mapping unit is used to sort the attributes belonging to the same configuration item data in descending order according to the missing values, and perform global unique special character mapping on the missing data to obtain the attribute data.

[0082] Based on the above embodiment, as a preferred embodiment, the algorithm determination module includes:

[0083] The algorithm detection unit is used to perform feature distribution verification on each of the attribute data; if the attribute data passes the feature distribution verification, it is confirmed that the attribute data is in a non-identical distribution, and a pattern analysis algorithm is used as the corresponding anomaly detection algorithm; if the attribute data does not pass the feature distribution verification, it is confirmed that the attribute data is in an identical distribution, and an association analysis algorithm is used as the corresponding anomaly detection algorithm.

[0084] Based on the above embodiments, as a preferred embodiment, it also includes:

[0085] The audit display module is used to display the audit results of the configuration item data using a preset chart.

[0086] Based on the above embodiment, as a preferred embodiment, the audit module includes:

[0087] The pattern analysis unit is used to determine missing data and non-missing data in the attribute data; perform data labeling on the missing data and classify it into an abnormal data set; perform string splitting and string encoding on the non-missing data, use an unsupervised first anomaly detection model to perform anomaly detection on the frequency analysis result after string encoding, and classify the obtained anomaly detection result into the abnormal data set.

[0088] Based on the above embodiment, as a preferred embodiment, the audit module includes:

[0089] The association analysis unit is used to determine the missing data and non-missing data in the attribute data according to the global unique identifier; perform data labeling on the missing data and classify it into an abnormal data set; for the non-missing data, determine whether there is a common subsequence of the non-missing data; if the non-missing data has the common subsequence, remove the common subsequence and perform string similarity analysis to obtain a similarity result; use the second anomaly detection model to perform anomaly detection and identification on the similarity result, and classify and label the identified abnormal data into the abnormal data set.

[0090] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0091] The present application also provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.

[0092] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0093] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0094] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for auditing a configuration management database, characterized in that: include: Obtaining configuration item data in the configuration management database; Perform data mapping on the configuration item data to obtain the attribute data; Determine the corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data; Performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm; Among them, the configuration item data is mapped to obtain various attribute data including: The attributes of the same configuration item data are sorted in descending order according to the missing values, and the missing data are mapped to globally unique special characters to obtain the attribute data.

2. The audit method according to claim 1, characterized in that: After obtaining the configuration item data in the configuration management database, the method further includes: The configuration item data in multi-dimensional states is structured to obtain a configuration item data set that meets the preset standard structure.

3. The audit method according to claim 1, characterized in that: The corresponding anomaly detection algorithm is determined according to the distribution characteristics of each attribute data, including: Performing feature distribution verification on each of the attribute data; If the attribute data passes the feature distribution check and confirms that the attribute data is in a non-identical distribution, a pattern analysis algorithm is used as a corresponding anomaly detection algorithm; If the attribute data fails the feature distribution check, it is confirmed that the attribute data is in the same distribution, and an association analysis algorithm is used as the corresponding anomaly detection algorithm.

4. The audit method according to claim 1, characterized in that: After performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm, the method further includes: The audit results of the configuration item data are displayed using preset charts.

5. The audit method according to claim 3, characterized in that: If the attribute data is in a non-identical distribution, performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm includes: Determine missing data and non-missing data in the attribute data; Performing data annotation on the missing data and classifying them into an abnormal data set; For the non-missing data, string splitting and string encoding are performed, anomaly detection is performed on the frequency analysis result after string encoding using an unsupervised first anomaly detection model, and the obtained anomaly detection result is classified into the anomaly data set.

6. The audit method according to claim 3, characterized in that: If the attribute data are in the same distribution, performing data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm includes: Determine missing data and non-missing data in the attribute data according to the global unique identifier; Performing data annotation on the missing data and classifying them into an abnormal data set; For the non-missing data, determining whether there is a common subsequence of the non-missing data; If the common subsequence exists in the non-missing data, remove the common subsequence and perform string similarity analysis to obtain a similarity result; The second anomaly detection model is used to perform anomaly detection and identification on the similarity results, and the identified anomaly data is classified and annotated and classified into the anomaly data set.

7. An audit system for a configuration management database, characterized in that: include: A data acquisition module, used to acquire configuration item data in the configuration management database; The data mapping module is used to map the configuration item data to obtain the attribute data; An algorithm determination module is used to determine the corresponding anomaly detection algorithm according to the distribution characteristics of each attribute data; An audit module, used to perform data integrity audit and data validity audit on the configuration item data using the anomaly detection algorithm; Wherein, the data mapping module includes: The mapping unit is used to sort the attributes belonging to the same configuration item data in descending order according to the missing values, and perform global unique special character mapping on the missing data to obtain the attribute data.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the audit method for the configuration management database as described in any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the audit method for the configuration management database as described in any one of claims 1 to 6 are implemented.

Images