A method and device for evaluating a security system based on a standard model

Abstract

The present invention discloses a method and device for evaluating a security system based on a standard model, which are used for evaluating the performance of a security system. The method comprises: obtaining a system performance measurement data set; the system performance measurement data set comprises a system operation monitoring data subset, a system fault data subset, an environmental data subset, a human factor data subset, a system development data subset, a system use and maintenance data subset and a system daily management data subset; preprocessing and data mining the system performance measurement data set to obtain a data set to be evaluated; using a standard model set, evaluating the data set to be evaluated to obtain an information performance evaluation result information set; the information performance evaluation result information set comprises overall performance evaluation result information, system reliability evaluation result information and system quality evaluation result information; the standard model set comprises an overall evaluation model, a reliability evaluation model and a quality evaluation model.

Description

Technical Field

[0001] The present invention relates to the field of indicator system evaluation, and in particular to a guarantee system evaluation method and device based on a standard model. Background Art

[0002] With the development of modern society, a large number of infrastructure support systems, such as computing systems and network communication systems, have been widely adopted. These large-scale support systems have become increasingly complex with the influence of factors such as cities, economy, war, and environment. The performance of these support systems is not only affected by a single factor, but by the combined effect of multiple interrelated and mutually influential factors. In different occasions and application environments, the actual performance of the support system will vary greatly. Therefore, it is particularly important to obtain accurate measurement data, establish an accurate evaluation model based on the measurement data, and conduct an accurate and comprehensive evaluation of the performance of the support system. Summary of the invention

[0003] The present invention mainly solves the problem of obtaining accurate measurement data, establishing an accurate evaluation model based on the measurement data, and accurately and comprehensively evaluating the performance of a security system. The present invention discloses a security system evaluation method and device based on a standard model, which are used to evaluate the performance of a security system. The method includes:

[0004] S1, obtaining a system performance measurement data set; the system performance measurement data set includes a system operation monitoring data subset, a system fault data subset, an environmental data subset, a human factors data subset, a system development data subset, a system use and maintenance data subset, and a system daily management data subset;

[0005] S2, performing preprocessing and data mining on the system performance measurement data set to obtain a data set to be evaluated;

[0006] S3, using a set of standard models, evaluates the data set to be evaluated to obtain an information performance evaluation result information set; the information performance evaluation result information set includes overall performance evaluation result information, system reliability evaluation result information and system quality evaluation result information; the standard model set includes an overall evaluation model, a reliability evaluation model and a quality evaluation model.

[0007] The preprocessing and data mining of the system performance measurement data to obtain a data set to be evaluated includes:

[0008] S21, performing a preprocessing operation on each data subset in the system performance measurement data set to obtain a preprocessed data set;

[0009] S22, performing data mining processing on the preprocessed data set to obtain a mining data set;

[0010] S23, determining the preprocessed data set and the mining data set as the data set to be evaluated.

[0011] The preprocessing operation is performed on each data subset in the system performance measurement data set to obtain a preprocessed data set, including:

[0012] S211, performing data cleaning processing on the system performance measurement data set to obtain a cleaned data set;

[0013] S212, performing unified processing of collected information on the cleaned data set to obtain a standardized data set;

[0014] S213, performing boundary check and category consistency check processing on the standard data set to obtain a preprocessed data set.

[0015] The step of uniformly processing the collected information of the cleaned data set to obtain a standardized data set includes:

[0016] S2121, setting a unified data collection time and a unified data collection location for the measurement data of the same data attribute contained in the same data subset of the cleaned data set;

[0017] S2122, for the measurement data of the same data attribute contained in each data subset, aligning the collection time and collection location of the measurement data according to the unified data collection time and the unified data collection location to obtain standardized measurement data; and updating each data subset using the standardized measurement data;

[0018] S2123, using all updated data subsets, construct a standard data set;

[0019] The performing boundary checking and category consistency checking on the standard data set to obtain a consistent data set includes:

[0020] S2131, for each data attribute in each standard data subset of the standard data set, preset a corresponding value range; the value range includes an upper limit value of the value range and a lower limit value of the value range;

[0021] S2132, for each measurement data of each standard data subset of the standard data set, judging whether the measurement data value is within the value range according to the value range of its data attribute; when the measurement data is within the value range, not processing the measurement data; when the measurement data is not within the value range, setting the measurement data value to the value range upper limit value or value range lower limit value of the value range closest to the measurement data;

[0022] S2133, for each measurement data in the standard data subset in the standard data set, determine whether its data category belongs to a preset category range of the standard data subset, and delete the measurement data whose data category does not belong to the preset category range of the standard data subset from the standard data subset;

[0023] S2134, execute S2132 and S2133 on each standard data subset of the standard data set to obtain a preprocessed data set.

[0024] The performing data mining processing on the preprocessed data set to obtain a mining data set includes:

[0025] Performing reliability mining processing on the system operation monitoring data subset, the system fault data subset, and the environment data subset to obtain a system reliability measurement data set;

[0026] Sequence mining is performed on the human factor data subset, the system development data subset, the system use and maintenance data subset, and the system daily management data subset to obtain a system quality assessment data set.

[0027] The method of using the standard model set to evaluate the data set to be evaluated to obtain an information performance evaluation result information set includes:

[0028] Using the overall evaluation model, the preprocessed data set is processed to obtain overall performance evaluation result information; the overall performance evaluation result information is used to evaluate the overall performance of the security system;

[0029] Using a reliability evaluation model, the system reliability measurement data set is processed to obtain system reliability evaluation result information; the system reliability evaluation result information is used to characterize the reliability of the support system;

[0030] The system quality assessment data set is processed using a quality assessment model to obtain system quality assessment result information; the system quality assessment result information is used to characterize the quality of the security system.

[0031] The overall evaluation model is used to process the preprocessed data set to obtain overall performance evaluation result information, including:

[0032] Get the standard values ​​of all data attributes;

[0033] Using the standard values ​​of all data attributes, a standard vector is constructed;

[0034] For all the measurement data of each data attribute in the preprocessed data set, construct a measurement sequence of the data attribute;

[0035] For the measurement sequence of each data attribute, the corresponding standard value is subtracted to obtain the difference measurement sequence of each data attribute;

[0036] A first measurement matrix is ​​constructed using the difference measurement sequences of all data attributes; a row vector of the first measurement matrix is ​​a difference measurement sequence of a data attribute; a column vector of the first measurement matrix represents measurement data of all data attributes collected at the same time;

[0037] Performing polynomial extraction processing on the first measurement matrix to obtain a characteristic polynomial;

[0038] Performing difference feature calculations on all column vectors of the first measurement matrix and the standard vector respectively to obtain difference vectors;

[0039] The expression for calculating the difference feature is:

[0040]

[0041] Among them, p j represents the jth term of the difference vector, A ij represents the element in the i-th row and j-th column of the first measurement matrix, b i represents the i-th element of the standard vector, M represents the row dimension of the first measurement matrix, α1, α2, α3 are preset weighting coefficients, b represents the standard vector, A j represents the j-th column of the first measurement matrix;

[0042] Determine the sequence number of the element with the largest value of the difference vector as the feature sequence number;

[0043] Substitute the characteristic sequence number into the characteristic polynomial to calculate and obtain the overall performance evaluation result information.

[0044] The present invention also discloses a security system evaluation device based on a standard model, the device comprising:

[0045] A memory storing executable program code;

[0046] a processor coupled to the memory;

[0047] The processor calls the executable program code stored in the memory to execute the standard model-based assurance system assessment method.

[0048] The present invention also discloses a computer storable medium, wherein the computer storable medium stores computer instructions, and when the computer instructions are called, they are used to execute the standard model-based assurance system assessment method.

[0049] The present invention also discloses an information data processing terminal, which is used to implement the standard model-based security system evaluation method.

[0050] The beneficial effects of the present invention are:

[0051] This technology realizes the multi-layer indicator evaluation of the performance of the security system. It uses the proposed evaluation model algorithm and feature data processing method to achieve a comprehensive, systematic and accurate evaluation of complex business scenarios. It is mainly used in decision support, risk management and performance evaluation of enterprises, institutions or projects. It provides users with efficient, accurate and reliable evaluation methods, improves evaluation efficiency, and reduces the impact of human factors on evaluation results, thereby ensuring the objectivity and fairness of the evaluation results.

[0052] The evaluation system based on standards will change with the iteration of standards. The present invention dynamically collects performance data, constructs a multi-level evaluation index system by hierarchical configuration weights, and flexibly binds each final-level index to the index evaluation model constructed based on standard data, thereby realizing the ability of multi-layer index evaluation. The evaluation model can be constructed externally, so it supports different types of models such as standard calculation models and expert review models, and also supports manual input of values, and the connection of external system data to realize the evaluation and calculation of the final-level index, and then calculates step by step through the hierarchical weight rules to form the final evaluation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flowchart of the implementation of the security system evaluation method of the present invention;

[0054] Figure 2 It is a flow chart for implementing the system evaluation method of the present invention. DETAILED DESCRIPTION

[0055] In order to better understand the content of the present invention, two embodiments are given here.

[0056] Figure 1 is a flowchart of the implementation of the security system evaluation method of the present invention; Figure 2 It is a flow chart for implementing the system evaluation method of the present invention.

[0057] Embodiment 1:

[0058] In a first aspect of an embodiment of the present application, a standard model-based security system evaluation method is disclosed, which is used to evaluate the performance of a security system, including:

[0059] S1, obtaining a system performance measurement data set; the system performance measurement data set includes a system operation monitoring data subset, a system fault data subset, an environmental data subset, a human factors data subset, a system development data subset, a system use and maintenance data subset, and a system daily management data subset;

[0060] The system operation monitoring data subset is obtained by detecting the system operation process; the system fault data subset is a collection of data collected during the system fault process; the environmental data subset is a collection of system environmental data collected; the human factors data subset is a collection of human factors engineering data during the system operation process; the system development data subset is a collection of data collected during the system development process; the system use and maintenance data subset is a collection of data collected during the system use and maintenance process; the system daily management data subset is a collection of data collected during the daily management and maintenance of the system.

[0061] S2, performing preprocessing and data mining on the system performance measurement data set to obtain a data set to be evaluated;

[0062] S3, using a standard model set to evaluate the data set to be evaluated, and obtain an information performance evaluation result information set; the information performance evaluation result information set includes overall performance evaluation result information, system reliability evaluation result information and system quality evaluation result information; the standard model set includes an overall evaluation model, a reliability evaluation model and a quality evaluation model;

[0063] The preprocessing and data mining of the system performance measurement data to obtain a data set to be evaluated includes:

[0064] S21, performing a preprocessing operation on each data subset in the system performance measurement data set to obtain a preprocessed data set;

[0065] S22, performing data mining processing on the preprocessed data set to obtain a mining data set;

[0066] S23, determining the preprocessed data set and the mining data set as the data set to be evaluated.

[0067] The preprocessing operation is performed on each data subset in the system performance measurement data set to obtain a preprocessed data set, including:

[0068] S211, performing data cleaning processing on the system performance measurement data set to obtain a cleaned data set;

[0069] S212, performing unified processing of collected information on the cleaned data set to obtain a standardized data set;

[0070] S213, performing boundary check and category consistency check processing on the standard data set to obtain a preprocessed data set;

[0071] The data cleaning process is to perform data cleaning on each data subset in the system performance measurement data set, including filling missing values, smoothing noise data, smoothing or deleting outliers; the smoothed noise data is to first identify the noise data, and then smooth the noise data according to the data before and after the noise data; the noise data is a value whose value is less than the detection sensitivity of the sensor of the measurement data, or greater than the measurement upper limit of the sensor of the measurement data. The outlier point can be identified by the Kalman filtering method. The filling value of the missing value can be determined by averaging the measured values ​​within a certain sampling interval before and after the missing value.

[0072] The step of uniformly processing the collected information of the cleaned data set to obtain a standardized data set includes:

[0073] S2121, setting a unified data collection time and a unified data collection location for the measurement data of the same data attribute contained in the same data subset of the cleaned data set;

[0074] S2122, for the measurement data of the same data attribute contained in each data subset, aligning the collection time and collection location of the measurement data according to the unified data collection time and the unified data collection location to obtain standardized measurement data; and updating each data subset using the standardized measurement data;

[0075] S2123, using all updated data subsets, construct a standard data set;

[0076] The aligning of the acquisition time and the acquisition location of the measurement data comprises:

[0077] When the time interval for collecting the measurement data is greater than the time interval for the unified data collection time, interpolation processing is performed on the adjacent measurement data to obtain the measurement data value at the unified data collection time, and the value is used as the standard measurement data;

[0078] When the time interval for collecting the measurement data is less than the time interval for the unified data collection time, sampling the measurement data to obtain measurement data consistent with the unified data collection time, and using the measurement data as standard measurement data;

[0079] When the spatial interval of the measurement data collection is greater than the spatial interval of the unified data collection location, interpolation processing is performed on the adjacent measurement data to obtain the measurement data value at the unified data collection location, and the value is used as the standard measurement data;

[0080] When the spatial interval of the measurement data collection is smaller than the spatial interval of the unified data collection location, the measurement data is sampled to obtain measurement data consistent with the unified data collection location, which is used as the standard measurement data.

[0081] The performing boundary checking and category consistency checking on the standard data set to obtain a consistent data set includes:

[0082] S2131, for each data attribute in each standard data subset of the standard data set, preset a corresponding value range; the value range includes an upper limit value of the value range and a lower limit value of the value range;

[0083] S2132, for each measurement data of each standard data subset of the standard data set, judging whether the measurement data value is within the value range according to the value range of its data attribute; when the measurement data is within the value range, not processing the measurement data; when the measurement data is not within the value range, setting the measurement data value to the value range upper limit value or value range lower limit value of the value range closest to the measurement data;

[0084] S2133, for each measurement data in the standard data subset in the standard data set, determine whether its data category belongs to a preset category range of the standard data subset, and delete the measurement data whose data category does not belong to the preset category range of the standard data subset from the standard data subset;

[0085] S2134, executing S2132 and S2133 on each standard data subset of the standard data set to obtain a preprocessed data set;

[0086] The performing data mining processing on the preprocessed data set to obtain a mining data set includes:

[0087] Performing reliability mining processing on the system operation monitoring data subset, the system fault data subset, and the environment data subset to obtain a system reliability measurement data set;

[0088] Performing sequence mining on the human factor data subset, the system development data subset, the system use and maintenance data subset, and the system daily management data subset to obtain a system quality assessment data set;

[0089] The reliability mining process may be implemented by using a pattern evaluation algorithm or a graph pattern mining algorithm;

[0090] The sequence mining process can be implemented by a sequence mining algorithm or an itemset mining algorithm.

[0091] The reliability mining process is performed on the system operation monitoring data subset, the system fault data subset, and the environment data subset to obtain the system reliability measurement data set, which may be:

[0092] Reliability mining is performed on the system operation monitoring data subset, the system fault data subset, and the environment data subset to obtain corresponding reliable measurement data subsets; all reliable measurement data subsets are merged to obtain a system reliability measurement data set;

[0093] For sequence mining processing, the above method can also be referred to, and each data subset can be mined separately, and the mining results can be merged to obtain a system quality assessment data set;

[0094] In this embodiment, the overall performance evaluation result information is a primary indicator, and the system reliability evaluation result information and the system quality evaluation result information are secondary indicators.

[0095] The standard model set is used to evaluate the data set to be evaluated, and an information performance evaluation result information set is obtained, including:

[0096] Using the overall evaluation model, the preprocessed data set is processed to obtain overall performance evaluation result information; the overall performance evaluation result information is used to evaluate the overall performance of the security system, and the higher the value, the better the system performance;

[0097] Using a reliability evaluation model, the system reliability measurement data set is processed to obtain system reliability evaluation result information; the system reliability evaluation result information is used to characterize the reliability of the security system, and the higher the value, the better the reliability;

[0098] Using the quality assessment model, the system quality assessment data set is processed to obtain system quality assessment result information; the system quality assessment result information is used to characterize the quality of the security system, and the higher the value, the better the system quality;

[0099] The overall evaluation model is used to process the preprocessed data set to obtain overall performance evaluation result information, including:

[0100] Obtaining standard values ​​of all data attributes; the standard values ​​are values ​​measured for each data attribute when the security system is operating normally; constructing a standard vector using the standard values ​​of all data attributes;

[0101] For all the measurement data of each data attribute in the preprocessed data set, construct a measurement sequence of the data attribute;

[0102] The measurement sequence is constructed by arranging all the measurement data of the same data attribute in the preprocessed data set in chronological order according to their measurement time.

[0103] For the measurement sequence of each data attribute, the corresponding standard value is subtracted to obtain the difference measurement sequence of each data attribute;

[0104] A first measurement matrix is ​​constructed using the difference measurement sequences of all data attributes; a row vector of the first measurement matrix is ​​a difference measurement sequence of a data attribute; a column vector of the first measurement matrix represents measurement data of all data attributes collected at the same time;

[0105] The data attributes corresponding to the row numbers of the first measurement matrix are the same as the data attributes corresponding to the same element numbers of the standard vector.

[0106] Performing polynomial extraction processing on the first measurement matrix to obtain a characteristic polynomial;

[0107] Performing difference feature calculations on all column vectors of the first measurement matrix and the standard vector respectively to obtain difference vectors;

[0108] The expression for calculating the difference feature is:

[0109]

[0110] Among them, p j represents the jth term of the difference vector, Ai j represents the element in the i-th row and j-th column of the first measurement matrix, b i represents the i-th element of the standard vector, M represents the row dimension of the first measurement matrix, α1, α2, α3 are preset weighting coefficients, b represents the standard vector, A j represents the j-th column of the first measurement matrix;

[0111] Determine the sequence number of the element with the largest value of the difference vector as the feature sequence number;

[0112] Substituting the characteristic sequence number into the characteristic polynomial to calculate and obtain the overall performance evaluation result information;

[0113] The security system may be an information system, a power security system, a network communication system, or a computing power system;

[0114] The reliability evaluation model can be implemented by using a fuzzy comprehensive evaluation method, a hierarchical analysis method or a comparative evaluation method;

[0115] The quality evaluation model can be implemented by using an evaluation method based on AHP (Analytic Hierarchy Process) or an evaluation method based on Bayesian network.

[0116] Performing polynomial extraction processing on the first measurement matrix to obtain a characteristic polynomial includes:

[0117] Decomposing the first measurement matrix to obtain a left decomposition matrix, a characteristic matrix and a right decomposition matrix of the first measurement matrix;

[0118] The decomposition process is calculated as follows:

[0119] Y=UAV,

[0120] Among them, U is the left decomposition matrix, A is the characteristic matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0121] The decomposition process can be implemented by using a matrix singular value decomposition algorithm.

[0122] The diagonal elements of the feature matrix are extracted to obtain a feature vector; the feature vector is represented by I a , I a =[λ1,λ2,…,λ N1 ], N1 is the number of elements contained in the eigenvector;

[0123] Performing linear fitting processing on the elements and element sequence values ​​of the characteristic vector to obtain a characteristic polynomial;

[0124] The linear fitting process uses the characteristic vector element serial number value Ix as a known independent variable and the characteristic vector element value as a known dependent variable, and constructs the curve to be approximated using the known independent variable and the known dependent variable, and uses the function approximation method to perform curve fitting on the curve to be approximated to obtain the optimal evaluation polynomial f(Ix).

[0125] The curve fitting of the curve to be approximated by the function approximation method may be performed by using the best consistent linear approximation method. The characteristic polynomial f(Ix) is expressed as:

[0126] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+a0,

[0127] Wherein, P1 is the order of the characteristic polynomial f(Ix), α0, a1, α2, ..., a P1 are the coefficients of the characteristic polynomial f(Ix);

[0128] The quality assessment model is used to process the system quality assessment data set to obtain system quality assessment result information, including:

[0129] Using the system quality assessment data set, a quality assessment matrix is ​​constructed; the row vector of the quality assessment matrix is ​​a sequence of measurement data of each data attribute;

[0130] Performing eigenvalue calculation processing on the quality assessment matrix to obtain an eigenvalue sequence;

[0131] For each row vector of the quality assessment matrix, a corresponding mean is calculated;

[0132] The eigenvalue sequence is used as a weighted vector, and the mean corresponding to each row vector is weighted summed to obtain the system quality assessment result information.

[0133] The reliability evaluation model is used to process the system reliability measurement data set to obtain system reliability evaluation result information, including:

[0134] S201, constructing a second capability matrix using the system reliability measurement data set; a row vector of the second capability matrix is ​​a sequence of measurement data of each data attribute in the system reliability measurement data set;

[0135] S202, performing cross-correlation calculation processing on the second capability matrix to obtain a cross-correlation matrix;

[0136] S203, performing feature extraction processing on the cross-correlation matrix to obtain a weighted vector;

[0137] S204: Using the weighted vector, perform weighted summation on the means of all row vectors of the second capability matrix to obtain system reliability evaluation result information.

[0138] The step of performing cross-correlation value calculation processing on the second capability matrix to obtain a cross-correlation matrix includes:

[0139] Performing cross-correlation calculation processing on the row vectors of the second capability matrix to obtain cross-correlation values, and constructing a cross-correlation matrix using the cross-correlation values; the elements of the i-th row and j-th column of the cross-correlation matrix are obtained by performing cross-correlation calculation on the i-th row vector and the j-th row vector of the second capability matrix;

[0140] The performing feature extraction processing on the mutual correlation matrix to obtain a weighted vector includes:

[0141] The eigenvalue sequence and the corresponding eigenvector of the cross-correlation matrix are calculated; the cross-correlation matrix is ​​expressed as:

[0142]

[0143] Among them, rij represents the mutual correlation coefficient between the i-th row vector and the j-th row vector of the first measurement matrix, i, j = 1, 2, ..., z, z = 5; the eigenvalue sequence is expressed as λ1, λ2, ..., λ z , where λ i represents the eigenvalue of the i-th row vector, and the eigenvector corresponding to the i-th row vector is f i ;

[0144] The eigenvalues ​​are used to perform normalized accumulation processing on the eigenvectors to obtain normalized accumulated eigenvectors; the normalized accumulation processing includes:

[0145] Perform normalized accumulation calculation on each element of the feature vector to obtain the corresponding normalized accumulated value; use all the normalized accumulated values ​​to construct a normalized accumulated feature vector;

[0146] The process of normalized cumulative calculation is:

[0147]

[0148] Among them, f ij Denotes the eigenvector f i The jth element of u represents the eigenvalue of the uth row vector of the first measurement matrix, l ij represents the i-th normalized cumulative feature vector l i The j-th element of .

[0149] Using the eigenvalue sequence, the corresponding normalized accumulated eigenvectors are weighted and summed to obtain a weighted vector; the weighted vector is represented as [y1, y2, ..., y 10 ], where the jth element y of the weighted vector j The calculation expression is:

[0150]

[0151] Among them, y j It represents the j-th element of the weighted vector and also the weighted value corresponding to the j-th row vector.

[0152] In a second aspect of the embodiment of the present application, a security system evaluation device based on a standard model is disclosed, the device comprising:

[0153] A memory storing executable program code;

[0154] a processor coupled to the memory;

[0155] The processor calls the executable program code stored in the memory to execute the standard model-based assurance system assessment method.

[0156] In a third aspect of an embodiment of the present application, a computer storable medium is disclosed, wherein the computer storable medium stores computer instructions, and when the computer instructions are called, they are used to execute the standard model-based assurance system assessment method.

[0157] In a fourth aspect of the embodiments of the present application, an information data processing terminal is disclosed, wherein the information data processing terminal is used to implement the standard model-based security system evaluation method.

[0158] Embodiment 2:

[0159] This embodiment relates to a multi-layer indicator system evaluation device based on a model of standard data. In order to solve the problems existing in the current key indicator system evaluation, this embodiment focuses on the following issues when constructing the system evaluation device:

[0160] The need for a multi-layer indicator system: A multi-layer indicator system is an evaluation framework that breaks down a complex system into multiple levels and aspects. It allows evaluators to evaluate the system from multiple dimensions and angles, thereby gaining a more comprehensive understanding of the system's performance and characteristics. However, how to build a reasonable and effective multi-layer indicator system and how to use this system for accurate and reliable evaluation has become a problem that needs to be solved.

[0161] Importance of standard data: In the evaluation of multi-layer indicator system, the accuracy and reliability of data are crucial to the quality of evaluation results. Standard data refers to data that has been strictly defined, measured and verified, and they have high accuracy and reliability. Using standard data for evaluation can greatly improve the accuracy and credibility of evaluation results.

[0162] Application of models based on standard data: Models based on standard data are an evaluation method that combines standard data with a multi-layer indicator system. This method uses standard data to quantify evaluation indicators and conducts a comprehensive analysis of the indicators through mathematical models to obtain a comprehensive evaluation result of the system. This method can not only improve the accuracy and reliability of the evaluation, but also realize an automated and efficient evaluation process.

[0163] The multi-layer index system evaluation device based on the standard data model provided by the present invention includes:

[0164] The indicator evaluation system construction method is used to build a standard-based indicator system that supports multi-level construction;

[0165] The indicator weight configuration method is used to configure the weight of each terminal node of the indicator system;

[0166] The indicator evaluation model construction method is used to construct the logical method required for indicator calculation. The evaluation model can be constructed externally, so it supports different types of models such as standard calculation model and expert review model. It also supports manual input of quantity values ​​and the connection of external system data.

[0167] The indicator calculation node model binding method flexibly binds each final-level indicator to the indicator evaluation model built based on standard data;

[0168] The indicator system evaluation method is used to evaluate the quality of the entire indicator system. This technical solution solves the problems existing in the current indicator system in paperless evaluation.

[0169] The method for constructing the evaluation index system includes selecting an index level, wherein the index level supports multi-level construction.

[0170] Indicator weight configuration method, used to configure the weight of each node of the indicator body;

[0171] The indicator evaluation model construction method is used to construct the logical method required for indicator calculation;

[0172] The indicator evaluation model construction method includes construction based on reference standard text, construction based on reference standard data, and construction based on existing mature models.

[0173] The standard data supports direct model assignment and third-party interface.

[0174] The mature models include grey correlation analysis, DEA evaluation system, fuzzy mathematical evaluation model, hybrid model, economic evaluation model, journey model, engineering model and the like.

[0175] The indicator calculation node model binding method is used to establish an association relationship between the indicator calculation node and the evaluation model.

[0176] The indicator system evaluation method is used to evaluate the quality of the entire indicator system.

[0177] The method for constructing the indicator evaluation system includes entering the system name, indicator system level, system description, creating new indicators, and importing indicators through third-party connection; the new indicators include indicator name, indicator level, parent indicator, and indicator description, and the indicators imported through third-party connection include template downloading, template data filling, and template data importing.

[0178] The indicator weight configuration method includes equal configuration and custom configuration.

[0179] The indicator evaluation model construction method includes construction based on standard text and construction based on standard data.

[0180] The index calculation node model binding method includes establishing an association relationship between the index evaluation model and the calculation node.

[0181] The indicator system evaluation method includes calculating the indicator system evaluation score according to the indicator weight and the indicator model calculation result.

[0182] The multi-layer indicator system evaluation device is based on standard models and construction technology. During the construction of the multi-layer indicator system, weights must be given to each node. The terminal nodes of the indicator system will be bound to the evaluation model or filled with data from an external system. The evaluation model can be constructed by the user. Finally, the device obtains the evaluation system results based on the weights and bound models or data from external systems under the calculation of the evaluation algorithm.

[0183] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A standard model-based security system evaluation method, characterized in that: Used to evaluate the performance of the support system, including: S1, obtaining a system performance measurement data set; the system performance measurement data set includes a system operation monitoring data subset, a system fault data subset, an environmental data subset, a human factors data subset, a system development data subset, a system use and maintenance data subset, and a system daily management data subset; S2, performing preprocessing and data mining on the system performance measurement data set to obtain a data set to be evaluated; S3, using a standard model set to evaluate the data set to be evaluated, and obtain an information performance evaluation result information set; the information performance evaluation result information set includes overall performance evaluation result information, system reliability evaluation result information and system quality evaluation result information; the standard model set includes an overall evaluation model, a reliability evaluation model and a quality evaluation model; The preprocessing and data mining of the system performance measurement data to obtain a data set to be evaluated includes: S21, performing a preprocessing operation on each data subset in the system performance measurement data set to obtain a preprocessed data set; S22, performing data mining processing on the preprocessed data set to obtain a mining data set; S23, determining the preprocessed data set and the mining data set as the data set to be evaluated; The preprocessing operation is performed on each data subset in the system performance measurement data set to obtain a preprocessed data set, including: S211, performing data cleaning processing on the system performance measurement data set to obtain a cleaned data set; S212, performing unified processing of collected information on the cleaned data set to obtain a standardized data set; S213, performing boundary check and category consistency check processing on the standard data set to obtain a preprocessed data set; The method of using the standard model set to evaluate the data set to be evaluated to obtain an information performance evaluation result information set includes: Using the overall evaluation model, the preprocessed data set is processed to obtain overall performance evaluation result information; the overall performance evaluation result information is used to evaluate the overall performance of the security system; Using a reliability evaluation model, the system reliability measurement data set is processed to obtain system reliability evaluation result information; the system reliability evaluation result information is used to characterize the reliability of the support system; Using the quality assessment model, the system quality assessment data set is processed to obtain system quality assessment result information; the system quality assessment result information is used to characterize the quality of the security system; The overall evaluation model is used to process the preprocessed data set to obtain overall performance evaluation result information, including: Get the standard values ​​of all data attributes; Using the standard values ​​of all data attributes, a standard vector is constructed; For all measurement data of each data attribute in the preprocessed data set, construct a measurement sequence of the data attribute; For each data attribute’s measurement sequence, subtract the corresponding standard value to obtain the difference measurement sequence of each data attribute; A first measurement matrix is ​​constructed using the difference measurement sequences of all data attributes; a row vector of the first measurement matrix is ​​a difference measurement sequence of a data attribute; a column vector of the first measurement matrix represents measurement data of all data attributes collected at the same time; Performing polynomial extraction processing on the first measurement matrix to obtain a characteristic polynomial; Performing difference feature calculations on all column vectors of the first measurement matrix and the standard vector respectively to obtain difference vectors; The expression for calculating the difference feature is: Among them, p j represents the jth term of the difference vector, A ij represents the element in the i-th row and j-th column of the first measurement matrix, b i represents the i-th element of the standard vector, M represents the row dimension of the first measurement matrix, α1, α2, α3 are preset weighting coefficients, b represents the standard vector, A j represents the j-th column of the first measurement matrix; Determine the sequence number of the element with the largest value of the difference vector as the feature sequence number; Substitute the characteristic sequence number into the characteristic polynomial to calculate and obtain the overall performance evaluation result information.

2. The standard model-based security system evaluation method according to claim 1, characterized in that: The step of uniformly processing the collected information of the cleaned data set to obtain a standardized data set includes: S2121, setting a unified data collection time and a unified data collection location for the measurement data of the same data attribute contained in the same data subset of the cleaned data set; S2122, for the measurement data of the same data attribute contained in each data subset, aligning the collection time and collection location of the measurement data according to the unified data collection time and the unified data collection location to obtain standardized measurement data; and updating each data subset using the standardized measurement data; S2123, using all updated data subsets, construct a standard data set; The performing boundary checking and category consistency checking on the standard data set to obtain a consistent data set includes: S2131, for each data attribute in each standard data subset of the standard data set, preset a corresponding value range; the value range includes an upper limit value of the value range and a lower limit value of the value range; S2132, for each measurement data of each standard data subset of the standard data set, judging whether the measurement data value is within the value range according to the value range of its data attribute; when the measurement data is within the value range, not processing the measurement data; when the measurement data is not within the value range, setting the measurement data value to the value range upper limit value or value range lower limit value of the value range closest to the measurement data; S2133, for each measurement data in the standard data subset in the standard data set, determine whether its data category belongs to a preset category range of the standard data subset, and delete the measurement data whose data category does not belong to the preset category range of the standard data subset from the standard data subset; S2134, execute S2132 and S2133 on each standard data subset of the standard data set to obtain a preprocessed data set.

3. The standard model-based security system evaluation method according to claim 1, characterized in that: The performing data mining on the preprocessed data set to obtain a mining data set includes: Performing reliability mining processing on the system operation monitoring data subset, the system fault data subset, and the environment data subset to obtain a system reliability measurement data set; Sequence mining is performed on the human factor data subset, the system development data subset, the system use and maintenance data subset, and the system daily management data subset to obtain a system quality assessment data set.

4. A security system evaluation device based on a standard model, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the standard model-based assurance system assessment method according to any one of claims 1 to 3.

5. A computer storable medium, characterized in that: The computer storable medium stores computer instructions, and when the computer instructions are called, they are used to execute the standard model-based assurance system assessment method according to any one of claims 1 to 3.

6. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the standard model-based security system evaluation method as described in any one of claims 1 to 3.

Images