Network security capability evaluation method, device, equipment, medium and program product

Abstract

The embodiment of the application provides a network security capability evaluation method, device, equipment, medium and program product, relates to the network and information security technical field, and the method comprises the following steps: acquiring multi-period observation data and multi-source evidence data, adaptively updating index weights based on the observation data, dynamically correcting the weights with the security situation, and overcoming the evaluation lag caused by the static weights; meanwhile, an evidence matrix is constructed, a first evidence value representing strong evidence and a second evidence value representing weak evidence are distinguished, different intensities of evidence are subjected to differential influence, and low-frequency key signals are retained, so that the dynamic weight updating and the differential evidence fusion are cooperated with each other, the sensitivity and the distinguishability of the evaluation result to the change of the security capability are improved from two dimensions of time-varying adaptability and contribution degree, and the objectivity and the reliability of the evaluation system are enhanced.

Description

Technical Field

[0001] This invention relates to the field of network and information security technology, and in particular to a network security capability evaluation method, a network security capability evaluation device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Cybersecurity capability evaluation is a crucial basis for enterprises to understand their own security operation level and guide their security construction investment. In practical applications, the evaluation process requires comprehensive information from multiple dimensions, including management behavior, operational behavior, technical protection data, security logs, vulnerability handling records, and attack and defense drill results. By setting evaluation indicators and assigning corresponding weights, a quantitative assessment of overall security capabilities is ultimately formed. Currently, the industry commonly uses classic methods such as the Analytic Hierarchy Process (AHP) and fuzzy comprehensive evaluation to construct evaluation systems. Experts pre-determine the weight values ​​of each indicator, and then combine this with collected operational data for scoring calculations.

[0003] However, in the above process, firstly, the indicator weights in the relevant evaluation system remain fixed once manually set, and cannot be adaptively adjusted based on the positive and negative results data continuously accumulated during actual security operations. When the security situation changes or operational strategies are adjusted, the static weights are difficult to reflect the true contribution of each indicator to the current security capability, resulting in a lag between the evaluation results and the actual security operation status. Secondly, when faced with heterogeneous behavioral data from multiple channels and of various types, the relevant evaluation methods adopt a unified fusion strategy, making it difficult to adaptively distinguish and weight data according to the strength or importance of evidence. This causes low-frequency but critically indicative security event signals to be easily drowned out, resulting in insufficient sensitivity and discriminative power of the evaluation results to changes in actual security capabilities. Summary of the Invention

[0004] This invention provides a method, apparatus, device, medium, and program product for evaluating cybersecurity capabilities, in order to solve or partially solve the problems that the weights of indicators in the cybersecurity capability evaluation process cannot be adaptively updated with actual operational data, and that the lack of differentiated processing of weak signals and key evidence during multi-source evidence fusion leads to lagging evaluation results and insufficient discrimination.

[0005] This invention discloses a method for evaluating network security capabilities, comprising: Obtain multi-period observation data corresponding to multiple evaluation indicators, as well as evidence data corresponding to each evaluation indicator; Based on the multi-period observation data, update the weight values ​​of the evaluation indicators in the evaluation system; An evidence matrix containing the evidence data is constructed, and a first evidence value and a second evidence value for each of the evaluation indicators in the evidence matrix are determined. A weighted fusion calculation is performed based on the weight value, the first evidence value, and the second evidence value to obtain the security capability score of the evaluation indicator. Based on the security capability scores of each of the aforementioned evaluation indicators, a cybersecurity capability evaluation result is generated.

[0006] In some feasible implementations, the multi-period observation data includes the number of historical positive results and the number of historical negative results for each evaluation indicator in multiple historical periods. Updating the weight values ​​of the evaluation indicators in the evaluation system based on the multi-period observation data includes: Based on the number of historical positive results and the number of historical negative results, the prior probability distribution parameters of each evaluation index are determined; Collect the number of positive and negative results of each evaluation indicator within the current evaluation period, and use this data as the observation data for the current period. Generate a posterior probability distribution based on the prior probability distribution parameters and the current periodic observation data; The weight values ​​of each evaluation index are determined based on the characteristic values ​​of the posterior probability distribution.

[0007] In some feasible implementations, determining the prior probability distribution parameters of each evaluation index based on the number of historical positive results and the number of historical negative results includes: The sum of the number of positive results of each evaluation indicator in multiple historical periods is taken as the prior positive count, and the sum of the number of negative results of each evaluation indicator in multiple historical periods is taken as the prior negative count. The prior positive counts and the prior negative counts are combined to form the prior probability distribution parameters of each evaluation index.

[0008] In some feasible implementations, generating a posterior probability distribution based on the prior probability distribution parameters and the current periodic observation data includes: The sum of the prior positive count and the target positive result count collected in the current period is taken as the posterior positive count, and the sum of the prior negative count and the negative result count collected in the current period is taken as the posterior negative count. The positive posterior counts and the negative posterior counts are combined to form the corresponding posterior probability distribution.

[0009] In some feasible implementations, determining the weight values ​​of each evaluation index based on the feature values ​​of the posterior probability distribution includes: Calculate the expected value corresponding to the posterior probability distribution; The expected value is used as the weight value for each of the evaluation indicators.

[0010] In some feasible implementations, the evidence data includes at least two of the following: management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise result data.

[0011] In some feasible implementations, constructing the evidence matrix containing the evidence data includes: The source of the evidence data obtained; Obtain the quantitative value of the evaluation index under the evidence source; An evidence matrix is ​​constructed from the quantitative values ​​of each evaluation indicator under the aforementioned evidence source. In this matrix, each row corresponds to one of the evaluation indicators, and each column corresponds to one of the sources of evidence.

[0012] In some feasible implementations, determining the first evidence value and the second evidence value of each of the evaluation indicators in the evidence matrix includes: Extract the evidence vector corresponding to each evaluation indicator from the evidence matrix, wherein the evidence vector contains the quantitative value of the evaluation indicator under the evidence source; The maximum value in the evidence vector is taken as the first evidence value, and the minimum value in the evidence vector is taken as the second evidence value.

[0013] In some feasible implementations, the step of performing a weighted fusion calculation based on the weight value, the first evidence value, and the second evidence value to obtain the security capability score of the evaluation index includes: Obtain the first adjustment parameter and the second adjustment parameter; Calculate the product of the first power of the first adjustment parameter of the first evidence value and the second power of the second adjustment parameter of the second evidence value; Multiplying the product by the weight value corresponding to the evaluation index yields the safety capability score of the evaluation index.

[0014] In some feasible implementations, obtaining the first adjustment parameter and the second adjustment parameter includes: Extract the feature parameters of the evidence matrix, wherein the feature parameters include at least one of evidence conflict degree, evidence coverage and data freshness; A first initial value is determined based on the degree of evidence conflict, and the first initial value is negatively correlated with the degree of evidence conflict; A second initial value is determined based on the evidence coverage rate and the data freshness, wherein the second initial value is positively correlated with the evidence coverage rate and the data freshness; The first initial value is used as the first adjustment parameter, and the second initial value is used as the second adjustment parameter.

[0015] In some feasible implementations, determining the first initial value based on the degree of evidence conflict includes: Calculate the sum between the first preset value and the degree of evidence conflict, and use the reciprocal of the sum as the first initial value; The step of determining the second initial value based on the evidence coverage and the data freshness includes: Calculate the exponent of the product between the preset decay coefficient (base of the natural constant and negative exponent) and the data freshness, and use the product of the evidence coverage and the exponent as the second initial value.

[0016] In some feasible implementations, using the first initial value as the first adjustment parameter and the second initial value as the second adjustment parameter includes: Using the aforementioned feature parameters as input features, the first initial value and the second initial value are fine-tuned, and the fine-tuned first initial value is used as the first adjustment parameter, and the fine-tuned second initial value is used as the second adjustment parameter.

[0017] In some feasible implementations, the fine-tuning of the first initial value and the second initial value, using the fine-tuned first initial value as a first adjustment parameter and the fine-tuned second initial value as a second adjustment parameter, includes: The feature parameters in the historical evaluation period are used as training samples, and the adjustment parameters corresponding to the manually labeled adjustment parameters or standard evaluation results in the historical evaluation period are used as labels for training to obtain a trained machine learning model. The feature parameters of the current evaluation period are input into the trained machine learning model for adjustment to obtain a first adjustment amount for the first initial value and a second adjustment amount for the second initial value. The sum of the first initial value and the first adjustment amount is used as the first adjustment parameter, and the sum of the second initial value and the second adjustment amount is used as the second adjustment parameter.

[0018] This invention also discloses a network security capability evaluation device, comprising: The data acquisition module is used to acquire multi-period observation data corresponding to multiple evaluation indicators, as well as evidence data corresponding to each evaluation indicator. The weight update module is used to update the weight value of the evaluation index in the evaluation system based on the multi-period observation data. The evaluation module is used to construct an evidence matrix containing the evidence data, determine the first evidence value and the second evidence value of each evaluation indicator in the evidence matrix, and perform a weighted fusion calculation based on the weight value, the first evidence value and the second evidence value to obtain the security capability score of the evaluation indicator. The result generation module is used to generate network security capability evaluation results based on the security capability scores of each of the evaluation indicators.

[0019] In some feasible implementations, the multi-period observation data includes the number of historical positive results and the number of historical negative results for each evaluation indicator in multiple historical periods, and the weight update module is specifically used for: Based on the number of historical positive results and the number of historical negative results, the prior probability distribution parameters of each evaluation index are determined; Collect the number of positive and negative results of each evaluation indicator within the current evaluation period, and use this data as the observation data for the current period. Generate a posterior probability distribution based on the prior probability distribution parameters and the current periodic observation data; The weight values ​​of each evaluation index are determined based on the characteristic values ​​of the posterior probability distribution.

[0020] In some feasible implementations, the weight update module is specifically used for: The sum of the number of positive results of each evaluation indicator in multiple historical periods is taken as the prior positive count, and the sum of the number of negative results of each evaluation indicator in multiple historical periods is taken as the prior negative count. The prior positive counts and the prior negative counts are combined to form the prior probability distribution parameters of each evaluation index.

[0021] In some feasible implementations, the weight update module is specifically used for: The sum of the prior positive count and the target positive result count collected in the current period is taken as the posterior positive count, and the sum of the prior negative count and the negative result count collected in the current period is taken as the posterior negative count. The positive posterior counts and the negative posterior counts are combined to form the corresponding posterior probability distribution.

[0022] In some feasible implementations, the weight update module is specifically used for: Calculate the expected value corresponding to the posterior probability distribution; The expected value is used as the weight value for each of the evaluation indicators.

[0023] In some feasible implementations, the evidence data includes at least two of the following: management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise result data.

[0024] In some feasible implementations, the evaluation module is specifically used for: The source of the evidence data obtained; Obtain the quantitative value of the evaluation index under the evidence source; An evidence matrix is ​​constructed from the quantitative values ​​of each evaluation indicator under the aforementioned evidence source. In this matrix, each row corresponds to one of the evaluation indicators, and each column corresponds to one of the sources of evidence.

[0025] In some feasible implementations, the evaluation module is specifically used for: Extract the evidence vector corresponding to each evaluation indicator from the evidence matrix, wherein the evidence vector contains the quantitative value of the evaluation indicator under the evidence source; The maximum value in the evidence vector is taken as the first evidence value, and the minimum value in the evidence vector is taken as the second evidence value.

[0026] In some feasible implementations, the evaluation module is specifically used for: Obtain the first adjustment parameter and the second adjustment parameter; Calculate the product of the first power of the first adjustment parameter of the first evidence value and the second power of the second adjustment parameter of the second evidence value; Multiplying the product by the weight value corresponding to the evaluation index yields the safety capability score of the evaluation index.

[0027] In some feasible implementations, the evaluation module is specifically used for: Extract the feature parameters of the evidence matrix, wherein the feature parameters include at least one of evidence conflict degree, evidence coverage and data freshness; A first initial value is determined based on the degree of evidence conflict, and the first initial value is negatively correlated with the degree of evidence conflict; A second initial value is determined based on the evidence coverage rate and the data freshness, wherein the second initial value is positively correlated with the evidence coverage rate and the data freshness; The first initial value is used as the first adjustment parameter, and the second initial value is used as the second adjustment parameter.

[0028] In some feasible implementations, the evaluation module is specifically used for: Calculate the sum between the first preset value and the degree of evidence conflict, and use the reciprocal of the sum as the first initial value; The step of determining the second initial value based on the evidence coverage and the data freshness includes: Calculate the exponent of the product between the preset decay coefficient (base of the natural constant and negative exponent) and the data freshness, and use the product of the evidence coverage and the exponent as the second initial value.

[0029] In some feasible implementations, the evaluation module is specifically used for: Using the aforementioned feature parameters as input features, the first initial value and the second initial value are fine-tuned, and the fine-tuned first initial value is used as the first adjustment parameter, and the fine-tuned second initial value is used as the second adjustment parameter.

[0030] In some feasible implementations, the evaluation module is specifically used for: The feature parameters in the historical evaluation period are used as training samples, and the adjustment parameters corresponding to the manually labeled adjustment parameters or standard evaluation results in the historical evaluation period are used as labels for training to obtain a trained machine learning model. The feature parameters of the current evaluation period are input into the trained machine learning model for adjustment to obtain a first adjustment amount for the first initial value and a second adjustment amount for the second initial value. The sum of the first initial value and the first adjustment amount is used as the first adjustment parameter, and the sum of the second initial value and the second adjustment amount is used as the second adjustment parameter.

[0031] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0032] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0033] This invention also discloses a computer program product, characterized in that it includes computer execution instructions, which, when executed by a processor, implement the method described in this invention.

[0034] The embodiments of the present invention have the following advantages: In this embodiment of the invention, multi-period observation data corresponding to multiple evaluation indicators and evidence data corresponding to each evaluation indicator are acquired. Then, the weight values ​​of each evaluation indicator in the evaluation system are updated based on the multi-period observation data. Next, an evidence matrix containing evidence data is constructed, and the first and second evidence values ​​of each evaluation indicator in the evidence matrix are determined. A weighted fusion calculation is performed based on the weight values, the first evidence value, and the second evidence value to obtain the security capability score for each evaluation indicator. Finally, a network security capability evaluation result is generated based on each security capability score. Thus, by dynamically associating the weight settings of the evaluation indicators with the multi-period observation data continuously accumulated during actual security operations, the weight values ​​can be adjusted according to the security state. The system adaptively adjusts to changes in the situation, overcoming the shortcomings of traditional static weights that cause evaluation results to lag behind actual security conditions. Furthermore, by constructing an evidence matrix and distinguishing between first evidence values ​​representing strong evidence and second evidence values ​​representing weak evidence, it applies differentiated influences to evidence of different strengths during fusion calculations. This ensures that low-frequency but critically indicative security event signals are effectively preserved rather than drowned out. At the same time, dynamic weight updates and differentiated evidence fusion work together to improve the sensitivity and discriminativeness of evaluation results to changes in actual security capabilities from two dimensions: the time-varying adaptability of indicator importance and the contribution of multi-source evidence. This enhances the objectivity of the evaluation system and the credibility of the evaluation results. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the steps of a network security capability evaluation method provided in this embodiment of the invention. Figure 2 This is a system architecture diagram of the evaluation system provided in the embodiments of the present invention; Figure 3 This is a structural block diagram of a network security capability evaluation device provided in an embodiment of the present invention. Detailed Implementation

[0036] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] As examples, large enterprise groups typically establish routine security capability evaluation systems in their cybersecurity operations management. These systems periodically collect and comprehensively score security operation indicators from subordinate units to form a quantitative assessment of the group's overall cybersecurity protection capabilities. The evaluation results are widely used in typical scenarios such as monthly security posture analysis, performance appraisal, attack and defense drill debriefing, and third-party compliance assessments. These evaluation systems usually employ classic multi-indicator comprehensive evaluation methods such as the analytic hierarchy process (AHP), fuzzy comprehensive evaluation, or weighted summation. Security experts pre-set fixed weights for each evaluation indicator within the system based on industry standards or management requirements, and these weights remain unchanged over a relatively long period.

[0038] However, the actual operating environment of cybersecurity is constantly changing. The frequency of security incidents, the evolution of attack methods, and the effectiveness of defensive measures all fluctuate significantly over time. Fixed weights cannot reflect the impact of this time-varying characteristic on the importance of various indicators, leading to a deviation between the evaluation results and the actual security situation. This deviation will intensify over time. Furthermore, cybersecurity capability evaluation involves a wide range of data sources, including management system process records, technical equipment security logs, vulnerability handling work orders, and attack and defense exercise test results. These data vary significantly in format, semantics, and credibility, making it difficult to directly perform unified quantitative comparisons and comprehensive calculations. Existing evaluation methods, when dealing with such multi-source heterogeneous data, typically rely on the subjective experience of technical personnel to set fusion strategies and adjust parameters. When the data distribution characteristics change, existing parameters often need to be manually readjusted, which is not only inefficient but also makes it difficult to guarantee the consistency and reproducibility of evaluation results due to differences in the experience of different personnel.

[0039] In this invention, based on the core mechanism of dynamic weight adaptive update and multi-source evidence differential fusion, when the network security management system initiates an evaluation request, the evaluation system can obtain multi-period observation data of multiple evaluation indicators and evidence data from multiple data sources corresponding to each evaluation indicator from the target network security management system. The weight values ​​of each evaluation indicator are adaptively updated according to the multi-period observation data, and the strongest and weakest evidence values ​​of each evaluation indicator under multiple evidence sources are distinguished by constructing an evidence matrix. The weight values ​​and differential evidence values ​​are weighted and fused to calculate the security capability score of each indicator, and finally, the network security capability evaluation result is generated. This allows the evaluation system to dynamically evolve with the accumulation of security operation data and to effectively fuse multi-source heterogeneous evidence in a differentiated manner.

[0040] Reference Figure 1 The diagram illustrates a flowchart of a network security capability evaluation method provided in an embodiment of the present invention, which may specifically include the following steps: Step 101: Obtain multi-period observation data corresponding to multiple evaluation indicators, and evidence data corresponding to each evaluation indicator; Optionally, embodiments of the present invention can be applied to a network security capability evaluation system, referred to as the evaluation system in the following embodiments. This evaluation system can be deployed within the network security operation and management platform of an enterprise group, and connect with the enterprise group's comprehensive network security dispatch platform system for data exchange. The comprehensive network security dispatch platform system is a centralized platform used by the enterprise group to uniformly manage the network security work of its subordinate units. This platform system stores multi-level network security indicators and structured data, including security operation indicator values ​​regularly reported by each subordinate unit, security incident records, vulnerability handling work order flow information, and attack and defense exercise test results. Accordingly, the evaluation system can collect the required basic evaluation data from the platform system through the application programming interface or data export interface provided by the platform. The evaluation system can be deployed independently as a dedicated evaluation service module, or integrated into the group's security situation awareness platform or security management center, operating as a sub-functional module. Furthermore, the evaluation system may include a data acquisition unit, a data processing unit, a weight update unit, a fusion calculation unit, and a result output unit, respectively used to perform operations such as data acquisition, weight calculation, evidence fusion, and result generation.

[0041] In this embodiment of the invention, the evaluation system can first obtain the basic data required for the evaluation from the target network security management system. The target network security management system can be a comprehensive network security dispatch platform system of an enterprise group. This platform system stores multi-level network security indicators and structured data, including security operation indicator values ​​regularly reported by subordinate units, security incident records, vulnerability handling work order flow information, and attack and defense exercise test results. Based on this, the evaluation system can obtain multi-period observation data corresponding to multiple evaluation indicators through the application programming interface or data export interface provided by the platform.

[0042] The evaluation indicators can be quantitative metrics used to measure cybersecurity operational capabilities. These can be directly collected from the group's integrated cybersecurity dispatch platform system, such as the number of successfully intercepted attacks, the timeliness of emergency response, the number of exposure surfaces converged, the number of vulnerabilities discovered and patched, and the number of threat intelligence reports. The construction of these evaluation indicator systems can be predetermined with reference to the security operation management requirements issued by the group company and relevant industry standards, and the composition and number of indicators can be adjusted in a timely manner according to actual operational conditions.

[0043] Accordingly, multi-period observation data can be the statistical data accumulated by the evaluation indicators over multiple historical evaluation periods, including the number of positive and negative results for each evaluation indicator in each historical period. Specifically, the number of positive results represents the number of times the evaluation indicator exhibits a positive security effect within the evaluation period, i.e., results that are beneficial to network security defense and meet the expected security objectives; the number of negative results represents the number of times the evaluation indicator exhibits a negative security effect within the evaluation period, i.e., results that are detrimental to network security defense and deviate from the expected security objectives.

[0044] Taking the number of successfully intercepted attacks as an example, this indicator can be used to measure the ability of security protection devices to intercept attack behavior. The number of positive results can represent the number of attacks that were successfully intercepted, while the number of negative results can represent the number of times the attacked party broke through the boundary and failed to be intercepted.

[0045] Taking the number of vulnerabilities discovered and fixed as an example, the number of positive results can represent the number of vulnerabilities that were fixed within the specified time limit, while the number of negative results can represent the number of vulnerabilities that were not fixed after the time limit.

[0046] Taking the number of converged exposure surfaces as an example, the number of positive results can represent the number of Internet exposure surfaces that were actively identified and successfully converged, while the number of negative results can represent the number of newly added or failed-to-converge exposure surfaces.

[0047] Taking the timeliness of emergency response as an example, the number of positive results can represent the number of times the incident was handled within the time limit stipulated in the emergency response, while the number of negative results can represent the number of times the incident was handled beyond the time limit or was not handled at all.

[0048] Taking the number of threat intelligence reports as an example, the number of positive results can represent the number of intelligence reports that have been verified as valid and adopted by higher-level units, while the number of negative results can represent the number of intelligence reports that have been verified as invalid or of low quality.

[0049] Furthermore, the evaluation system can simultaneously acquire evidence data corresponding to each evaluation indicator. This evidence data can be raw behavioral data from multiple data sources related to each indicator, including management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise results data. The evidence data can originate from heterogeneous data records generated at different stages of the enterprise group's cybersecurity operations. For example, taking the emergency response capability evaluation indicator as an example, its evidence data can simultaneously come from multiple sources, such as the handling time records in the emergency response process management system, the interception success rate logs in security devices, and the measured response speed data in the attack and defense exercise platform.

[0050] It should be noted that the difference between evidence data and multi-period observation data is that multi-period observation data is a statistical data of the number of positive and negative results of the evaluation index in each period, which is a value after preliminary aggregation; while evidence data is a more granular original behavior record, which contains detailed data from various evidence sources that have not been aggregated, such as the specific score of a certain attack and defense exercise, the detailed processing time of a certain emergency response, etc. This invention does not limit this.

[0051] In some feasible implementations, multi-period observation data can specifically include the number of historical positive results and the number of historical negative results for each evaluation indicator across multiple historical periods. Specifically, when the evaluation system acquires multi-period observation data, it needs to determine the time granularity of the evaluation period, which can be monthly, quarterly, or a time interval customized according to management needs.

[0052] Taking monthly evaluation as an example, the evaluation system can obtain the number of positive and negative results for each evaluation indicator over the past several calendar months from the platform system. The number of historical periods can be set according to data accumulation and stability requirements; for example, historical data from the past 6 or 12 months can be obtained as multi-period observation data. For each evaluation indicator, there will be a corresponding number of positive and negative results in each historical period, forming a positive and negative sequence for that indicator across multiple periods.

[0053] Accordingly, taking the vulnerability remediation timeliness rate as an example, if the number of historical periods is 3, the evaluation system will obtain the number of remediated and unremediated vulnerabilities in the first historical period, the number of remediated and unremediated vulnerabilities in the second historical period, and the number of remediated and unremediated vulnerabilities in the third historical period, for a total of 6 values ​​as multi-period observation data for this indicator.

[0054] In some feasible implementation methods, the evidence data specifically includes at least two of the following: management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise results data.

[0055] Among them, management behavior data can be process data generated by enterprise groups in cybersecurity management activities, such as security system implementation inspection records, security training assessment results, and the performance of security personnel. This data usually comes from the process approval records and system implementation inspection forms of the enterprise's office automation system or special security management system.

[0056] Correspondingly, operational behavior data can be operation records generated by the security operations center in daily security operations and maintenance activities, such as security inspection records, security incident work order processing records, and daily security hardening operation records. These data usually come from the security operations work order system and inspection management platform.

[0057] Correspondingly, technical protection data can be the protection effect data generated during the operation of various security technical protection devices, such as firewall interception logs, intrusion detection system alarm records, and endpoint protection software security event reporting records. These data usually come from the event log export interface of security devices.

[0058] Correspondingly, security log data can be the underlying security audit logs generated by the operating system, applications, and network devices themselves, such as system login logs, database access logs, and network connection logs. These data usually come from the log collection and aggregation platforms of various information systems.

[0059] Correspondingly, vulnerability handling data can be the full lifecycle data of vulnerability discovery, verification, repair and retesting recorded in the vulnerability management platform, such as vulnerability submission time, vulnerability verification results, vulnerability repair completion time, and post-repair retesting results. This data usually comes from the enterprise vulnerability management platform.

[0060] Correspondingly, the data from offensive and defensive drills can be the evaluation results generated in regularly conducted live-fire offensive and defensive drills, such as weaknesses found in the drills, the defender's response speed score, and the attack path identification status. This data usually comes from the offensive and defensive drill evaluation report system.

[0061] The above-mentioned types of evidence data may differ significantly in format. Management behavior data and operational behavior data can be presented in a highly structured process form format, technical protection data and security log data can be presented in a semi-structured or unstructured log text format, vulnerability handling data can be presented in a structured record format of a work order system, and attack and defense exercise result data can be presented in an evaluation report format. This invention does not impose any restrictions on these.

[0062] Step 102: Update the weight values ​​of the evaluation indicators in the evaluation system based on the multi-period observation data; In this embodiment of the invention, after acquiring multi-period observation data for each evaluation indicator, the evaluation system can adaptively update the weight values ​​of each evaluation indicator in the evaluation system using this data. The weight value can be a coefficient representing the importance of each evaluation indicator in the comprehensive evaluation system, used to measure the contribution of that indicator to the overall safety capability evaluation result.

[0063] Optionally, the evaluation system can employ a dynamic update mechanism based on Bayesian theory. This mechanism uses the accumulated positive and negative results of each evaluation indicator over historical periods as prior knowledge, and the positive and negative results collected in the current evaluation period as new observation data. Through Bayesian inference, the prior knowledge and new observation data are fused to generate posterior knowledge, and the weight of the indicator in the current evaluation period is determined based on this posterior knowledge. In this process, the proportion of positive results exhibited by each evaluation indicator in network security defense can be considered an observation of the indicator's true capability level. This observation fluctuates with changes in the security situation. By fusing the historically accumulated prior distribution with the observation data of the current period, a more accurate estimate of the indicator's current true capability level can be formed, allowing the weight values ​​to be adaptively updated in response to changes in the security situation.

[0064] In some feasible implementations, as described in the above embodiments, the multi-period observation data includes the number of historical positive results and the number of historical negative results for each evaluation indicator in multiple historical periods. In the process of updating the weight values, the prior probability distribution parameters of each evaluation indicator can be determined first based on the number of historical positive results and the number of historical negative results. Then, the number of target positive results and the number of target negative results for each evaluation indicator in the current evaluation period are collected as the current period observation data. Based on the prior probability distribution parameters and the current period observation data, a posterior probability distribution is generated. Finally, the weight values ​​of each evaluation indicator are determined based on the eigenvalues ​​of the posterior probability distribution.

[0065] The prior probability distribution parameter is used to quantify the prior understanding of the importance of each evaluation indicator based on historical data before the current period's data is observed. Optionally, the evaluation system can use the sum of the number of positive results of each evaluation indicator in multiple historical periods as the prior positive count, and the sum of the number of negative results of each evaluation indicator in multiple historical periods as the prior negative count. Then, the prior positive count and the prior negative count are combined to form the prior probability distribution parameter of each evaluation indicator.

[0066] For example, taking a certain evaluation index as an example, if the number of positive results in the past three historical periods are 45, 52, and 48, and the number of negative results are 5, 3, and 7, respectively, then the prior positive count is 45 + 52 + 48 = 145, and the prior negative count is 5 + 3 + 7 = 15. Then, the prior probability distribution parameters of each evaluation index are obtained by combining the prior positive count and the prior negative count.

[0067] Furthermore, the evaluation system collects the number of positive and negative outcomes for each evaluation indicator within the current evaluation period as the current period's observation data. Next, the sum of the prior positive outcomes and the current period's collected positive outcomes is used as the posterior positive outcome count, and the sum of the prior negative outcomes and the current period's collected negative outcomes is used as the posterior negative outcome count. The posterior positive and negative outcomes are then combined to form the corresponding posterior probability distribution. Thus, within the Bayesian inference framework, when the likelihood functions of the prior distribution and the observed data belong to the conjugate distribution family, the posterior distribution and the prior distribution have the same mathematical form, significantly simplifying the calculation process. Finally, the evaluation system can calculate the expected value corresponding to the posterior probability distribution and then use the expected value as the weight value for each evaluation indicator.

[0068] Through the above mechanism, when the security situation changes or the operation strategy is adjusted, the weight values ​​of each evaluation indicator can be adaptively corrected according to the positive and negative result data continuously accumulated in the actual security operation process, reflecting the true contribution of each indicator to the current security capability, thereby overcoming the lag deviation between the evaluation results and the actual security operation status caused by static weights.

[0069] In some examples, the process of updating the weight values ​​described above can be performed using a probability distribution model. This probability distribution model can be a Beta distribution, which is a continuous probability distribution defined on the interval [0,1] and determined by two shape parameters α and β. Its probability density function can be expressed as f(x;α,β)=x^(α-1)(1-x)^(β-1) / B(α,β), where B(α,β) is the Beta function.

[0070] It should be noted that the Beta distribution, as the conjugate prior distribution of the binomial distribution, means that when the prior distribution is a Beta distribution and the observed data follows a binomial distribution, the posterior distribution is also a Beta distribution. The shape parameters of the posterior distribution can be obtained by simply adding the shape parameters of the prior distribution to the number of successes and failures in the observed data. In this embodiment of the invention, the two shape parameters of the Beta distribution are the aforementioned prior positive and negative counts, respectively.

[0071] In practical implementation, the evaluation system can maintain a prior Beta distribution Beta(a0) for each evaluation index. i ,b0 i ), where a0 i b0 can be the prior positive count of the i-th evaluation index. iThis can be the prior negative count for the i-th evaluation indicator. Correspondingly, the prior positive and negative counts can be determined based on the sum of the historical positive and negative counts of this evaluation indicator across multiple historical periods. Further, the evaluation system collects the target network security management system's target positive result count S within the current evaluation period. i The number of negative results F of the target i After that, the posterior Beta distribution Beta(a0) can be generated. i +S i , b0 i +F i Finally, the evaluation system calculates the expected value of the posterior Beta distribution, which is formulated as E[X] = a / (a+b). The positive posterior frequency a0 is then used to calculate the expected value. i +S i Divide by the sum of the positive and negative posterior exponents (a0) i +S i )+( b0 i +F i The expected value is obtained, and this expected value is used as the weight value w of the evaluation index. i .

[0072] For example, taking the attack interception rate evaluation index as an example, assuming that the security operation requirements issued by the group company stipulate an interception rate of 90%, the prior Beta distribution can be set to Beta(9,1), that is, the prior positive number is 9 and the prior negative number is 1. This reflects the prior trust of the evaluation system in the management requirements when historical data is lacking. If the number of successful interceptions collected in the current period is 850 and the number of failed interceptions is 150, then the posterior Beta distribution is Beta(859,151), and its expected value is 859 divided by the sum of 859 and 151, which is 1010, approximately 0.8505. That is, the weight value of this index in the current period is updated to 0.8505.

[0073] In another example, if an evaluation metric has accumulated a high proportion of positive results over multiple historical periods, for example, if the prior Beta distribution is Beta(90,10), and the current period's observation data also shows a high proportion of positive results, such as S... i =85, F iIf the weight is 15, then the posterior Beta distribution is Beta(175,25), with an expected value of 175 divided by the sum of 175 and 25 (200), which equals 0.875. This is lower than the prior expected value of 0.9 but still relatively high, reflecting the adaptive adjustment of the weight values ​​with the actual data. Simultaneously, as the amount of historical data accumulates, the value of the prior parameter gradually increases, and the marginal influence of new observations on the posterior distribution gradually decreases, causing the weight values ​​to stabilize. This demonstrates the convergence of the Bayesian update estimate under large sample conditions, avoiding drastic impacts on the weight values ​​from single-period outliers.

[0074] Step 103: Construct an evidence matrix containing the evidence data, and determine the first evidence value and the second evidence value of each evaluation index in the evidence matrix. Perform a weighted fusion calculation based on the weight value, the first evidence value and the second evidence value to obtain the security capability score of the evaluation index. In this embodiment of the invention, after obtaining the dynamic weight values ​​of each evaluation indicator, the evaluation system can further fuse and calculate the evidence data from multi-source heterogeneous data to obtain the security capability score of each evaluation indicator. The security capability score can be a quantitative measure of the performance level of each evaluation indicator within the current evaluation period; its value reflects the actual performance of the security capability dimension represented by that indicator.

[0075] In practical implementation, due to the diverse types of evidence data—including at least two of the following: management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise results data—these data vary significantly in format, semantics, and credibility, making simple weighted averaging or summation impossible. Therefore, the evaluation system can first construct an evidence matrix containing the evidence data, unifying heterogeneous evidence data from different sources into a structured matrix form. The evidence matrix can be a two-dimensional data structure, with rows corresponding to various evaluation indicators and columns corresponding to various evidence sources. Each element in the matrix is ​​the quantified value of a specific evaluation indicator under a specific evidence source.

[0076] In some feasible implementations, the process of constructing the evidence matrix involves the evaluation system first acquiring the evidence sources of the evidence data, i.e., determining the multiple data source channels corresponding to each evaluation indicator. Then, it acquires the quantified value of each evaluation indicator under each evidence source, i.e., standardizing and quantizing the original evidence data to enable comparison on a uniform scale. Finally, the corresponding evidence matrix is ​​constructed from the quantified values ​​of each evaluation indicator under each evidence source. In this matrix, each row corresponds to an evaluation indicator, and each column corresponds to an evidence source.

[0077] Taking emergency response capability as an evaluation indicator as an example, it can correspond to three sources of evidence: Source 1 is the emergency response process management system, which provides average handling time data; Source 2 is the security device log system, which provides automatic interception rate data for security incidents; and Source 3 is the attack and defense exercise platform, which provides emergency response score data during exercises. The evaluation system extracts raw data from these three sources respectively, and after quantification mapping, obtains three quantified values, for example, 0.82, 0.76, and 0.91. These three quantified values ​​constitute the evidence vector corresponding to this indicator and serve as the three elements in the corresponding column position of the row in the evidence matrix.

[0078] After constructing the evidence matrix, the evaluation system can also extract the evidence vectors corresponding to each evaluation indicator from the evidence matrix. The evidence vectors contain the quantitative values ​​of the evaluation indicator under multiple evidence sources. Then, the maximum value in the evidence vectors is used as the first evidence value, and the minimum value in the evidence vectors is used as the second evidence value.

[0079] In its implementation, the evaluation system extracts the corresponding evidence vector from the evidence matrix for each evaluation indicator. This evidence vector contains a set of quantified values ​​for that indicator across all evidence sources. Each quantified value represents an independent observation or assessment of the indicator's performance level from a single evidence source. Among these quantified values, the evaluation system designates the maximum value as the first evidence value and the minimum value as the second evidence value. The first evidence value represents the most favorable or positive assessment result across all evidence sources for that indicator, and can be considered the strongest evidence supporting its high performance. The second evidence value represents the most unfavorable or negative assessment result across all evidence sources for that indicator, and can be considered the weakest evidence reflecting potential problems or weaknesses in the indicator.

[0080] Accordingly, following the example of emergency response capability described above, the evidence vector is [0.82, 0.76, 0.91]. The evaluation system determines the maximum value of 0.91 as the first evidence value of this indicator and the minimum value of 0.76 as the second evidence value. When there are multiple equal maximum or minimum values, one of them can be selected as the first or second evidence value, or the equal values ​​can be averaged to obtain the corresponding evidence value. This invention does not impose any restrictions on this.

[0081] Furthermore, after obtaining the first evidence value and the second evidence value, the evaluation system can perform a weighted fusion calculation based on both and the weight value to obtain the security capability score corresponding to each evaluation indicator. In some feasible implementations, during the calculation of the security capability score, the evaluation system can obtain a first adjustment parameter and a second adjustment parameter, calculate the product of the first power of the first adjustment parameter of the first evidence value and the second power of the second adjustment parameter of the second evidence value, and multiply this product by the weight value corresponding to the evaluation indicator to obtain the security capability score of that evaluation indicator.

[0082] The first and second adjustment parameters are used to control the influence of the first and second evidence values ​​in the fusion calculation, respectively. In the specific implementation, after obtaining the first and second adjustment parameters, the evaluation system calculates the product of the first power of the adjustment parameter and the second power of the adjustment parameter of the first evidence value. This product is then multiplied by the weight value corresponding to the evaluation indicator to obtain the safety capability score of that indicator. Thus, through the aforementioned fusion calculation method, the weight value determines the fundamental importance of the indicator in the overall evaluation. Under the control of the first adjustment parameter, the first evidence value determines the positive reinforcement contribution of strong evidence to the final score, while the second evidence value, under the control of the second adjustment parameter, determines the negative retention contribution of weak evidence to the final score. This allows the evaluation result to simultaneously consider the mainstream judgment of most evidence and the warning effect of a few key weak signals.

[0083] Alternatively, the formula for weighted fusion can be expressed as: A i =w i ×max(R i )^α×min(R i )^β, where A i For the security capability score of the i-th evaluation indicator, w i R represents the weight value corresponding to the i-th evaluation index. i Let R be the evidence vector corresponding to the i-th evaluation index, and max(R) i ) represents the first evidence value, i.e., the maximum value in the evidence vector, min(R) i ) is the minimum value in the second evidence vector, α is the first adjustment parameter, and β is the second adjustment parameter.

[0084] In the above example of emergency response capability, if the weight value w i The first adjustment parameter α is 0.844, the second adjustment parameter β is 0.85, and the first evidence value max(R) is 0.90. i The second evidence value min(R) is 0.91. i If the score is 0.76, then the safety capability score is A. i=0.844×0.91^0.85×0.76^0.90, which yields approximately 0.844×0.924×0.782≈0.610. This score reflects both the weight of this indicator in the overall evaluation and the comprehensive assessment results from multiple evidence sources. In the above calculation, the second evidence value of 0.76, as weak evidence, is appropriately retained under the control of the second adjustment parameter β=0.90. This ensures that the score does not ignore the weaknesses reflected by the attack and defense exercise platform and avoids obscuring actual problems due to optimistic assessments based on strong evidence.

[0085] Furthermore, in some feasible implementations, the aforementioned first and second adjustment parameters can be obtained in the following ways. Specifically, the evaluation system first extracts feature parameters from the evidence matrix, including at least one of evidence conflict degree, evidence coverage, and data freshness.

[0086] Among them, the degree of evidence conflict is used to measure the degree of inconsistency between the quantitative values ​​of various evidence sources in the evidence matrix. When the quantitative values ​​of each source differ greatly, the degree of evidence conflict is high, indicating that different evidence sources have given contradictory evaluation conclusions on the same evaluation index. In this case, it is necessary to reduce the sensitivity to weak evidence to avoid being misled by unreliable weak evidence.

[0087] Correspondingly, evidence coverage is used to measure the completeness of valid data in the evidence matrix. When the coverage is high, the evaluation system can obtain more comprehensive information. At this time, the weight of key evidence can be appropriately strengthened to make full use of high-quality evidence.

[0088] In addition, data freshness is used to measure the timeliness of evidentiary data. When the data is newer, it has a stronger ability to reflect the current security situation. In this case, weak evidence can be appropriately retained to capture the latest security situation signals.

[0089] In specific implementations, the degree of evidence conflict can be determined based on the degree of dispersion among elements in the same row of the evidence matrix, for example, by using statistical measures such as standard deviation, range, or coefficient of variation; the evidence coverage can be determined based on the proportion of non-empty elements in the evidence matrix to all elements; and the data freshness can be determined based on the difference between the timestamp of the evidence data and the current time, for example, by taking the average or maximum value of the difference between the timestamp of each piece of evidence data and the current time. This invention does not impose any restrictions on this.

[0090] After obtaining the aforementioned characteristic parameters, the evaluation system can determine a first initial value based on the degree of evidence conflict. The first initial value is negatively correlated with the degree of evidence conflict; that is, the higher the degree of evidence conflict, the smaller the first initial value. This reduces the risk of strong evidence being overemphasized and prevents strong evidence from gaining an unreasonable dominant position when there is evidence conflict. Correspondingly, the evaluation system can determine a second initial value based on evidence coverage and data freshness. The second initial value is positively correlated with evidence coverage and data freshness; that is, the higher the evidence coverage and data freshness, the larger the second initial value. This increases the likelihood that weak evidence will be effectively preserved, enabling the evaluation to make full use of comprehensive and fresh evidence information.

[0091] Optionally, the evaluation system can directly use the first initial value as the first adjustment parameter and the second initial value as the second adjustment parameter. In a specific numerical example, if the degree of evidence conflict C is 0.2, then the first initial value α0 can be calculated as 1 divided by the sum of 1 and 0.2, approximately 0.833; if the evidence coverage rate cov is 0.9, the data freshness τ is 0.1, and the preset decay coefficient λ is 0.5, then the second initial value β0 can be calculated as 0.9 multiplied by the product of the natural constant e (-0.5) and 0.1 raised to the power of e, i.e., 0.9 multiplied by e (-0.05), approximately 0.856. The above values ​​are only examples; the parameter values ​​in actual applications can be adjusted according to specific scenarios.

[0092] In some feasible implementations, the specific calculation method for determining the first initial value based on the degree of evidence conflict can be as follows: calculate the sum between the first preset value and the degree of evidence conflict, and use the reciprocal of this sum as the first initial value. The first preset value can be 1, then the first initial value α0 = 1 / (1+C), where C is the degree of evidence conflict. When C=0, it indicates that the quantified values ​​of each evidence source are completely consistent and there is no conflict. At this time, α0=1, and strong evidence participates in the fusion with its original strength. When C approaches infinity, α0 approaches 0, and the reinforcing effect of strong evidence is completely suppressed.

[0093] Accordingly, the specific calculation method for determining the second initial value based on evidence coverage and data freshness can be as follows: Calculate the exponent of the product between a preset decay coefficient (base e, negative exponent) and data freshness, i.e., e^(-λτ), and use the product of evidence coverage and this exponent as the second initial value, i.e., β0 = cov × e^(-λτ), where cov is the evidence coverage, τ is the data freshness, and λ is the preset decay coefficient. The preset decay coefficient λ is used to control the decay rate of data freshness on the second initial value, and can be set according to the actual business scenario, for example, taking values ​​of 0.1, 0.5, or 1.0. When the evidence coverage is higher and the data is fresher, the value of β0 is larger, and the degree of retention of weak evidence in fusion is higher; when the evidence coverage is lower or the data is older, the value of β0 is smaller, and the degree of retention of weak evidence is reduced, so as to avoid drawing unreliable evaluation conclusions based on incomplete or outdated data.

[0094] Furthermore, in some feasible implementations, after obtaining the first and second initial values ​​through the aforementioned methods, the evaluation system can further fine-tune the initial values ​​using a machine learning model to obtain the final first and second adjustment parameters. Optionally, the evaluation system can use feature parameters from historical evaluation periods as training samples, and use manually labeled adjustment parameters or adjustment parameters corresponding to standard evaluation results from historical evaluation periods as labels for training to obtain a trained machine learning model. Then, the feature parameters of the current evaluation period are input into the trained machine learning model for adjustment to obtain a first adjustment amount for the first initial value and a second adjustment amount for the second initial value. Finally, the sum of the first initial value and the first adjustment amount is used as the first adjustment parameter, and the sum of the second initial value and the second adjustment amount is used as the second adjustment parameter.

[0095] In practical implementation, the machine learning model can employ a decision tree regression model, using three dimensions—evidence conflict, evidence coverage, and data freshness—as input features. The decision tree regression model is a non-parametric regression method based on a tree structure. It recursively divides the feature space into several sub-regions, using a constant (i.e., the output value of the leaf node) for prediction within each sub-region. During the training phase, the decision tree splits based on feature values, selecting the features and split points that minimize the mean squared error within the sub-regions, recursively constructing a binary tree structure until a preset stopping condition is met. Each leaf node stores the optimal parameter adjustment amount for the corresponding feature combination.

[0096] For example, suppose the evaluation system has accumulated empirical data for 6 historical evaluation periods. The feature parameters and manually labeled optimal adjustment parameters for each period are shown in Table 1 below:

[0097] Table 1 The decision tree regression model was trained on the aforementioned six samples. Assuming the model uses the evidence conflict degree C as the splitting feature for the root node, and selects a splitting threshold of C=0.25, then periods 1, 2, and 4 (C values ​​of 0.05, 0.18, and 0.08 respectively, all less than 0.25) are assigned to the left child node, and periods 3, 5, and 6 (C values ​​of 0.35, 0.22, and 0.45 respectively, with period 5 having C=0.22 which is less than 0.25; the actual splitting needs to be determined based on the optimal splitting point, this is only an example) are assigned to the right child node. Within the left child node, the model further splits based on the evidence coverage rate (cov). For example, if a threshold of cov=0.90 is chosen, periods 1 and 4 (cov=0.92 and 0.95 respectively) are assigned to the left leaf node, where the Δα prediction value stored is (+0.02+0.01) / 2=+0.015; period 2 (cov=0.88) is assigned to the right leaf node, where the Δα prediction value stored is -0.03. This process continues, with each leaf node storing the average adjustment value under the corresponding feature combination as the prediction output. By setting hyperparameters such as a maximum tree depth of 4 layers and a minimum number of samples per leaf node of 1, the model can fit the training data while avoiding overfitting.

[0098] When the evaluation system executes the current evaluation cycle, if the extracted evidence matrix feature parameters are C=0.12, cov=0.91, and τ=0.05, then this feature vector is input into the trained decision tree model. According to the decision tree splitting rules, the model sequentially judges from the root node: if C=0.12 is less than the splitting threshold, it enters the left subtree; in the left subtree, it judges whether cov=0.91 is greater than or equal to the threshold 0.90, and enters the corresponding leaf node. This leaf node outputs a predicted Δα value of +0.015 and a predicted Δβ value of +0.015. Simultaneously, the evaluation system calculates initial values ​​based on the degree of evidence conflict, evidence coverage, and data freshness: α0=1 / (1+0.12)≈0.893, β0=0.91×e^(-0.5×0.05)=0.91×e^(-0.025)≈0.888. The evaluation system adds the initial values ​​to the adjustment amounts, resulting in the final first adjustment parameter α = 0.893 + 0.015 = 0.908 and the second adjustment parameter β = 0.888 + 0.015 = 0.903. The above example demonstrates that, under conditions of low evidence conflict, high evidence coverage, and high data freshness, historical experience indicates a slight need for positive adjustments to both α and β. That is, under this combination of features, moderately increasing the reinforcement of strong evidence and the retention of weak evidence can achieve better consistency in the evaluation results.

[0099] In other examples, if the feature parameters extracted in the current evaluation period are C=0.38, cov=0.72, and τ=0.18, then after inputting them into the decision tree model, the root node determines that C=0.38 is greater than the threshold of 0.25, and proceeds to the right subtree. In the right subtree, the feature parameter combination is further evaluated, and the result is finally placed in a leaf node, outputting a predicted value of Δα of -0.09 and a predicted value of Δβ of -0.06. At this point, the initial values ​​are α0=1 / (1+0.38)≈0.725 and β0=0.72×e^(-0.5×0.18)=0.72×e^(-0.09)≈0.658. The final adjusted parameters are α=0.725-0.09=0.635 and β=0.658-0.06=0.598. Therefore, when the degree of evidence conflict is high and the evidence coverage and data freshness are both low, historical experience suggests that the adjustment parameters for strong evidence and weak evidence should be significantly reduced to prevent unreliable evidence from gaining too much influence during the fusion process, thereby improving the stability of the evaluation results.

[0100] In other examples, the aforementioned machine learning model can also employ the K-nearest neighbor algorithm. The K-nearest neighbor algorithm can be an instance-based nonparametric learning method that, for a given input sample, finds the K nearest neighbors in the training sample set whose features are most similar to its input sample, and then predicts the output by weighting or voting on the label values ​​of these K nearest neighbors.

[0101] In this embodiment of the invention, when the evaluation system needs to predict the adjustment amount of the adjustment parameter for the feature parameters of the current evaluation period, the K-nearest neighbor algorithm searches for the K historical samples in the historical training sample set that are closest to the current feature vector in terms of Euclidean distance. The weighted average of the manually labeled adjustment amounts of these K historical samples is taken as the predicted adjustment amount for the current period. For example, if K=3, the current feature vector is C=0.15, cov=0.89, and τ=0.07, the evaluation system calculates the Euclidean distance to the feature vector in the historical sample set and finds the three closest historical samples with labeled Δα of +0.01, -0.02, and +0.015, respectively. Then, the predicted value of Δα is approximately +0.002 and the predicted value of Δβ is approximately +0.025, calculated using the inverse distance weighted average. The evaluation system adds the corresponding adjustment amounts to the initial values ​​α0=1 / (1+0.15)≈0.870 and β0=0.89×e^(-0.5×0.07)≈0.859 respectively, to obtain the final adjustment parameters α≈0.872 and β≈0.884.

[0102] It should be noted that the advantage of the K-nearest neighbors algorithm is that it does not require an explicit training process and can flexibly adapt to the distribution characteristics of training samples. As historical experience data accumulates, the prediction accuracy can be continuously improved.

[0103] By introducing machine learning models for fine-tuning, the evaluation system can make refined adjustments based on historical experience data on the initial values, effectively reducing inconsistencies caused by manual parameter tuning and improving the stability and accuracy of the fusion process.

[0104] Step 104: Generate network security capability evaluation results based on the security capability scores of each of the evaluation indicators.

[0105] In this embodiment of the invention, after obtaining the security capability scores of each evaluation indicator, the evaluation system can generate a network security capability evaluation result. Specifically, the evaluation system can summarize the security capability scores of each evaluation indicator according to the requirements of the evaluation system. For example, it can sum the security capability scores of each indicator to obtain a comprehensive evaluation score, or assign secondary weights to the scores of each indicator and then perform a weighted summation to obtain the comprehensive security capability score of the target network security management system within the evaluation period. Then, the evaluation system can use this comprehensive security capability score as the core value of the network security capability evaluation result, and can also include detailed scores for each evaluation indicator, as well as intermediate data such as the dynamic weight value, first evidence value, second evidence value, and adjustment parameter values ​​of each indicator in this evaluation, to form a complete evaluation report.

[0106] Optionally, the evaluation results can be output in the form of scoring reports, visual dashboards, or structured data files for use in scenarios such as network security management decisions, third-party security assessments, and continuous improvement of security mechanisms. This invention does not impose any limitations on this.

[0107] For example, the evaluation results can be output as core scoring data in the monthly security posture analysis report of an enterprise group, used to rank and assess the security operation performance of each subordinate unit; they can also be output as quantitative basis for capability improvement in attack and defense exercise review reports, helping to identify weak links in the security defense system; and they can also serve as objective reference data for third-party security assessment agencies to conduct compliance reviews.

[0108] In one application scenario, the security operations center of a large enterprise group needs to conduct monthly evaluations of the cybersecurity capabilities of its subordinate units. Specifically, the evaluation system extracts observation data for multiple evaluation indicators from the group's integrated cybersecurity dispatch platform. Taking the number of converged exposure surfaces as an example, the evaluation system obtains the number of positive results for this indicator in the past three historical periods, i.e., the number of successfully converged exposure surfaces is 12, 15, and 13, respectively; and the number of negative results, i.e., the number of newly added or non-converged exposure surfaces, is 3, 2, and 4, respectively. Then, based on this historical data, the evaluation system determines the prior Beta distribution parameters for this indicator as a prior positive count of 40 and a prior negative count of 9, i.e., Beta(40,9). In the current evaluation period, the evaluation system collected 14 positive results and 1 negative result for this indicator. Therefore, the posterior Beta distribution is Beta(54,10), with an expected value of 54 divided by the sum of 54 and 10 (64), approximately 0.844. This means the weight of this indicator in the current evaluation period is updated to 0.844. Simultaneously, the evaluation system obtains multiple evidence sources for this indicator, including quantified data from the vulnerability scanning system (0.85), access control logs from the perimeter firewall (0.78), and penetration test results from the attack and defense exercise platform (0.92), forming an evidence vector [0.85, 0.78, 0.92]. The evaluation system then uses the maximum value of this evidence vector, 0.92, as the first evidence value and the minimum value, 0.78, as the second evidence value. The evaluation system further extracts feature parameters from the evidence matrix, calculating an evidence conflict degree of 0.14, an evidence coverage rate of 0.95, and a data freshness of 0.08. Based on this, the first adjustment parameter is determined to be approximately 0.877, and the second adjustment parameter is approximately 0.913. Further, the evaluation system calculates the product of 0.92 raised to the power of 0.877 and 0.78 raised to the power of 0.913, multiplying this by a weight value of 0.844, resulting in a security capability score of approximately 0.681 for this indicator in the current evaluation period. The evaluation system performs the above calculations on all evaluation indicators, summarizing the scores of each indicator to generate the organization's comprehensive cybersecurity capability score for the month.

[0109] In another application scenario, a subordinate unit experienced a real ransomware attack during its monthly evaluation period, causing significant fluctuations in its vulnerability remediation timeliness rate. Historical data for this indicator shows that the timeliness rates for the past three periods were 92%, 88%, and 90%, with corresponding positive and negative outcomes of 46, 44, and 45, and 4, 6, and 5, respectively. Based on this historical data, the evaluation system determined a prior Beta distribution of Beta(135,15) with an expected value of 0.90. In the current evaluation period, due to the ransomware attack causing a large number of vulnerabilities to be patched, the unit actually patched 30 vulnerabilities that were not patched within the time limit, resulting in 10 positive outcomes and 10 negative outcomes. The posterior Beta distribution is Beta(165,25), with an expected value of approximately 0.868. The weighting value was lowered from 0.90 to 0.868, accurately reflecting the actual decline in the evaluation indicator's capabilities due to security incidents. Meanwhile, the evidence data corresponding to this indicator includes a quantified value of 0.75 for repair timeliness data from the vulnerability handling work order system, 0.88 for protection effectiveness data from security device detection logs, and 0.62 for practical testing data from the attack and defense exercise platform. The evidence vector is [0.75, 0.88, 0.62], with a maximum value of 0.88 and a minimum value of 0.62. In the fusion calculation, the minimum value of 0.62, as weak signal evidence, is appropriately retained under the control of the second adjustment parameter. This ensures that the security capability score of this evaluation indicator does not completely ignore the weaknesses revealed by practical testing in attack and defense exercises. The final evaluation result not only quantifies the degree of decline in the unit's security capability level for the month but also retains the specific weakness information reflected by each dimension of evidence through a differentiated evidence fusion mechanism, providing data support for the selection of the direction of subsequent security construction investment.

[0110] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that those skilled in the art can make further settings according to actual needs under the guidance of the ideas in the embodiments of the present invention, and the present invention does not limit such settings.

[0111] In this embodiment of the invention, multi-period observation data corresponding to multiple evaluation indicators and evidence data corresponding to each evaluation indicator are acquired. Then, the weight values ​​of each evaluation indicator in the evaluation system are updated based on the multi-period observation data. Next, an evidence matrix containing evidence data is constructed, and the first and second evidence values ​​of each evaluation indicator in the evidence matrix are determined. A weighted fusion calculation is performed based on the weight values, the first evidence value, and the second evidence value to obtain the security capability score for each evaluation indicator. Finally, a network security capability evaluation result is generated based on each security capability score. Thus, by dynamically associating the weight settings of the evaluation indicators with the multi-period observation data continuously accumulated during actual security operations, the weight values ​​can be adjusted according to the security state. The system adaptively adjusts to changes in the situation, overcoming the shortcomings of traditional static weights that cause evaluation results to lag behind actual security conditions. Furthermore, by constructing an evidence matrix and distinguishing between first evidence values ​​representing strong evidence and second evidence values ​​representing weak evidence, it applies differentiated influences to evidence of different strengths during fusion calculations. This ensures that low-frequency but critically indicative security event signals are effectively preserved rather than drowned out. At the same time, dynamic weight updates and differentiated evidence fusion work together to improve the sensitivity and discriminativeness of evaluation results to changes in actual security capabilities from two dimensions: the time-varying adaptability of indicator importance and the contribution of multi-source evidence. This enhances the objectivity of the evaluation system and the credibility of the evaluation results.

[0112] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are provided for illustrative purposes: As some examples, see Figure 2 The diagram illustrates the system architecture of the evaluation system provided in an embodiment of the present invention. Figure 2As shown, the evaluation system can include at least a weight update module, an evidence fusion module, and a parameter tuning module. The weight update module executes a dynamic Bayesian network algorithm to improve the weight update mechanism in the current security capability evaluation system. Specifically, the input to the weight update module is multi-level cybersecurity indicators and structured data collected from the group's integrated cybersecurity scheduling platform system, including the number of positive and negative results for each evaluation indicator in multiple historical periods. The weight update module maintains a prior Beta distribution for each evaluation indicator. This prior distribution is set according to the security operation requirements issued by the group company or the assessment benchmarks in industry standards, reflecting the prior understanding of the indicator's importance when current period data is lacking. The weight update module uses the number of positive and negative results collected in the current period as observation data, performs Bayesian fusion with the prior Beta distribution to generate a posterior Beta distribution, and calculates the expected value of the posterior distribution as the dynamic weight value for each evaluation indicator. Therefore, the weight update module enables the evaluation index weights to be adaptively corrected as actual safety operation data accumulates, so that the weight values ​​can track the time-varying characteristics of the safety situation, thereby overcoming the defect of the evaluation results lagging behind the actual safety situation caused by traditional static weights.

[0113] Accordingly, the evidence fusion module is used to construct a multi-source behavioral evidence fusion model to calculate the security capability level of each indicator by combining the dynamic weights output by the weight update module. The input to the evidence fusion module includes two parts: one is the dynamic weight value of each evaluation indicator output by the weight update module; the other is evidence data from multiple data sources corresponding to each evaluation indicator collected from the group's cybersecurity integrated dispatch platform system. Evidence data can cover various types, including management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise results data. The evidence fusion module first standardizes and quantizes the aforementioned multi-source heterogeneous data, constructing an evidence matrix from the quantified values ​​of each evaluation indicator under each evidence source. For each evaluation indicator, the evidence fusion module extracts the corresponding evidence vector from the evidence matrix, determining the maximum value in the evidence vector as the strongest evidence and the minimum value as the weakest evidence. Subsequently, the evidence fusion module obtains the first and second adjustment parameters output by the parameter tuning module. It then uses a sparse fusion operator to fuse the dynamic weights, the strongest evidence, the weakest evidence, and the two adjustment parameters. Specifically, it calculates the product of the first adjustment parameter raised to the power of the strongest evidence and the second adjustment parameter raised to the power of the weakest evidence, and multiplies this product by the dynamic weights to obtain the security capability score for each evaluation indicator. The evidence fusion module further summarizes the security capability scores for each evaluation indicator to generate a comprehensive evaluation score. Thus, the evidence fusion module achieves differentiated fusion of multi-source heterogeneous evidence, effectively preserving rather than burying low-frequency but critically indicative weak signal evidence, while ensuring that strong evidence receives a reasonable contribution boost, thereby improving the sensitivity and discriminative power of the evaluation results to changes in actual security capabilities.

[0114] Furthermore, the parameter tuning module is used to construct an AI parameter adjuster to achieve adaptive tuning of the multi-source evidence fusion model algorithm. The input to the parameter tuning module is the evidence matrix constructed by the evidence fusion module. Intuitive feature parameters of the evidence matrix, including evidence conflict degree, evidence coverage, data freshness, and evidence quantity, are extracted as the basis for parameter adjustment. The parameter tuning module first calculates the initial values ​​of the adjustment parameters based on preset rules: the initial value of the first adjustment parameter is determined based on the evidence conflict degree. The initial value of the first adjustment parameter is negatively correlated with the evidence conflict degree; that is, the greater the evidence conflict, the smaller the first adjustment parameter, thus reducing the sensitivity to weak evidence. The initial value of the second adjustment parameter is determined based on the evidence coverage and data freshness. The initial value of the second adjustment parameter is positively correlated with the evidence coverage and data freshness; that is, the higher the evidence coverage and the newer the data, the larger the second adjustment parameter, thus strengthening the weight of key evidence. Building upon this foundation, the parameter tuning module further utilizes a machine learning model to fine-tune the first and second adjustment parameters: It uses feature parameters from historical evaluation periods as training samples, and manually labeled preferred adjustment parameters or adjustment parameters corresponding to standard evaluation results from historical evaluation periods as labels for training, thus obtaining a trained machine learning model; it inputs the feature parameters of the current evaluation period into the trained model to obtain a first adjustment amount for the first initial value and a second adjustment amount for the second initial value; finally, it adds the initial value to the corresponding adjustment amount to obtain the final first and second adjustment parameters. Therefore, the parameter tuning module achieves automatic recommendation and fine-tuning of adjustment parameters during evidence fusion, effectively replacing manual parameter tuning, reducing inconsistencies caused by manual tuning, improving the stability and accuracy of the fusion process, and enhancing the adaptive capability and decision consistency of the evaluation system when working collaboratively between modules.

[0115] Through the collaborative work of the three core modules, the evaluation system achieves a complete evaluation chain: adaptively updating indicator weights from multi-period observation data, differentially fusing and calculating security capability scores from multi-source heterogeneous evidence, and intelligently optimizing fusion parameters based on machine learning. The dynamic weights output by the weight update module provide the evidence fusion module with basic coefficients reflecting the time-varying importance of indicators, while the optimized adjustment parameters output by the parameter tuning module provide the evidence fusion module with fusion control parameters that are adaptive to the data distribution characteristics. The three modules form a closed-loop relationship of data flow and parameter feedback, jointly supporting the evaluation system to provide objective, accurate, and adaptive cybersecurity capability evaluation in a dynamic security operation environment.

[0116] Based on the above system architecture, in some application scenarios, suppose the security operations center of a large enterprise group needs to conduct a monthly evaluation of the cybersecurity capabilities of its subordinate unit A. The evaluation system extracts observational and evidentiary data for multiple evaluation indicators from the group's integrated cybersecurity dispatch platform.

[0117] Taking the number of converged exposure surfaces as an example, this indicator measures the ability of subordinate units to proactively identify and converge internet exposure surfaces. The evaluation system first obtains the number of positive and negative results for this indicator from the platform system over the past three historical periods: In the first historical period, 12 exposure surfaces were successfully converged, and 3 new non-converged exposure surfaces were added; in the second historical period, 15 exposure surfaces were successfully converged, and 2 new non-converged exposure surfaces were added; in the third historical period, 13 exposure surfaces were successfully converged, and 4 new non-converged exposure surfaces were added. Meanwhile, the safety operation requirements issued by the group company stipulate that the benchmark for the exposure surface convergence rate is 90%, i.e., the prior positive count 'a'. 0i It can be set to 9, with the prior negative count b. 0i It can be set to 1. The evaluation system uses the sum of the number of positive results in the historical period as the prior positive count, i.e., a. 0i =12+15+13=40, the sum of the number of negative results in the historical cycle is taken as the prior negative count, i.e., b 0i =3+2+4=9, thus forming the prior Beta distribution of this index: prior i =Beta(a 0i ,b 0i =Beta(40,9) In the current evaluation period, the evaluation system collected data on 14 successfully converged exposure surfaces and 1 newly added non-converged exposure surface for this unit, which is the number of positive results S in the current period. i =14, the number of negative results F in the current period i =1. The evaluation system fuses the prior Beta distribution with the current period's observation data to form the posterior Beta distribution: posterior i =Beta(a 0i +S i ,b 0i +F i =Beta(40+14,9+1)=Beta(54,10) The evaluation system calculates the expected value of the posterior Beta distribution as the weight value of this indicator in the current evaluation period: w i =(a 0i +S i ) / =54 / (54+10)=54 / 64=0.844 This weighting value reflects 90% of the performance evaluation criteria in the group company's management requirements (through prior parameter a). 0i =40、b 0i=9 implicitly reflects the fact that its prior expected value is 40 / (40+9)=40 / 49≈0.816, and also incorporates the actual performance of the unit in the past three historical cycles (the historical positive proportion is approximately (12+15+13) / (12+15+13+3+2+4)=40 / 49≈0.816). At the same time, it also incorporates the latest observation data of the current cycle (the current positive proportion is 14 / 15≈0.933). The posterior expected value of 0.844 is between the prior expected value of 0.816 and the current observation proportion of 0.933, reflecting the characteristic of organic integration of historical experience and current data.

[0118] Meanwhile, the evaluation system acquires data from multiple evidence sources for this metric. The convergence capability of this metric's exposure surface can be evaluated from three evidence sources: Evidence source 1 is the exposure surface detection data from the vulnerability scanning system, whose quantification value is r. i1 =0.85; Evidence source 2 is the access control log analysis data of the border firewall, and its quantification value is r. i2 =0.78; Evidence source 3 is the penetration test evaluation data of the attack and defense exercise platform, and its quantitative value is r. i3 =0.92. This forms the evidence vector for this indicator: R i =[r i1 ,r i2 ,r i3 =[0.85,0.78,0.92] By combining the evidence vectors of multiple evaluation indicators, a complete evidence matrix can be formed, where each row corresponds to an evaluation indicator and each column corresponds to a source of evidence. R=[r ij ] nxm Where n is the number of evaluation indicators, m is the number of evidence sources, and r ij This represents the quantitative value of the i-th evaluation index under the j-th source of evidence.

[0119] For the current evaluation index, the evaluation system determines its first evidence value and second evidence value: the first evidence value is the maximum value in the evidence vector, i.e., max(R). i The value of 0.92 represents the strongest evidence from the attack and defense exercise platform, indicating that the unit demonstrated excellent convergence capability of its exposed surface in realistic combat testing. The second piece of evidence is the minimum value in the evidence vector, i.e., min(R). iThe value 0.78 represents the weakest evidence from the border firewall logs, indicating that the convergence of the exposure surface is still insufficient in daily operational monitoring. The evaluation system further extracts characteristic parameters from the evidence matrix: the evidence conflict degree C measures the dispersion between the quantified values ​​of each evidence source. In this example, the standard deviations of the three quantified values ​​(0.85, 0.78, and 0.92) are approximately 0.057, and the normalized evidence conflict degree C is approximately 0.14; the evidence coverage rate cov is the proportion of non-empty elements in the evidence matrix. In this example, all evidence sources have data, so cov = 1.0; the data freshness τ is a measure of the timeliness of the evidence data, ranging from [0,1]. In this example, τ = 0.08 indicates that the data is relatively fresh. The evaluation system calculates the initial values ​​of the first and second adjustment parameters based on the above characteristic parameters: α0=1 / (1+C)=1 / (1+0.14)≈0.877 β0=cov×e - λτ If the attenuation coefficient λ = 0.5, then: β0 = 1.0 × e -0 · 5x0 · 08 =1.0×e -0 · 04 ≈0.961 The results indicate that strong evidence participates in fusion with greater intensity when the degree of evidence conflict is low, while weak evidence is moderately preserved when the evidence coverage is complete and the data is fresh. The evaluation system substitutes these parameters into the fusion calculation formula to obtain the security capability score for this indicator: A i =w i ×[max(R i )] ×[min(R i )]

[0120] Substitute the specific values: A i =0.844×0.92 0 · 877 ×0.78 0 · 96 ¹ Among them, 0.92 0 · 877 This indicates the strongest evidence is the enhancement value under the control of the first adjustment parameter, which is 0.78. 0 · 96 ¹ represents the retention value of the weakest evidence under the control of the second adjustment parameter. The calculated value is 0.92. 0 · 877 ≈0.929, 0.78 0 ·96 ¹≈0.787, then: A i ≈0.844×0.929×0.787≈0.617 The score reflects the weight of this indicator in the current evaluation system (0.844) and also integrates the comprehensive evaluation results from multiple evidence sources (both strong evidence 0.929 and weak evidence 0.787 contributed to the final score). The weak evidence of 0.78 was retained as 0.787, which means that the underestimation from the border firewall logs was not completely ignored, thus preserving the weak signals reflected by this dimension.

[0121] In other feasible implementations, the evaluation system can further utilize machine learning models to fine-tune the adjustment parameters. Continuing the example above, after obtaining the initial values ​​of the first adjustment parameter α0≈0.877 and the second adjustment parameter β0≈0.961, the evaluation system inputs the feature parameters C=0.14, cov=1.0, and τ=0.08 for the current evaluation period into the trained decision tree regression model. This model outputs adjustments to the initial values ​​based on historical experience data. Assuming the model outputs a first adjustment Δα=+0.015 and a second adjustment Δβ=+0.010, the evaluation system fine-tunes the initial values ​​to obtain the final first and second adjustment parameters: α =α0+Δα=0.877+0.015=0.892 β =β0+Δβ=0.961+0.010=0.971 The evaluation system substitutes the fine-tuned adjustment parameters into the fusion calculation formula to obtain the optimized safety capability score: A i opt =w i ×[max(R i )] ×[min(R i )]

[0122] Substitute the specific values: A i opt =0.844×0.92 0 · 89 ²×0.78 0 · 97 ¹ The calculated value is 0.92. 0 · 89 ²≈0.928, 0.78 0 · 97 ¹≈0.785, then: A i opt ≈0.844×0.928×0.785≈0.615 The score differs slightly from the unadjusted score of 0.617. This difference stems from the fine-tuning of the parameters by the machine learning model based on historical experience data. The fine-tuned score better incorporates the optimal parameter experience under similar feature combinations in historical evaluation periods, thereby improving the consistency and stability of the evaluation results.

[0123] After obtaining the security capability score for this indicator, the evaluation system performs the same calculation process for all evaluation indicators to obtain the security capability scores A1, A2, ..., A for each indicator. n The scores for each item are then combined to obtain the unit's overall safety capability score for the month. Score=Σ i=1 n A i In this example, assuming there are 5 evaluation indicators, and the security capability scores for each indicator are 0.617, 0.723, 0.584, 0.691, and 0.748 respectively, then the overall security capability score is: Score=0.617+0.723+0.584+0.691+0.748=3.363 In the case of a maximum score of 5 points, the evaluation system will use this comprehensive score as the core numerical output of the organization's monthly cybersecurity capability evaluation result.

[0124] In another application scenario of this invention, a subordinate unit experienced a real ransomware attack during its monthly evaluation period, causing significant fluctuations in its vulnerability remediation timeliness rate. Historical data for this indicator shows that the vulnerability remediation timeliness rates for the past three periods were 92%, 88%, and 90%, with corresponding positive and negative results of 46, 44, and 45, and 4, 6, and 5, respectively. Based on this historical data, the evaluation system determines the prior Beta distribution to be Beta(135,15), with an expected value of 135 / (135+15) = 135 / 150 = 0.90. In the current evaluation period, due to the ransomware attack causing a large number of vulnerabilities to be patched, the unit actually patched 30 vulnerabilities that month, and 10 vulnerabilities were not patched within the time limit. The target number of positive results is S. i =30, the number of negative outcomes of the objective F i=10, the posterior Beta distribution is Beta(165,25), and the expected value is 165 / (165+25)=165 / 190≈0.868. The weight value is reduced from 0.90 to 0.868, which accurately reflects the actual decline in the performance of this evaluation indicator due to the impact of security incidents. Meanwhile, among the evidence data corresponding to this indicator, the quantified value of the repair timeliness data from the vulnerability handling work order system is 0.75, the quantified value of the protection effectiveness data from the security device detection logs is 0.88, and the quantified value of the practical test data from the attack and defense exercise platform is 0.62. The evidence vector is R. i =[0.75,0.88,0.62], first evidence value max(R) i =0.88, second evidence value min(R) i =0.62. In the fusion calculation, the minimum value of 0.62, as evidence of weak signals, is moderately retained under the control of the second adjustment parameter. This ensures that the security capability score of this evaluation index does not completely ignore the weaknesses revealed by the practical test of attack and defense exercises. The final evaluation result not only quantifies the degree of decline in the unit's security capability level in that month, but also retains the specific weakness information reflected by the evidence in each dimension through the differentiated evidence fusion mechanism, providing data support for the selection of the direction of subsequent security construction investment.

[0125] In the above process, by dynamically linking the weights of the evaluation indicators with the multi-cycle observation data continuously accumulated during actual safety operations, not only can the weight values ​​be adaptively corrected as the safety situation changes, overcoming the defect of traditional static weights that cause the evaluation results to lag behind the actual safety situation, but also by constructing an evidence matrix and distinguishing between the first evidence value representing strong evidence and the second evidence value representing weak evidence, differentiating the influence of evidence of different intensities is applied during the fusion calculation, so that low-frequency but key indicative safety event signals can be effectively preserved rather than submerged. At the same time, the dynamic weight update and differentiated evidence fusion work together to improve the sensitivity and distinguishability of the evaluation results to changes in actual safety capabilities from two dimensions: the time-varying adaptability of the importance of the indicators and the contribution of multi-source evidence, thereby improving the objectivity of the evaluation system and the credibility of the evaluation results.

[0126] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0127] Reference Figure 3The diagram illustrates a structural block diagram of a network security capability evaluation device provided in an embodiment of the present invention, which may specifically include the following modules: The data acquisition module 301 is used to acquire multi-period observation data corresponding to multiple evaluation indicators, as well as evidence data corresponding to each evaluation indicator; The weight update module 302 is used to update the weight value of the evaluation index in the evaluation system based on the multi-period observation data. Evaluation module 303 is used to construct an evidence matrix containing the evidence data, determine the first evidence value and the second evidence value of each evaluation indicator in the evidence matrix, and perform weighted fusion calculation based on the weight value, the first evidence value and the second evidence value to obtain the security capability score of the evaluation indicator. The result generation module 304 is used to generate network security capability evaluation results based on the security capability scores of each of the evaluation indicators.

[0128] In some feasible implementations, the multi-period observation data includes the number of historical positive results and the number of historical negative results for each evaluation indicator in multiple historical periods, and the weight update module 302 is specifically used for: Based on the number of historical positive results and the number of historical negative results, the prior probability distribution parameters of each evaluation index are determined; Collect the number of positive and negative results of each evaluation indicator within the current evaluation period, and use this data as the observation data for the current period. Generate a posterior probability distribution based on the prior probability distribution parameters and the current periodic observation data; The weight values ​​of each evaluation index are determined based on the characteristic values ​​of the posterior probability distribution.

[0129] In some feasible implementations, the weight update module 302 is specifically used for: The sum of the number of positive results of each evaluation indicator in multiple historical periods is taken as the prior positive count, and the sum of the number of negative results of each evaluation indicator in multiple historical periods is taken as the prior negative count. The prior positive counts and the prior negative counts are combined to form the prior probability distribution parameters of each evaluation index.

[0130] In some feasible implementations, the weight update module 302 is specifically used for: The sum of the prior positive count and the target positive result count collected in the current period is taken as the posterior positive count, and the sum of the prior negative count and the negative result count collected in the current period is taken as the posterior negative count. The positive posterior counts and the negative posterior counts are combined to form the corresponding posterior probability distribution.

[0131] In some feasible implementations, the weight update module 302 is specifically used for: Calculate the expected value corresponding to the posterior probability distribution; The expected value is used as the weight value for each of the evaluation indicators.

[0132] In some feasible implementations, the evidence data includes at least two of the following: management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise result data.

[0133] In some feasible implementations, the evaluation module 303 is specifically used for: The source of the evidence data obtained; Obtain the quantitative value of the evaluation index under the evidence source; An evidence matrix is ​​constructed from the quantitative values ​​of each evaluation indicator under the aforementioned evidence source. In this matrix, each row corresponds to one of the evaluation indicators, and each column corresponds to one of the sources of evidence.

[0134] In some feasible implementations, the evaluation module 303 is specifically used for: Extract the evidence vector corresponding to each evaluation indicator from the evidence matrix, wherein the evidence vector contains the quantitative value of the evaluation indicator under the evidence source; The maximum value in the evidence vector is taken as the first evidence value, and the minimum value in the evidence vector is taken as the second evidence value.

[0135] In some feasible implementations, the evaluation module 303 is specifically used for: Obtain the first adjustment parameter and the second adjustment parameter; Calculate the product of the first power of the first adjustment parameter of the first evidence value and the second power of the second adjustment parameter of the second evidence value; Multiplying the product by the weight value corresponding to the evaluation index yields the safety capability score of the evaluation index.

[0136] In some feasible implementations, the evaluation module 303 is specifically used for: Extract the feature parameters of the evidence matrix, wherein the feature parameters include at least one of evidence conflict degree, evidence coverage and data freshness; A first initial value is determined based on the degree of evidence conflict, and the first initial value is negatively correlated with the degree of evidence conflict; A second initial value is determined based on the evidence coverage rate and the data freshness, wherein the second initial value is positively correlated with the evidence coverage rate and the data freshness; The first initial value is used as the first adjustment parameter, and the second initial value is used as the second adjustment parameter.

[0137] In some feasible implementations, the evaluation module 303 is specifically used for: Calculate the sum between the first preset value and the degree of evidence conflict, and use the reciprocal of the sum as the first initial value; The step of determining the second initial value based on the evidence coverage and the data freshness includes: Calculate the exponent of the product between the preset decay coefficient (base of the natural constant and negative exponent) and the data freshness, and use the product of the evidence coverage and the exponent as the second initial value.

[0138] In some feasible implementations, the evaluation module 303 is specifically used for: Using the aforementioned feature parameters as input features, the first initial value and the second initial value are fine-tuned, and the fine-tuned first initial value is used as the first adjustment parameter, and the fine-tuned second initial value is used as the second adjustment parameter.

[0139] In some feasible implementations, the evaluation module 303 is specifically used for: The feature parameters in the historical evaluation period are used as training samples, and the adjustment parameters corresponding to the manually labeled adjustment parameters or standard evaluation results in the historical evaluation period are used as labels for training to obtain a trained machine learning model. The feature parameters of the current evaluation period are input into the trained machine learning model for adjustment to obtain a first adjustment amount for the first initial value and a second adjustment amount for the second initial value. The sum of the first initial value and the first adjustment amount is used as the first adjustment parameter, and the sum of the second initial value and the second adjustment amount is used as the second adjustment parameter.

[0140] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0141] In addition, this invention also discloses a computer program product, characterized in that it includes computer execution instructions, which, when executed by a processor, implement the method described in this invention.

[0142] This invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described network security capability evaluation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0143] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described network security capability evaluation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0144] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0145] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program code.

[0146] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0147] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0149] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0150] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0151] The foregoing has provided a detailed description of a network security capability evaluation method and a network security capability evaluation device provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for evaluating network security capabilities, characterized in that, include: Obtain multi-period observation data corresponding to multiple evaluation indicators, as well as evidence data corresponding to each evaluation indicator; Based on the multi-period observation data, update the weight values ​​of the evaluation indicators in the evaluation system; Construct an evidence matrix containing the evidence data, and determine the first evidence value and the second evidence value of each evaluation indicator in the evidence matrix. Perform a weighted fusion calculation based on the weight value, the first evidence value and the second evidence value to obtain the security capability score of the evaluation indicator. Based on the security capability scores of each of the aforementioned evaluation indicators, a cybersecurity capability evaluation result is generated.

2. The method according to claim 1, characterized in that, The multi-period observation data includes the number of historical positive results and the number of historical negative results for each evaluation indicator in multiple historical periods. Updating the weight values ​​of the evaluation indicators in the evaluation system based on the multi-period observation data includes: Based on the number of historical positive results and the number of historical negative results, the prior probability distribution parameters of each evaluation index are determined; Collect the number of positive and negative results of each evaluation indicator within the current evaluation period, and use this data as the observation data for the current period. Generate a posterior probability distribution based on the prior probability distribution parameters and the current periodic observation data; The weight values ​​of each evaluation index are determined based on the characteristic values ​​of the posterior probability distribution.

3. The method according to claim 2, characterized in that, The step of determining the prior probability distribution parameters of each evaluation index based on the number of historical positive results and the number of historical negative results includes: The sum of the number of positive results of each evaluation indicator in multiple historical periods is taken as the prior positive count, and the sum of the number of negative results of each evaluation indicator in multiple historical periods is taken as the prior negative count. The prior positive counts and the prior negative counts are combined to form the prior probability distribution parameters of each evaluation index.

4. The method according to claim 3, characterized in that, The step of generating a posterior probability distribution based on the prior probability distribution parameters and the current periodic observation data includes: The sum of the prior positive count and the target positive result count collected in the current period is taken as the posterior positive count, and the sum of the prior negative count and the negative result count collected in the current period is taken as the posterior negative count. The positive posterior counts and the negative posterior counts are combined to form the corresponding posterior probability distribution.

5. The method according to claim 2, characterized in that, The step of determining the weight value of each evaluation index based on the feature value of the posterior probability distribution includes: Calculate the expected value corresponding to the posterior probability distribution; The expected value is used as the weight value for each of the evaluation indicators.

6. The method according to claim 1, characterized in that, The evidence data includes at least two of the following: management behavior data, operational behavior data, technical protection data, security log data, vulnerability handling data, and attack and defense exercise results data.

7. The method according to claim 1, characterized in that, The construction of the evidence matrix containing the evidence data includes: The source of the evidence data obtained; Obtain the quantitative value of the evaluation index under the evidence source; An evidence matrix is ​​constructed from the quantitative values ​​of each evaluation indicator under the aforementioned evidence source. In this matrix, each row corresponds to one of the evaluation indicators, and each column corresponds to one of the sources of evidence.

8. The method according to claim 7, characterized in that, Determining the first evidence value and the second evidence value of each of the evaluation indicators in the evidence matrix includes: Extract the evidence vector corresponding to each evaluation indicator from the evidence matrix, wherein the evidence vector contains the quantitative value of the evaluation indicator under the evidence source; The maximum value in the evidence vector is taken as the first evidence value, and the minimum value in the evidence vector is taken as the second evidence value.

9. The method according to claim 1, characterized in that, The step of performing a weighted fusion calculation based on the weight value, the first evidence value, and the second evidence value to obtain the security capability score of the evaluation index includes: Obtain the first adjustment parameter and the second adjustment parameter; Calculate the product of the first power of the first adjustment parameter of the first evidence value and the second power of the second adjustment parameter of the second evidence value; Multiplying the product by the weight value corresponding to the evaluation index yields the safety capability score of the evaluation index.

10. The method according to claim 9, characterized in that, The process of obtaining the first adjustment parameter and the second adjustment parameter includes: Extract the feature parameters of the evidence matrix, wherein the feature parameters include at least one of evidence conflict degree, evidence coverage and data freshness; A first initial value is determined based on the degree of evidence conflict, and the first initial value is negatively correlated with the degree of evidence conflict; A second initial value is determined based on the evidence coverage rate and the data freshness, wherein the second initial value is positively correlated with the evidence coverage rate and the data freshness; The first initial value is used as the first adjustment parameter, and the second initial value is used as the second adjustment parameter.

11. The method according to claim 10, characterized in that, Determining the first initial value based on the degree of evidence conflict includes: Calculate the sum between the first preset value and the degree of evidence conflict, and use the reciprocal of the sum as the first initial value; The step of determining the second initial value based on the evidence coverage and the data freshness includes: Calculate the exponent of the product between the preset decay coefficient (base of the natural constant and negative exponent) and the data freshness, and use the product of the evidence coverage and the exponent as the second initial value.

12. The method according to claim 10, characterized in that, The step of using the first initial value as the first adjustment parameter and the second initial value as the second adjustment parameter includes: Using the aforementioned feature parameters as input features, the first initial value and the second initial value are fine-tuned, and the fine-tuned first initial value is used as the first adjustment parameter, and the fine-tuned second initial value is used as the second adjustment parameter.

13. The method according to claim 12, characterized in that, The step of fine-tuning the first initial value and the second initial value, using the fine-tuned first initial value as the first adjustment parameter and the fine-tuned second initial value as the second adjustment parameter, includes: The feature parameters in the historical evaluation period are used as training samples, and the adjustment parameters corresponding to the manually labeled adjustment parameters or standard evaluation results in the historical evaluation period are used as labels for training to obtain a trained machine learning model. The feature parameters of the current evaluation period are input into the trained machine learning model for adjustment to obtain a first adjustment amount for the first initial value and a second adjustment amount for the second initial value. The sum of the first initial value and the first adjustment amount is used as the first adjustment parameter, and the sum of the second initial value and the second adjustment amount is used as the second adjustment parameter.

14. A network security capability evaluation device, characterized in that, include: The data acquisition module is used to acquire multi-period observation data corresponding to multiple evaluation indicators, as well as evidence data corresponding to each evaluation indicator. The weight update module is used to update the weight value of the evaluation index in the evaluation system based on the multi-period observation data. The evaluation module is used to construct an evidence matrix containing the evidence data, determine the first evidence value and the second evidence value of each evaluation indicator in the evidence matrix, and perform a weighted fusion calculation based on the weight value, the first evidence value and the second evidence value to obtain the security capability score of the evaluation indicator. The result generation module is used to generate network security capability evaluation results based on the security capability scores of each of the evaluation indicators.

15. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-13.

16. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-13.

17. A computer program product, characterized in that, It includes computer execution instructions, which, when executed by a processor, implement the method as described in any one of claims 1-13.

Images