A device and method for evaluating the security performance of open architecture software at different levels

Abstract

This invention discloses a security performance evaluation device and method for open architecture software at different levels. The device comprises six modules: test environment configuration, security threat simulation, indicator collection, data storage, environment cleanup, and data analysis and evaluation. It enables fully automated evaluation from environment setup, threat simulation, data collection and storage to environment cleanup and quantitative analysis. The method divides software security into three layers: system, application, and data, constructing an indicator system. It integrates indicators from each layer through multi-source data fusion and combines a dynamic weighting mechanism based on the analytic hierarchy process (AHP) to sequentially complete the normalization of secondary indicators, the quantification of primary indicators, and the calculation of a comprehensive layered security score. This invention is adapted to the multi-layered characteristics of open architectures, improving evaluation efficiency and resource utilization, outputting a comprehensive layered security evaluation result, and providing a scientific basis for security optimization and risk management.

Description

Technical Field

[0001] This invention relates to the fields of physical and automated testing technology, and in particular to a device and method for evaluating the security performance of open architecture software at different levels. Background Technology

[0002] In recent years, research on security assessment methods for open architecture software at different levels has faced numerous challenges. The main technical difficulty of open architecture based on hierarchical models lies in the complexity of the weights of security assessment indicators. Security assessment is a complex problem involving multiple objectives and criteria, combining qualitative and quantitative methods. Open architecture military equipment software involves numerous security factors, and the key challenge in the assessment process lies in how to scientifically and rationally determine the weights of each indicator. In practical applications, when using the Analytic Hierarchy Process (AHP), the weights of each evaluation indicator are selected through manual judgment, which can easily lead to excessive information loss, and the subjective factors of experts dominate, affecting the objectivity of the assessment results. Furthermore, when using the entropy weight method for analysis, some particularly dangerous evaluation indicators may be neutralized by other indicators with lower risk indices, thereby reducing the overall risk level of the assessment and compromising impartiality. Therefore, how to truly reflect the impact of each security risk factor on the software, scientifically and objectively determine the magnitude of each indicator, and construct a multi-level, multi-dimensional security assessment system has become an important direction for current research. Summary of the Invention

[0003] The purpose of this invention is to address the problems existing in the prior art by providing a security assessment method for different layers of open architecture software. This method resolves the subjectivity issues in determining the weights of security assessment indicators and the lack of a systematic and comprehensive security assessment approach in existing technologies. Specifically, this invention aims to overcome the shortcomings of traditional Analytic Hierarchy Process (AHP) and entropy weighting methods in security assessment. It determines the weights of security indicators at each layer using a scientific and reasonable method, reducing subjectivity and information loss due to human judgment, and avoiding the lack of impartiality caused by the neutralization of risk indicators. Simultaneously, this invention strives to construct a multi-layered, multi-dimensional assessment system that can comprehensively and systematically reflect the security risks and their impacts at different layers of open architecture software, thereby providing a more objective and accurate basis for software security assessment and assisting relevant decision-makers in effective risk management. Furthermore, this invention further improves the flexibility and adaptability of the assessment method by introducing multi-source data fusion and dynamic weight adjustment mechanisms, enabling it to provide more comprehensive security assessment results under different scenarios and conditions.

[0004] The approach to achieving the objectives of this invention is based on the security requirements of open architecture software. Considering its multi-layered characteristics and the shortcomings of existing security assessment tools and methods, this invention proposes a security assessment device for different layers of open architecture software. The data acquisition module continuously interacts with other sub-modules during the testing process, collecting and persistently storing security-related data in real time, thus improving the efficiency of security assessment. The environment cleanup module cleans the test environment after the test cycle ends, avoiding excessive resource consumption. This invention proposes a security assessment method for different layers of open architecture software, proposing a comprehensive evaluation model for security test indicators. The security of open architecture software is assessed at three levels: system level, application level, and data level. Through multi-source data fusion technology, security indicator data from different levels are integrated. Combined with a dynamic weight adjustment mechanism, weights are dynamically allocated according to the actual scenario and the importance of the data source, forming a security assessment method tailored to the multi-layered characteristics of open architecture software. This allows software providers and users to obtain a comprehensive security evaluation of the software from different levels according to their own needs, thereby providing a scientific basis for security optimization and risk management.

[0005] The device of the present invention includes a test environment configuration module, a security threat simulation module, an indicator acquisition module, a data storage module, an environment cleanup module, and a data analysis and evaluation module, wherein:

[0006] The test environment configuration module simplifies the test environment configuration process through automation tools and scripts, supports multiple test environment configurations to adapt to different levels of software architecture and different testing needs; it interacts with the software's authentication service to obtain management permissions to perform necessary configurations and operations on the software; it imports a customized test image, installs necessary security testing tools, creates virtual machines according to relevant configuration information, binds an external network floating IP to each virtual machine, and configures the network topology to simulate a real network environment; before the test begins, it selects the test items to be executed and configures the relevant test parameters in detail.

[0007] The security threat simulation module is used to connect to the software under test through the installed security testing tools, according to the test parameters and various configurations, to simulate various network attacks and security threat scenarios.

[0008] The indicator acquisition module is used to perform security testing on the software under test. It collects and integrates security-related indicator data at the system, application, and data levels through a custom script in a periodic polling manner.

[0009] The data storage module is used to store the test parameter configuration and the measured security index data at the three levels into the backend database for persistent storage.

[0010] The environment cleanup module is used to perform a comprehensive cleanup of the test environment, delete virtual machines and related resources created during the test configuration phase, clean up logs, temporary data and temporary files generated during the test, and restore the system to its initial state to avoid excessive consumption of system resources;

[0011] The data analysis and evaluation module is used to analyze the collected security indicator data, calculate the normalized values ​​of the secondary security indicators at each level, calculate the quantitative values ​​of the primary security indicators at each level, and calculate the comprehensive security performance score at each level.

[0012] The steps of the method of the present invention include the following:

[0013] Step 1, Configure the test environment:

[0014] (1a) The test environment configuration module obtains software management permissions by interacting with the software's authentication service, and then interacts with the software through these permissions to obtain the corresponding configuration information. Before testing, it imports the test customized image into the software under test and configures the virtual machine information.

[0015] (1b) The test environment configuration module creates virtual machines based on the test customized image and related configuration information, binds an external network floating IP to each virtual machine, configures the network topology, and installs relevant security testing tools outside the software under test;

[0016] (1c) Before starting the test, the test environment configuration module selects the test items to be executed and configures the test parameters involved in the test items. After configuring the test parameters, the security threat simulation module connects to the software under test through the security testing tool installed outside the software under test according to the test parameters to simulate network attack and security threat scenarios of specified specifications and types.

[0017] Step 2: Collect secondary security performance indicators at the system, application, and data levels.

[0018] The indicator acquisition module performs security testing on the software under test. It collects and integrates secondary security indicator data at the system, application, and data levels through a custom script in a periodic polling manner. The data storage module stores the test parameter configuration and the measured secondary security performance indicator data at the three levels into the backend database for persistent storage.

[0019] Step 3: Calculate the primary indicators that indirectly reflect the software security performance at each level:

[0020] (3a) The data analysis and evaluation module calculates the normalized values ​​of the secondary safety performance indicators at each level during the test according to the following formula:

[0021]

[0022] in, Indicates the first The first level The normalized values ​​of the secondary indicators The closer the value is to 1, the more important the secondary indicator becomes. The better the evaluation, The value is 1, 2, or 3, representing the system layer, application layer, and data layer, respectively. Indicates the first The serial number of the secondary indicator under each level. When the value is 1, 2, or 3, the corresponding The maximum values ​​are 3, 6, and 2 respectively. This indicates the root-finding operation. The symbol ∏ represents the absolute value operation, and ∏ represents the product operation. Indicates the first The first level A set of security performance data tests for each secondary indicator. Represents a set of safety performance data tests China's target The first level The number of times the safety performance data of each secondary indicator was tested. Represents a set of safety performance data tests The serial number of the security performance data test. Indicates the ideal state during the test. The first level The benchmark values ​​for each secondary indicator are the optimal performance values ​​under ideal conditions. This represents the first safety performance data test result obtained under non-ideal conditions. The first level Each secondary indicator value;

[0023] (3b) The data analysis and evaluation module calculates the quantitative value of the primary safety performance index for each level according to the following formula:

[0024]

[0025] in, Indicates the first The first level Each primary indicator value The closer the value is to 1, the higher the level of the primary indicator. The better the evaluation, Indicates the first The sequence number of the first-level indicator under each level When the value is 1, 2, or 3, the corresponding The maximum values ​​are 1, 2, and 3 respectively. Indicates the first The first level A set of secondary indicators under a primary indicator. Represents the set of secondary indicators The serial number of the secondary indicator. When the value is 1, 2, or 3, the corresponding The maximum values ​​are 4, 2, and 4 respectively, and ∑ represents the summation operation. Indicates the first The first level The normalized values ​​of the secondary indicators express The weights are determined by the analytic hierarchy process.

[0026] Step 4, Clean the environment to be tested:

[0027] The environment cleanup module cleans up the test environment, deleting virtual machines and related resources created during the test environment configuration phase, and cleaning up logs, temporary data, and temporary files generated during the test to avoid excessive consumption of system resources.

[0028] Step 5: The data analysis and evaluation module calculates the comprehensive safety performance score for each level according to the following formula:

[0029]

[0030] in, Indicates the first A comprehensive score for the security performance of open architecture software at each level. The larger the value, the more... The higher the overall security performance score of an open architecture software at each level, the better. Indicates the first The set of all first-level indicators at each level. Represents the set of primary indicators The serial number of the first-level indicator. Indicates the first Each primary indicator value express The weights are determined by the analytic hierarchy process (AHP).

[0031] Compared with the prior art, the advantages of this invention are:

[0032] First, because the data storage module in the device of the present invention continuously interacts with other sub-modules during the testing process, it achieves persistent storage of test data, avoiding the trouble of testers having to repeatedly test the tested software in the prior art. This gives the device of the present invention the advantage of significantly improving the efficiency of open architecture software security performance testing.

[0033] Secondly, because the environment cleaning module in the device of the present invention realizes the automated cleaning of the test environment, deletes the virtual machines and related resources created in the test environment configuration stage, and cleans up the logs, temporary data and temporary files generated during the test, it overcomes the problem of excessive system resource consumption in the prior art, so that the device of the present invention has the advantages of lightweight and automated testing.

[0034] Third, because the method of the present invention collects secondary security performance indicators at the system, application, and data levels, and performs data normalization on all secondary indicators and aggregates them into primary indicators that can indirectly reflect the security performance of open architecture software, it overcomes the problem that the existing technology's evaluation indicators for assessing the security performance of open architecture software are too simple and difficult to quantify. This gives the method of the present invention the advantage of quantitatively and comprehensively evaluating the security performance of open architecture software.

[0035] Fourth, because the method of this invention takes into account the system layer, application layer and data layer of open architecture software, it proposes a security performance evaluation method for different layers of open architecture software, and clearly proposes a security performance scoring formula for open architecture software, and evaluates the security performance of each layer. This overcomes the problem that the existing technology for evaluating the security performance of open architecture software does not consider the existence of different layers of software and lacks methods for collecting indicators of different layers. This makes the method of this invention have the advantage of effectively comparing the security performance of different open architecture software. Attached Figure Description

[0036] Figure 1 This is a block diagram of the device of the present invention.

[0037] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings.

[0039] Reference Figure 1 The apparatus of the present invention will be described clearly and completely.

[0040] The device of this invention includes a test environment configuration module, a security threat simulation module, an indicator acquisition module, a data storage module, an environment cleanup module, and a data analysis and evaluation module. Wherein:

[0041] The test environment configuration module interacts with the software's authentication service to obtain management permissions, enabling the import of the customized test image into the software and the configuration of virtual machines. Virtual machines are created based on this configuration information, each bound to a floating external network IP address, and the network topology is configured to simulate a real network environment. Security testing tools are installed outside the software under test. Before testing begins, the test items to be executed are selected, and the relevant test parameters are configured in detail. System-level system security isolation indicators require a test environment containing multiple independent security domains to verify the system's isolation capabilities between different security domains. System-level system security reliability indicators require a test environment with redundancy mechanisms and disaster recovery capabilities to evaluate the system's security assurance capabilities in the event of a failure. Application-level application security protection indicators require a test environment with multi-layered security protection mechanisms to verify the application's defense capabilities against common security attacks. Application-level application security compatibility indicators require a test environment supporting multiple security protocols and standards to evaluate the application's compatibility performance under different security environments. Data security performance indicators at the data layer require the establishment of a test environment with data encryption and access control mechanisms to collect data layer security performance data. Data integrity indicators at the data layer require the establishment of a test environment supporting data verification and anti-tampering mechanisms to verify the integrity assurance capabilities of data during transmission and storage. Data privacy protection indicators at the data layer require the establishment of a test environment with data anonymization and access auditing functions to evaluate the level of privacy protection in different scenarios.

[0042] The security threat simulation module is used to connect to the software under test through the installed security testing tools, based on test parameters and configurations, to simulate various network attacks and security threat scenarios. A load-spreading cluster is formed by installing the security testing tools on three different servers. The main functions of this cluster include sending different types of attack loads for system security performance tests at the system level, application security performance tests at the application level, and data security performance tests at the data level. By running various security attack load assemblies (such as SQL injection attacks, cross-site scripting attacks, and man-in-the-middle attacks), the system security isolation, application security protection capabilities, and data privacy protection capabilities of the open architecture software are tested.

[0043] The indicator acquisition module is used to perform security testing on the software under test. It collects and integrates secondary security performance indicators at the system, application, and data levels through a custom script in a periodic polling manner.

[0044] The data storage module is used to store the test parameter configuration and the measured secondary safety performance index data at the three levels into the backend database for persistent storage.

[0045] The environment cleanup module is used to perform a comprehensive cleanup of the test environment, delete virtual machines and related resources created during the test configuration phase, clean up logs, temporary data and temporary files generated during the test, and restore the system to its initial state to avoid excessive consumption of system resources.

[0046] The data analysis and evaluation module is used to analyze the collected security indicator data, calculate the normalized values ​​of the secondary security indicators at each level, calculate the quantitative values ​​of the primary security indicators at each level, and calculate the comprehensive security performance score for each level.

[0047] Reference Figure 2 The method of the present invention will be described in further detail below.

[0048] The specific steps of the method of the present invention include the following:

[0049] Step 1: Configure the test environment.

[0050] The test environment configuration module interacts with the software's authentication service to obtain software management permissions, and then uses these permissions to interact with the software to obtain the corresponding configuration information. Before testing, it imports the customized test image into the software under test and configures the virtual machine configuration information.

[0051] The virtual machine configuration information includes network interfaces, NTP service, key pairs, security group / firewall rules, and VLANs.

[0052] The test environment configuration module creates virtual machines based on the customized test image and virtual machine configuration information, binds floating IPs from external networks to the virtual machines, configures the network topology of multiple virtual machines, and installs relevant security testing tools on three servers outside the software under test to form a load balancing cluster.

[0053] The test environment configuration module allows users to select the test items to be executed and configure the corresponding test parameters before starting the test. The security threat simulation module, through the installed security testing tools, connects to the software under test according to the test parameters and configurations to simulate various network attacks and security threat scenarios. The security attack load programs in system-level testing are divided into three types: privilege escalation attacks, man-in-the-middle attacks, and denial-of-service attacks. The load programs in application-level testing are divided into SQL injection attacks, cross-site scripting attacks, and API abuse attacks. The load programs in data-level testing are divided into data theft attacks, data tampering attacks, and data integrity attacks.

[0054] The test parameters include operating system version, authentication mechanism, firewall rules, username of the software under test, password of the software under test, application API interface address, application session management mechanism, database transmission encryption method, and database integrity verification mechanism.

[0055] Configuring the test parameters involved in the test items refers to selecting the operating system version from CentOS 7, Ubuntu 20.04, and Windows 7; selecting the authentication mechanism from password-based, certificate-based, and multi-factor authentication; selecting firewall rules from iptables and firewalld; entering the username and password of the software under test; entering the application API interface address; entering the configuration parameters for the application session management mechanism (session timeout); entering the configuration parameters for the encryption method (SSL) for communication between the database and the application; and entering the configuration parameters for the database data integrity verification mechanism (checksum).

[0056] Step 2: Collect secondary security performance indicators at the system, application, and data levels.

[0057] The indicator acquisition module performs security testing on the software under test. It collects and integrates secondary security indicator data at the system, application, and data levels through a custom script in a periodic polling manner. The data storage module stores the test parameter configuration and the measured secondary security performance indicator data at the three levels into the backend database for persistent storage.

[0058] The secondary security performance indicators for the system, application, and data layers are as follows: The secondary security performance indicators for the system layer include identity authentication success rate, firewall rule hit rate, system network isolation effectiveness, and system encryption configuration generation efficiency; the secondary security performance indicators for the application layer include application authentication success rate and application session timeout rate; and the secondary security performance indicators for the data layer include database encrypted communication ratio, database access control interception rate, database integrity verification error rate, and database transmission packet loss rate.

[0059] Step 3: Calculate the primary indicators for each layer that can indirectly reflect the security performance of open architecture software.

[0060] The data analysis and evaluation module calculates the normalized values ​​of the secondary safety performance indicators at each level during the test according to the following formula:

[0061]

[0062] in, Indicates the first The first level The normalized values ​​of the secondary indicators The closer the value is to 1, the more important the secondary indicator becomes. The better the evaluation, The value is 1, 2, or 3, representing the system layer, application layer, and data layer, respectively. Indicates the first The serial number of the secondary indicator under each level. When the value is 1, 2, or 3, the corresponding The maximum values ​​are 3, 6, and 2 respectively. This indicates the root-finding operation. The symbol ∏ represents the absolute value operation, and ∏ represents the product operation. Indicates the first The first level A set of security performance data tests for each secondary indicator. Represents a set of safety performance data tests China's target The first level The number of times the safety performance data of each secondary indicator was tested. Represents a set of safety performance data tests The serial number of the security performance data test. Indicates the ideal state during the test. The first level The benchmark values ​​for each secondary indicator are the optimal performance values ​​under ideal conditions. This represents the first safety performance data test result obtained under non-ideal conditions. The first level The values ​​of each secondary indicator.

[0063] Calculate the weight of each indicator using the following formula:

[0064]

[0065] in, Indicates the first The weight of each indicator, The larger the value, the more representative the indicator is in the current criteria. This indicates the row number of the judgment matrix. This indicates the order of the judgment matrix, which is a relational judgment matrix that has passed the consistency test between different indicators. This indicates the number of columns in the judgment matrix. Represents the first in the judgment matrix Line number The numerical values ​​of column elements, representing indicators. Compared to indicators The importance of.

[0066] The data analysis and evaluation module calculates the quantitative value of the primary safety performance indicator for each level according to the following formula:

[0067]

[0068] in, Indicates the first The first level Each primary indicator value The closer the value is to 1, the higher the level of the primary indicator. The better the evaluation, Indicates the first The sequence number of the first-level indicator under each level When the value is 1, 2, or 3, the corresponding The maximum values ​​are 1, 2, and 3 respectively. Indicates the first The first level A set of secondary indicators under a primary indicator. Represents the set of secondary indicators The serial number of the secondary indicator. When the value is 1, 2, or 3, the corresponding The maximum values ​​are 4, 2, and 4 respectively, and ∑ represents the summation operation. Indicates the first The first level The normalized values ​​of the secondary indicators express The weights are determined by the analytic hierarchy process (AHP).

[0069] The primary security performance indicators for each layer are as follows: the primary indicator for the system layer is the basic system security performance; the primary indicators for the application layer include the basic application security performance and the comprehensive application security performance; and the primary indicators for the data layer include the basic data security performance, data integrity verification, and the effectiveness of data access control.

[0070] The set of secondary indicators is as follows: the system-level set of secondary indicators for basic system security performance; the application-level set of secondary indicators including the application security basic performance set and the application security comprehensive performance set; and the data-level set of secondary indicators including the data security basic performance set, the data integrity verification set, and the data access control effectiveness set.

[0071] The Analytic Hierarchy Process (AHP) is a decision-making method for determining the weights of various criteria in an evaluation model. This method divides the relevant elements of the evaluation model into three elements: objectives, criteria, and schemes, and calculates the weights by constructing a hierarchical model.

[0072] Step 4: Clean the environment to be tested.

[0073] The environment cleanup module cleans up the test environment, deleting virtual machines and related resources created during the test environment configuration phase, and cleaning up logs, temporary data, and temporary files generated during the test to prevent excessive consumption of system resources.

[0074] Step 5: The data analysis and evaluation module calculates the comprehensive safety performance score for each level according to the following formula:

[0075]

[0076] in, Indicates the first A comprehensive score for the security performance of open architecture software at each level. The larger the value, the more... The higher the overall security performance score of an open architecture software at each level, the better. Indicates the first The set of all first-level indicators at each level. Represents the set of primary indicators The serial number of the first-level indicator. Indicates the first Each primary indicator value express The weights are determined by the analytic hierarchy process (AHP).

Claims

1. A security performance evaluation device for different layers of open architecture software, comprising a test environment configuration module, a security threat simulation module, an indicator acquisition module, and a data storage module, characterized in that, It also includes an environmental cleanup module and a data analysis and evaluation module. These modules are sequentially linked along the entire testing process, forming a linear stage chain, in which: The environment cleanup module is used to perform a comprehensive cleanup of the test environment, delete virtual machines and related resources created during the test configuration phase, clean up logs, temporary data and temporary files generated during the test, and restore the system to its initial state to avoid excessive consumption of system resources; The data analysis and evaluation module is used to analyze the collected security indicator data, calculate the normalized values ​​of the secondary security indicators at each level, calculate the quantitative values ​​of the primary security indicators at each level, and calculate the comprehensive security performance score at each level.

2. The security performance evaluation device for different layers of open architecture software according to claim 1, characterized in that, The test environment configuration module is used to simplify the test environment configuration process through automated tools and scripts, supports multiple test environment configurations to adapt to different levels of software architecture and different testing needs; and obtains management permissions by interacting with the software's authentication service to perform necessary configurations and operations on the software. Import the customized test image, install the necessary security testing tools, create virtual machines according to the relevant configuration information, bind an external network floating IP to each virtual machine, and configure the network topology to simulate a real network environment; before the test begins, select the test items to be executed and configure the relevant test parameters in detail. The security threat simulation module is used to connect to the software under test through the installed security testing tools, according to the test parameters and various configurations, to simulate various network attacks and security threat scenarios. The indicator acquisition module is used to perform security testing on the software under test. It collects and integrates security-related indicator data at the system, application, and data levels through a custom script in a periodic polling manner. The data storage module is used to store the test parameter configuration and the measured security index data at the three levels into the backend database for persistent storage.

3. A method for evaluating the security performance of open architecture software at different levels, characterized in that, The evaluation method involves collecting secondary security performance indicators at three levels: system, application, and data. Primary indicators at each level are then calculated to indirectly reflect the security performance of open architecture software. A comprehensive security performance score for each level is calculated. The specific steps of this evaluation method are as follows: Step 1): Configure the test environment; Step 2): Collect secondary security performance indicators at three levels: system, application, and data; Step 3): Calculate the primary indicators that can indirectly reflect the software security performance at each level; Step 4): Clean the environment to be tested; Step 5): Calculate the overall security performance score for each level.

4. The method for evaluating the security performance of open architecture software at different levels according to claim 3, characterized in that, Step 1) specifically includes: Step 1a) The test environment configuration module interacts with the software's authentication service to obtain software management permissions, and then uses these permissions to interact with the software to obtain the corresponding configuration information. Before the test, it imports the customized test image into the software under test and configures the virtual machine information. The virtual machine configuration information includes network interface, NTP service, key pair, security group / firewall rules, and VLAN. Step 1b) The test environment configuration module creates virtual machines based on the customized test image and related configuration information, binds an external network floating IP to each virtual machine, configures the network topology, and installs relevant security testing tools outside the software under test; Step 1c) Before starting the test, the test environment configuration module selects the test items to be executed and configures the test parameters involved in the test items. After the test parameters are configured, the security threat simulation module connects to the software under test through the security testing tool installed outside the software under test according to the test parameters, and simulates network attack and security threat scenarios of specified specifications and types.

5. The method for evaluating the security performance of open architecture software at different levels according to claim 3, characterized in that, In step 2), the indicator acquisition module performs security testing on the software under test. It periodically polls and collects and integrates secondary security indicator data at the system, application, and data levels using a custom script. The data storage module stores the test parameter configuration and the measured secondary security performance indicator data at the three levels into the backend database for persistent storage. The secondary security performance indicators at the system, application, and data levels are as follows: System-level secondary security performance indicators include authentication success rate, firewall rule hit rate, system network isolation effectiveness, and system encryption configuration generation efficiency; application-level secondary security performance indicators include application authentication success rate and application session timeout rate; and data-level secondary security performance indicators include database encrypted communication ratio, database access control interception rate, database integrity verification error rate, and database transmission packet loss rate.

6. The method for evaluating the security performance of open architecture software at different levels according to claim 3, characterized in that, Step 3) specifically includes: Step 3a) The data analysis and evaluation module calculates the normalized values ​​of the secondary safety performance indicators at each level during the test according to the following formula: ,in, Indicates the first The first level The normalized values ​​of the secondary indicators The value is 1, 2, or 3, representing the system layer, application layer, and data layer, respectively. Indicates the first The serial number of the secondary indicator under each level. This indicates the root-finding operation. The symbol ∏ represents the absolute value operation, and ∏ represents the product operation. Indicates the first The first level A set of security performance data tests for each secondary indicator. Represents a set of safety performance data tests China's target The first level The number of times the safety performance data of each secondary indicator was tested. Represents a set of safety performance data tests The serial number of the security performance data test. Indicates the ideal state during the test. The first level The benchmark values ​​for each secondary indicator are the optimal performance values ​​under ideal conditions. This represents the first safety performance data test result obtained under non-ideal conditions. The first level Each secondary indicator value; Step 3b) The data analysis and evaluation module calculates the quantitative value of the primary safety performance indicator for each level according to the following formula: ,in, Indicates the first The first level Each primary indicator value Indicates the first The sequence number of the first-level indicator under each level Indicates the first The first level A set of secondary indicators under a primary indicator. Represents the set of secondary indicators The index of the secondary indicator, ∑ represents the summation operation. Indicates the first The first level The normalized values ​​of the secondary indicators express The weights are determined by the analytic hierarchy process (AHP).

7. The method for evaluating the security performance of open architecture software at different levels according to claim 3, characterized in that, Step 4) The environment cleanup module cleans up the test environment, deletes virtual machines and related resources created during the test environment configuration phase, and cleans up logs, temporary data and temporary files generated during the test to avoid excessive consumption of system resources.

8. The method for evaluating the security performance of open architecture software at different levels according to claim 3, characterized in that, In step 5), the data analysis and evaluation module calculates the comprehensive safety performance score for each level according to the following formula: ,in, Indicates the first A comprehensive score for the security performance of open architecture software at each level. Indicates the first The set of all first-level indicators at each level. Represents the set of primary indicators The serial number of the first-level indicator. Indicates the first Each primary indicator value express The weights are determined by the analytic hierarchy process (AHP).

9. The method for evaluating the security performance of open architecture software at different levels according to claim 4, characterized in that, The test parameters in step 1c) include operating system version, authentication mechanism, firewall rules, username of the software under test, password of the software under test, application API interface address, application session management mechanism, database transmission encryption method, and database integrity verification mechanism; Configuring the test parameters involved in the test items refers to selecting the operating system version from CentOS 7, Ubuntu 20.04, and Windows 7; selecting the authentication mechanism from password-based, certificate-based, and multi-factor authentication; selecting firewall rules from iptables and firewalld; entering the username and password of the software under test; entering the application API interface address; entering the configuration parameters for the application session management mechanism; entering the configuration parameters for the encryption method of communication between the database and the application; and entering the configuration parameters for the database data integrity verification mechanism.

10. A method for evaluating the security performance of open architecture software at different levels according to claim 6, characterized in that, The primary security performance indicators for each level mentioned in step 3b) are as follows: the primary indicator for the system level is the basic system security performance; the primary indicators for the application level include the basic application security performance and the comprehensive application security performance; and the primary indicators for the data level include the basic data security performance, data integrity verification, and the effectiveness of data access control. The set of secondary indicators is as follows: a set of secondary indicators for basic system security performance at the system level; a set of secondary indicators for application security performance at the application level, including a set of secondary indicators for basic application security performance and a set of secondary indicators for comprehensive application security performance; and a set of secondary indicators for data security performance at the data level, including a set of secondary indicators for basic data security performance, a set of secondary indicators for data integrity verification, and a set of secondary indicators for the effectiveness of data access control. The Analytic Hierarchy Process (AHP) is a decision-making method for determining the weights of various criteria in an evaluation model. This method divides the relevant elements of the evaluation model into three elements: objectives, criteria, and schemes, and calculates the weights by constructing a hierarchical model.

Images