Security application evaluation method, device, electronic device and storage medium

Abstract

The present application provides a security application evaluation method, device, electronic device and storage medium, which relates to the field of computer technology and can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. The method includes: based on the application type of the target security application to be evaluated, selecting at least one target protection test tool corresponding to the application type from the tool library; for at least one target protection test tool, performing the following operations respectively: running a target protection test tool to perform a protective test on the device installed with the target security application, and receiving the processing status returned by the device after the target security application processes the test information of a target protection test tool; based on the at least one processing status obtained, determining the protection capability evaluation result of the target security application. The present application can determine the specific defects of the security application so as to optimize and update the security application in a targeted manner.

Description

Background Art

[0002] With the continuous development of network security technology, various security software emerges in an endless stream, such as firewalls, terminal security software, intrusion detection software, etc. The protection capability of security software is very important for the security protection of devices. Generally, in order to determine the protection capability of security software, you can evaluate the security software through a credible third-party evaluation agency to obtain the evaluation results.

[0003] However, the evaluation results of third-party evaluation agencies are usually relatively simple, such as: strong protection capability, weak protection capability, etc., which cannot reflect the specific defects of the security software, and thus cannot optimize and update the security software in a targeted manner. Summary of the invention

[0004] The embodiments of the present application provide a security application evaluation method, device, electronic device and storage medium for evaluating security applications and determining specific defects of security applications so as to optimize and update the security applications in a targeted manner.

[0005] On the one hand, an embodiment of the present application provides a security application assessment method, including:

[0006] Based on the application type of the target security application to be evaluated, selecting at least one target protection test tool corresponding to the application type from a tool library, wherein the tool library includes a plurality of different protection test tools;

[0007] For the at least one target protection test tool, the following operations are respectively performed: running a target protection test tool to perform a protection test on a device installed with the target security application, and receiving a processing status returned by the device after the target security application processes the test information of the target protection test tool;

[0008] Based on the obtained at least one processing status, a protection capability evaluation result of the target security application is determined.

[0009] On the one hand, an embodiment of the present application provides a security application assessment method, including:

[0010] When receiving the test information of the target protection test tool sent by the test device, the test information is processed by the target security application to obtain a corresponding processing status; wherein the target protection test tool is selected by the test device from a tool library including a plurality of different protection test tools based on the application type of the target security application;

[0011] The corresponding processing status is sent to the testing device.

[0012] On the one hand, an embodiment of the present application provides a security application evaluation device, including:

[0013] A selection module, configured to select, based on an application type of a target security application to be evaluated, at least one target protection test tool corresponding to the application type from a tool library, wherein the tool library includes a plurality of different protection test tools;

[0014] The attack module is used to perform the following operations respectively for the at least one target protection test tool: run a target protection test tool to perform a protection test on the device installed with the target security application, and receive the processing status returned by the device after the target security application processes the test information of the target protection test tool;

[0015] The capability determination module is used to determine the protection capability evaluation result of the target security application based on at least one obtained processing state.

[0016] In a possible embodiment, the device further includes an acquisition module, configured to obtain the tool library in the following manner:

[0017] Collect multiple security events, and extract corresponding protection test tools from the multiple security events respectively;

[0018] Deduplication operations are performed on the extracted multiple protection test tools to obtain multiple different protection test tools;

[0019] The plurality of different protection testing tools are saved in the tool library.

[0020] In a possible embodiment, the application type includes an operating system type and a protection type; and the selection module is further used to:

[0021] Based on the operating system type of the target security application, selecting a plurality of candidate protection test tools corresponding to the operating system type from a tool library;

[0022] Based on the protection type of the target security application, at least one target protection testing tool corresponding to the protection type is selected from the plurality of candidate protection testing tools.

[0023] In a possible embodiment, if the target security application is used to intercept the target protection test tool, then the processing state corresponding to each target protection test tool includes at least one of the following:

[0024] Interception state, detection state, perception state, and no perception state;

[0025] The capability determination module is also used for:

[0026] Determine, based on at least one obtained processing state, the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived;

[0027] Based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived, the protection capability evaluation result of the target security application is determined.

[0028] In a possible embodiment, if the target security application is used to detect a target protection test tool, then the processing state corresponding to each target protection test tool includes at least one of the following:

[0029] Detection state, perception state, no perception state;

[0030] The capability determination module is also used for:

[0031] Determining, based on the obtained at least one processing state, a number of target protection test tools detected by the target security application, a number of target protection test tools perceived, and a number of target protection test tools not perceived;

[0032] Based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools, a protection capability evaluation result of the target security application is determined.

[0033] On the one hand, an embodiment of the present application provides a security application evaluation device, including:

[0034] A processing module, configured to, upon receiving test information of a target protection test tool sent by a test device, process the test information through a target security application to obtain a corresponding processing status; wherein the target protection test tool is selected by the test device from a tool library including a plurality of different protection test tools based on an application type of the target security application;

[0035] The sending module is used to send the corresponding processing status to the testing device.

[0036] In a possible embodiment, if the target security application is used to intercept the target protection test tool, the processing module is further used to:

[0037] The target security application intercepts the test information to obtain an interception status; or,

[0038] The target security application detects and processes the test information of the target protection test tool to obtain a detection status; or,

[0039] The test information is sensed and processed by the target security application to obtain a sensed state or a non-sense state.

[0040] In a possible embodiment, if the target security application is used to detect the target protection test tool, the processing module is further used to:

[0041] The target security application detects and processes the test information to obtain a detection status; or,

[0042] The test information is sensed and processed by the target security application to obtain a sensed state or a non-sense state.

[0043] On the one hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores a program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above-mentioned security application assessment methods.

[0044] On the one hand, an embodiment of the present application provides a computer storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the steps of any of the above-mentioned security application assessment methods.

[0045] On the one hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; when a processor of an electronic device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, so that the electronic device performs the steps of any one of the above-mentioned security application assessment methods.

[0046] Since the embodiment of the present application adopts the above technical solution, it has at least the following technical effects:

[0047] When evaluating a target security application, the embodiment of the present application first selects multiple target protection test tools supported by the target security application from a tool library based on the application type of the target security application, that is, multiple target protection test tools that the target security application needs to process, and uses multiple target protection test tools to perform protective tests on the devices installed with the target security application respectively to obtain the processing status of the target security application after processing each target protection test tool; finally, based on the multiple processing statuses obtained, the protection capability evaluation result of the target security application is determined.

[0048] In the above scheme, based on the processing status of the target security application for multiple target protection test tools, the protection capability of the target security application for multiple target protection test tools can be determined, for example: which target protection test tools have stronger protection capabilities and which target protection test tools have weaker protection capabilities, thereby determining the specific defects of the security application so that the security application can be optimized and updated in a targeted manner.

[0049] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A schematic diagram of an application scenario of a security application assessment method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart for establishing a tool library provided in an embodiment of the present application;

[0053] Figure 3 A flowchart for establishing another tool library provided in an embodiment of the present application;

[0054] Figure 4 A signaling interaction flow chart of a security application assessment method provided in an embodiment of the present application;

[0055] Figure 5A A flowchart of a security application evaluation method provided in an embodiment of the present application;

[0056] Figure 5B A logical schematic diagram of a security application assessment method provided in an embodiment of the present application;

[0057] Figure 6 A flowchart of a method for evaluating secure applications on a test device side provided in an embodiment of the present application;

[0058] Figure 7 A flowchart of another security application evaluation method provided in an embodiment of the present application;

[0059] Fig. 8AA flowchart of another security application evaluation method provided in an embodiment of the present application;

[0060] Figure 8B A flowchart of another security application evaluation method provided in an embodiment of the present application;

[0061] Fig. 9 A flowchart of a method for evaluating a secure application on a device under test provided in an embodiment of the present application;

[0062] Fig.10 A structural block diagram of a security application evaluation device provided in an embodiment of the present application;

[0063] Fig.11 A structural block diagram of another security application evaluation device provided in an embodiment of the present application;

[0064] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0065] Fig.13 It is a schematic diagram of the structure of another electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0067] In order to help those skilled in the art better understand the technical solution of the present application, some concepts involved in the present application are introduced below.

[0068] Protection test tool: refers to the attack tool used, developed, and customized by attackers to attack devices. Attack tools can include various types, such as: vulnerability exploitation tools, traffic packets, command lines, attack texts, etc.

[0069] Security application: software installed in a device to intercept, detect, and process attack tools when the device is attacked by attack tools; can include various types of software, such as firewall software, terminal security software, intrusion detection software, etc.

[0070] The word “exemplary” is used hereinafter to mean “serving as an example, example, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0071] The terms "first" and "second" in this document are used for descriptive purposes only and should not be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0072] The following is a brief introduction to the design concept of the embodiment of the present application:

[0073] In the related art, when determining the protection capability of security software, the security software is evaluated by a credible third-party evaluation agency to obtain the evaluation results. However, the evaluation results of the third-party evaluation agency are usually relatively simple, such as: strong protection capability, weak protection capability, etc., which cannot reflect the specific defects of the security software, and thus cannot optimize and update the security software in a targeted manner.

[0074] In view of this, the embodiments of the present application provide a security application evaluation method, device, electronic device and storage medium. Based on the processing status of the target security application after processing multiple target protection test tools, it can be determined which target protection test tools the target security application has stronger protection capabilities against and which target protection test tools the target security application has weaker protection capabilities against, thereby determining the specific defects of the security application so as to optimize and update the security application in a targeted manner.

[0075] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0076] like Figure 1 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes multiple devices under test 11 and a test device 12, and the devices under test 11 and the test device 12 can communicate with each other through a communication network.

[0077] In an optional implementation, the communication network may be a wired network or a wireless network.

[0078] In the embodiment of the present application, the tested device 11 can be a terminal device or a server; the test device 12 can also be a terminal device or a server; wherein the terminal devices include but are not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals and other devices; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0079] The security application evaluation method in the embodiment of the present application can be jointly executed by the tested device 11 and the test device 12. In a specific implementation, the target security application to be evaluated can be installed in the tested device 11. When evaluating the target security application, the test device 12 can select multiple target protection test tools that match the application type from the tool library based on the application type of the target security application to be evaluated, and then respectively run multiple target protection test tools to send test information to the tested device 11; each time the tested device 11 receives test information from a target protection test tool, the test information can be processed by the target security application, such as interception or detection, and the processing status of each target protection test tool processed by the target security application is sent to the test device 12; the test device 12 can determine the protection capability evaluation result of the target security application based on the processing status of the target security application on multiple target protection test tools.

[0080] It should be noted that Figure 1 The figure is only an example. In fact, the number of tested devices and testing devices is not limited, and is not specifically limited in the embodiments of the present application.

[0081] The following describes the safety application assessment method provided by the exemplary embodiment of the present application in combination with the application scenarios described above, with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown for the purpose of facilitating the understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect. In fact, the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0082] Before introducing the security application assessment method of the embodiment of the present application, the process of establishing the tool library used in the method is first introduced.

[0083] In some embodiments, Figure 2As shown, the tool library can be obtained through the following steps:

[0084] S201. Collect multiple security events, and extract corresponding protection testing tools from the multiple security events.

[0085] S202: De-duplicate the extracted multiple protection test tools to obtain multiple different protection test tools.

[0086] S203. Save a plurality of different protection test tools to a tool library.

[0087] Among them, multiple security incidents can come from security incidents that have occurred in the real environment and have been made public with a strong timeliness, such as some public security incidents on official websites, media, WeChat public accounts, Weibo, etc. Each security incident can include the process of using corresponding attack tools to attack related devices.

[0088] It is understandable that in the specific implementation of the present application, the collection, use and processing of various security events need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0089] For each security event, the attack features in the security event can be extracted, and the corresponding attack tools can be determined according to the attack features. For example, the attack features include but are not limited to: vulnerability exploit features, attack file features, traffic features, command line features, etc. The attack tool in each security event is used as a protection test tool in the embodiment of the present application.

[0090] Among them, vulnerability exploitation is to find an easy-to-attack vulnerability in the target system, and then use the vulnerability to obtain permissions, thereby achieving control over the target system. Usually, vulnerability exploitation features can include extracting POC (Proof of Concept) information or EXP (Exploit) information of the vulnerability. POC is a description or an attack sample, and EXP is a detailed description of how to exploit the vulnerability or a demonstrated vulnerability attack code, including the mechanism of the vulnerability and the method of exploitation.

[0091] The attack file features are usually expressed as executable files or scripts, and the executable files or scripts are used to perform certain permission operations on the attacked device, or to induce external users to access the executable files or scripts to achieve the purpose of the attack.

[0092] Traffic characteristics can include bandwidth attack characteristics, application attack characteristics, etc. The general form of bandwidth attack characteristics is that a large number of seemingly legitimate TCP (Transmission Control Protocol), UDP (User Datagram Protocol) or ICMP (Internet Group Message Protocol) data packets are transmitted to a specific destination; application attack characteristics use the behaviors defined by protocols such as TCP and HTTP to continuously occupy computing resources to block them from processing normal transactions and requests.

[0093] The command line feature may include the command line of the user attack. The command line generally refers to the command prompt, that is, a working prompt in the operating system that prompts for command input.

[0094] It should be noted that the above-mentioned various attack feature types are only exemplary, and in fact other attack feature types may also be included, which is not limited in the embodiments of the present application.

[0095] After obtaining the protection test tools corresponding to multiple security events through the above-mentioned various attack features, considering that the multiple protection test tools obtained may be the same, the multiple protection test tools obtained can be deduplicated.

[0096] In some optional embodiments, such as Figure 3 As shown in the figure, assuming that multiple protection test tools include vulnerability exploit features, attack file features, traffic features, and command line features, the following steps can be used to deduplicate the multiple protection test tools:

[0097] S301, traversal protection test tool;

[0098] S302, determine whether the traversal is completed; if not, execute S303, if yes, end the traversal.

[0099] S303, determine whether the protection test tool is an attack file feature; if not, execute S304, if yes, execute S308.

[0100] S304, determine whether the protection test tool is a vulnerability exploit feature; if not, execute S304, if yes, execute S307.

[0101] S305, determine whether the protection test tool is a command line feature; if not, execute S304; if yes, execute S307.

[0102] S306, determine whether the protection test tool is a traffic feature; if yes, execute S307.

[0103] S307, saving the attack feature as an attack text;

[0104] In this step, for vulnerability exploitation features, the POC information or EXP information can be saved as attack text; for command line features, the command line can be saved as attack text; for traffic features, data packets such as TCP, UDP or ICMP, or behavior information related to TCP and HTTP protocols can be saved as attack text.

[0105] S308, determining whether there is an attack file or attack text in the tool library, if not, executing S309, if yes, returning to S301.

[0106] S309, storing the attack file or attack text into the tool library.

[0107] After introducing the process of establishing the tool library, the specific implementation method of the security application assessment method of the embodiment of the present application is introduced below.

[0108] See also Figure 4 As shown, the embodiment of the present application provides a security application evaluation method, which can be performed by a testing device and a tested device in cooperation, including the following steps:

[0109] S401, a testing device selects at least one target protection testing tool corresponding to an application type from a tool library based on the application type of a target security application to be evaluated, wherein the tool library includes a plurality of different protection testing tools.

[0110] The test device may be a terminal device or a server, which has a tool library stored in advance, and the tool library can be obtained by collecting various protection test tools. The target security application may be various types of security software, such as firewall software, terminal security software, intrusion detection software, etc., which are not limited here.

[0111] Since the application types of security applications can be diverse, the protection test tools that need to be processed by each application type of security application may also be different. Therefore, it is necessary to select multiple target protection test tools that need to be processed by the target security application from the tool library based on the application type of the target security application.

[0112] For example, application types may include operating system types and protection types. Operating system types include Windows operating system, Linux operating system, etc. Security applications of each operating system type may also include multiple protection types, such as firewall type, intrusion detection type, virus detection type, etc.

[0113] Optionally, in S401, based on the protection type of the target security application to be evaluated, selecting at least one target protection test tool corresponding to the protection type from the tool library may include the following steps:

[0114] A1. Based on the operating system type of the target security application, select multiple candidate protection testing tools corresponding to the operating system type from the tool library.

[0115] Among them, the security application of each operating system type can support the protection test tools under the same operating system type. For example, the security application of the Windows operating system supports the protection test tools under the Windows operating system, but does not need to support the protection test tools under the Linux operating system.

[0116] For example, if the operating system type of the target security application is a Windows operating system, multiple candidate protection testing tools under the Windows operating system are selected from the tool library.

[0117] A2. Based on the protection type of the target security application, select at least one target protection test tool corresponding to the protection type from multiple candidate protection test tools.

[0118] For security applications of different protection types, the protection test tools they need to process may be the same or different. For example, firewall-type security applications need to process network attack-type protection test tools; intrusion detection-type security applications need to process intrusion-type protection test tools; virus detection-type security applications need to process virus-type protection test tools, etc. Among them, for each protection test tool, it can be determined whether it belongs to the network attack type, intrusion type, or virus type by analyzing the specific content of its attack characteristics.

[0119] S402: The testing devices respectively run at least one target protection testing tool to perform a protection test on the tested device on which the target security application is installed.

[0120] In this step, for each target protection test tool, a corresponding test method can be used to perform a protection test on the device under test. For example, if a target protection test tool is a command line, the specific content of the command line is sent to the device under test; for another example, if a target protection test tool is an attack file, the specific content of the attack file is sent to the device under test.

[0121] It should be noted that when the test device selects multiple target protection test tools corresponding to the target security application, it can be achieved by traversing the tool library. Optionally, the test device can run the target protection test tool to perform a protective test on the tested device with the target security application installed each time it traverses to a target protection test tool; or after traversing to obtain multiple target protection test tools, it can run multiple target protection test tools separately to perform a protective test on the tested device with the target security application installed, which is not limited in the embodiments of the present application.

[0122] S403, each time the tested device receives test information from a target protection test tool, it processes the test information through a target security application to obtain a corresponding processing status.

[0123] Among them, the test information of the target protection test tool can be an attack behavior launched against the tested device, and the function of the target security application can be to intercept the target protection test tool, that is, to block the test information of the target protection test tool. At this time, the target security application first needs to perceive the test information of the target protection test tool, that is, to record the test information of the target protection test tool; if the test information of the target protection test tool can be perceived, the test information can be detected, and if the test information can be detected, that is, it is found that the test information is abnormal information, the test information can be further intercepted; for example: for some virus detection type security applications, it is necessary to intercept the detected virus test information; for some firewall type security applications, its function can be to intercept traffic test information.

[0124] In addition, the function of the target security application can also be to detect the target protection test tool, that is, to detect that the test information of the target protection test tool is abnormal information; for example, for some intrusion detection type security applications, its function can be to detect intrusion test information.

[0125] In some possible embodiments, if the function of the target security application is to intercept the target protection test tool, then in S403, the test information is processed by the target security application to obtain a corresponding processing status, which may include the following three possible situations:

[0126] In the first case, the target security application intercepts and processes the test information of the target protection test tool to obtain the interception status.

[0127] In this case, the target security application first perceives the test information of the target protection test tool, that is, records the test information, further detects the test information, and intercepts it when it is detected that the test information is abnormal information to obtain the interception status. At this time, the corresponding status bit can be set for the target protection test tool, for example, the status bit is 1, to indicate that the target security application can intercept the target protection test tool.

[0128] In the second case, the test information of the target protection test tool is detected and processed by the target security application to obtain the detection status.

[0129] In this case, the target security application first perceives the test information of the target protection test tool, and then detects the test information. It can detect that the test information is abnormal information, but the test information cannot be further intercepted, and thus a detection status is obtained; at this time, a corresponding status bit can be set for the target protection test tool, for example, the status bit is 2, to indicate that the target security application can detect the target protection test tool.

[0130] In the third case, the test information of the target protection test tool is perceived and processed by the target security application to obtain a perceived state or a non-perceived state.

[0131] In this case, on the one hand, the target security application perceives the test information of the target protection test tool, but cannot detect that the test information is abnormal information, so it obtains a perception state. At this time, a corresponding status bit can be set for the target protection test tool, for example, the status bit is 3, to indicate that the target security application can perceive the target protection test tool. On the other hand, the target security application does not perceive the test information of the target protection test tool, that is, it does not record the test information, and obtains an unaware state. At this time, a corresponding status bit can be set for the target protection test tool, for example, the status bit is -1, to indicate that the target security application has no protection capability for the target protection test tool.

[0132] In this embodiment, since the function of the target security application is to intercept the target protection test tool, if the target security application can intercept the test information of the target protection test tool, it means that the protection capability (i.e., protection capability) for the protection test tool is strong; if the test information of the target protection test tool can be detected but the test information cannot be intercepted, it means that the protection capability for the protection test tool is weak; if the test information of the target protection test tool can be perceived but the test information cannot be detected, it means that the protection capability for the protection test tool is very weak; if the test information of the target protection test tool cannot be perceived, it means that there is no protection capability for the protection test tool.

[0133] In some other possible embodiments, the function of the target security application may also be to detect the target protection test tool. In the above S403, the target security application processes the test information to obtain a corresponding processing status, which may include the following two possible situations:

[0134] Case 1: The test information of the target protection test tool is detected and processed through the target security application to obtain the detection status.

[0135] In this case, the target security application first senses the test information of the target protection test tool. After sensing the test information, the test information is detected and the test information can be detected as abnormal information. However, the test information cannot be further intercepted, thereby obtaining a detection status. At this time, a corresponding status bit can be set for the target protection test tool, for example, the status bit is 2, to indicate that the target security application can detect the target protection test tool.

[0136] Case 2: The test information of the target protection test tool is perceived and processed through the target security application to obtain a perceived state or a non-perceived state.

[0137] In this case, on the one hand, the target security application perceives the test information of the target protection test tool, but cannot detect that the test information is abnormal information, and thus obtains a perception state. At this time, a corresponding status bit can be set for the target protection test tool, such as a status bit of 3, to indicate that the target security application can perceive the target protection test tool. On the other hand, the target security application does not perceive the test information of the target protection test tool and obtains an unaware state. At this time, a corresponding status bit can be set for the target protection test tool, such as a status bit of -1, to indicate that the target security application has no protection capability for the target protection test tool.

[0138] In this embodiment, since the function of the target security application is to detect the target protection test tool, if the target security application can detect the target protection test tool, it means that the protection capability for the protection test tool is strong; if the target protection test tool can be perceived but cannot be detected, it means that the protection capability for the protection test tool is weak; if the target protection test tool cannot be perceived, it means that there is no protection capability for the protection test tool.

[0139] S404, the tested device sends the processing status of the target security application processing the test information of each target protection test tool to the testing device.

[0140] Optionally, the device under test may obtain one processing status each time and send the processing status to the test device, or after obtaining multiple processing statuses, send the multiple processing statuses to the test device at the same time, which is not limited in this embodiment of the present application.

[0141] S405: The testing device determines a protection capability evaluation result of the target security application based on the obtained multiple processing states.

[0142] It can be seen from the above embodiments of the present application that the processing state of the target security application for a target protection test tool can be an interception state, a detection state, a perception state or an unaware state. According to the processing state of the target security application for each target protection test tool, the protection capability of the target security application for each target protection test tool can be determined. For example, when the corresponding status bit indicates the strength of the protection capability, the more status bits with strong protection capability, the stronger the protection capability of the target security application. Furthermore, when the protection capability of the target security application for some target protection test tools is not strong, the target security application can be updated and improved in a targeted manner.

[0143] In some embodiments, if the function of the target security application is to intercept the target protection test tool, the above S405, based on the obtained at least one processing state, determines the protection capability evaluation result of the target security application, which may include the following steps:

[0144] B1. Based on at least one obtained processing state, determine the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived.

[0145] Optionally, the processing status of the target security application for each target protection test tool can be represented by a corresponding status bit. For example, the status bit corresponding to the interception status is 1, the status bit corresponding to the detection status is 2, the status bit corresponding to the perception status is 3, and the status bit corresponding to the unaware status is -1. By counting the number of different status bits, the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived can be determined.

[0146] It should be noted that the above-mentioned status bit settings are merely exemplary and are not limited in the embodiments of the present application.

[0147] B2. Determine the protection capability assessment result of the target security application based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived.

[0148] For example, the more target protection test tools intercepted by the target security application, the more target protection test tools detected, and the more target protection test tools perceived, the stronger the protection capability of the target security application, where the ability to intercept target protection test tools > the ability to detect target protection test tools > the ability to perceive target protection test tools. On the contrary, the more target protection test tools that are not perceived, the weaker the protection capability of the target security application.

[0149] In addition, after determining the number of target protection testing tools for each of the above processing states, the overall proportion of target protection testing tools for each processing state can also be determined, and the protection capability assessment result of the target security application can be determined based on the overall proportion of target protection testing tools for each processing state.

[0150] In other embodiments, if the function of the target security application is to detect a target protection test tool, determining the protection capability evaluation result of the target security application based on the obtained at least one processing state in S405 may include the following steps:

[0151] C1. Based on at least one obtained processing state, determine the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of non-perceived target protection test tools.

[0152] Optionally, the processing status of the target security application for each target protection test tool can be represented by a corresponding status bit. For example, the status bit corresponding to the detection status is 2, the status bit corresponding to the perception status is 3, and the status bit corresponding to the non-perception status is -1. By counting the number of different status bits, the number of target protection test tools detected by the target security application, the number of target protection test tools perceived, and the number of target protection test tools not perceived can be determined.

[0153] C2. Determine the protection capability assessment result of the target security application based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools.

[0154] For example, the more target protection test tools detected by the target security application and the more target protection test tools perceived, the stronger the protection capability of the target security application, where the ability to detect the target protection test tools is greater than the ability to perceive the target protection test tools. On the contrary, the more target protection test tools are not perceived, the weaker the protection capability of the target security application.

[0155] Furthermore, for the target protection test tool that needs to be intercepted, if the target security application can detect but not intercept the target protection test tool, the target security application can be optimized and improved in a targeted manner. Similarly, if the target security application can sense but not detect the target protection test tool, or cannot sense the target protection test tool, the target security application can be optimized and improved in a targeted manner.

[0156] For the target protection test tool that needs to be detected, if the target security application can perceive but cannot detect the target protection test tool, or cannot perceive the target protection test tool, the target security application can be optimized and improved in a targeted manner.

[0157] Combine the following Figure 5A and Figure 5B The security application evaluation method of the embodiment of the present application is exemplarily introduced.

[0158] Figure 5A A logical schematic diagram of a security application assessment method provided in an embodiment of the present application is shown.

[0159] like Figure 5A As shown, assuming that the target security application is a security application of the Windows operating system, and the protection type of the security application is a virus detection type, when evaluating the target security application, the test device first selects multiple candidate protection test tools under the Windows operating system from a pre-established tool library, and then selects multiple target protection test tools that the virus detection type security application needs to process from the multiple candidate protection test tools.

[0160] Next, the test device runs each target protection test tool to perform a protective test on the device under test where the security application is located. The device under test processes the test information of each target protection test tool through the security application, and sends the processing status of each target protection test tool by the security application to the test device. Then, the test device evaluates the protection capability of the target security application based on the processing status of each target protection test tool by the target security application.

[0161] Above Figure 5A First, multiple target protection test tools for evaluating the target security application are selected from the tool library, and then the protection capabilities of the target security application against the multiple target protection test tools are determined respectively; in actual applications, the tool library can also be traversed, and each time a target protection test tool is traversed, the protection capability of the target security application against the target protection test tool is determined, for example, as follows Figure 5B shown.

[0162] like Figure 5BAs shown, assuming that the function of the target security application to be evaluated is to intercept the target protection test tool, the evaluation method may include the following steps:

[0163] S501, traverse all protection test tools in the tool library;

[0164] S502, determine whether the traversal is completed; if not, execute S503, if yes, terminate the execution.

[0165] S503, determine whether the target security application to be evaluated needs to support the protection test tool; if it does, continue to execute S504; if it does not need to support, execute S0508.

[0166] For example, a target security application for Windows systems does not need to support protection testing tools for Linux systems; in addition, a target security application for virus detection and removal does not need to support protection testing tools for attacking firewall-type security applications.

[0167] S504, determining whether the target security application can intercept the protection test tool; if it can be intercepted, executing S509; if it cannot be intercepted, executing S505.

[0168] S505, determining whether the target security application can detect the protection test tool; if it can, executing S510; if it cannot, executing S506.

[0169] S506, determine whether the target security application can perceive the protection test tool; if it can, execute S511; if it cannot, execute S507.

[0170] S507, setting the status bit of the protection test tool to: status = -1, to indicate that the target security application does not support the protection test tool.

[0171] S508, setting the status bit of the protection test tool to: status=0, to indicate that the target security application does not need to support the protection test tool, and returning to S502.

[0172] S509: Set the status bit of the protection test tool to: status=1, to indicate that the target security application can intercept the protection test tool.

[0173] Among them, the interception protection test tool refers to the target security application being able to discover and block the test information of the protection test tool.

[0174] S510, setting the status bit of the protection test tool to: status=2, to indicate that the target security application can detect the protection test tool.

[0175] Detecting a protection test tool means that although the target security application does not block the test information of the protection test tool, it can find that the test information of the protection test tool is abnormal information and has exposed target security application performance, that is, recording the abnormal information of the protection test tool to remind relevant personnel to handle it in time.

[0176] S511, setting the status bit of the protection test tool to: status=3, to indicate that the target security application can perceive the protection test tool.

[0177] Among them, perception of the protection test tool means that the target security application cannot find that the test information of the protection test tool is abnormal, and there is no exposed target security application performance to the protection test tool, but the log behavior of the protection test tool is recorded, and the test information of the protection test tool can only be traced back through the log behavior.

[0178] Furthermore, after obtaining the processing status of the target security application for each target protection test tool (which can be determined based on the status bit), the number of target protection test tools in each processing state can be determined, and the overall proportion of each processing state can be calculated. The more protection test tools that can intercept, detect, and perceive, the stronger the protection capability, and vice versa.

[0179] If the target security application can intercept, detect or perceive a certain protection test tool, a corresponding protection test tool probe is assigned to the target security application. The protection test tool probe represents the rules or functional points used by the target security application to intercept, detect or perceive the test information of the protection test tool.

[0180] The method of the embodiment of the present application has at least the following beneficial effects:

[0181] 1. It can extract the latest protection test tools based on the real-world security incidents that are timely, which can not only evaluate the protection capabilities of security applications, but also identify the technical points for improving security applications that are not supported by the protection test tools. It is better than the evaluation methods of third-party organizations in promoting software optimization and improvement, and more practical.

[0182] 2. A security incident may use protection test tools at various attack stages. Based on the evaluation method of the embodiment of the present application, all security applications that respond to the security incident can be evaluated at the same time, such as firewalls, terminal security software, intrusion detection software, etc.

[0183] The following is an introduction to the security application evaluation method on the test device side of an embodiment of the present application.

[0184] See also Figure 6As shown, a security application evaluation method provided in an embodiment of the present application, applied to a test device side, may include the following steps:

[0185] S601, based on the application type of the target security application to be evaluated, selecting at least one target protection test tool corresponding to the application type from a tool library, the tool library including a plurality of different protection test tools;

[0186] S602, for at least one target protection test tool, respectively perform the following operations: run a target protection test tool to perform a protection test on a device installed with a target security application, and receive a processing status returned by the device after the target security application processes the test information of the target protection test tool;

[0187] S603: Determine a protection capability evaluation result of the target security application based on the obtained at least one processing state.

[0188] In an embodiment of the present application, based on the processing status of the target security application for multiple target protection test tools, the protection capability of the target security application for multiple target protection test tools can be determined, for example: which target protection test tools have stronger protection capabilities and which target protection test tools have weaker protection capabilities, thereby determining the specific defects of the security application, so as to optimize and update the security application in a targeted manner.

[0189] In some possible embodiments, the tool library can be obtained in the following ways:

[0190] Collect multiple security events and extract corresponding protection testing tools from multiple security events;

[0191] Deduplication operations are performed on the extracted multiple protection test tools to obtain multiple different protection test tools;

[0192] Save multiple different protection testing tools to the tool library.

[0193] In some possible embodiments, the application type includes an operating system type and a protection type;

[0194] like Figure 7 As shown, in the above S601, based on the protection type of the target security application to be evaluated, selecting at least one target protection test tool corresponding to the protection type from the tool library may include the following steps:

[0195] S6011, based on the operating system type of the target security application, select multiple candidate protection test tools corresponding to the operating system type from the tool library;

[0196] S6012: Based on the protection type of the target security application, select at least one target protection testing tool corresponding to the protection type from a plurality of candidate protection testing tools.

[0197] In some possible embodiments, if the target security application is used to intercept the target protection test tool, then the processing state corresponding to each target protection test tool includes at least one of the following:

[0198] Interception state, detection state, perception state, and no perception state;

[0199] like Fig. 8A As shown, in the above S603, determining the protection capability evaluation result of the target security application based on the at least one processing state obtained may include the following steps:

[0200] S6031, determining, based on the at least one processing state obtained, the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived;

[0201] S6032, determining the protection capability evaluation result of the target security application based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived.

[0202] In a possible embodiment, if the target security application is used to detect the target protection test tool, then the processing state corresponding to each target protection test tool includes at least one of the following:

[0203] Detection state, perception state, no perception state;

[0204] like Figure 8B As shown, in the above S603, determining the protection capability evaluation result of the target security application based on the at least one processing state obtained may include the following steps:

[0205] S603_1, determining the number of target protection test tools detected by the target security application, the number of target protection test tools perceived, and the number of target protection test tools not perceived based on at least one processing state obtained;

[0206] S603_2: Determine a protection capability evaluation result of the target security application based on the number of target protection test tools detected by the target security application, the number of target protection test tools perceived, and the number of target protection test tools not perceived.

[0207] The following is an introduction to the security application evaluation method on the device under test side of an embodiment of the present application.

[0208] See also Fig. 9 As shown, the embodiment of the present application provides a security application evaluation method, which can be applied to a tested device, including the following steps:

[0209] S901, upon receiving test information of a target protection test tool sent by a test device, processing the test information through a target security application to obtain a corresponding processing status; wherein the target protection test tool is selected by the test device from a tool library including a plurality of different protection test tools based on an application type of the target security application;

[0210] S902, sending the corresponding processing status to the test device.

[0211] In an embodiment of the present application, by sending the processing status of the target security application for each target protection test tool to the test device, the test device can determine the protection capability of the target security application for multiple target protection test tools after obtaining the processing status of the target security application for multiple target protection test tools, for example: which target protection test tools have stronger protection capabilities and which target protection test tools have weaker protection capabilities, thereby determining the specific defects of the security application, so as to optimize and update the security application in a targeted manner.

[0212] In some possible embodiments, if the target security application is used to intercept the target protection test tool, the above S901 processes the test information by the target security application to obtain a corresponding processing status, which may include one of the following steps:

[0213] D1. Intercept and process the test information through the target security application to obtain the interception status.

[0214] D2. Detect and process the test information through the target security application to obtain the detection status.

[0215] D3. The test information is sensed and processed through the target security application to obtain a sensed state or a non-sense state.

[0216] In a possible embodiment, if the target security application is used to detect the target protection test tool, the above S901 processes the test information by the target security application to obtain a corresponding processing status, which may include one of the following steps:

[0217] E1. Detect and process the test information through the target security application to obtain a detection status;

[0218] E2. The target security application senses and processes the test information to obtain a sensed state or a non-sense state.

[0219] Based on the same inventive concept as the above method embodiments of the present application, a security application evaluation device is also provided in the embodiments of the present application. The principle of solving the problem by the device is similar to that of the method in the above embodiment. Therefore, the implementation of the device can refer to the implementation of the above method, and the repeated parts will not be repeated.

[0220] See also Fig.10 As shown, an embodiment of the present application provides a security application assessment device 100, including a selection module 101, an attack module 102 and a capability determination module 103.

[0221] A selection module 101 is used to select at least one target protection test tool corresponding to the application type from a tool library based on the application type of the target security application to be evaluated, and the tool library includes a plurality of different protection test tools;

[0222] The attack module 102 is used to perform the following operations respectively for at least one target protection test tool: run a target protection test tool to perform a protection test on a device installed with a target security application, and receive a processing status returned by the device after the target security application processes the test information of a target protection test tool;

[0223] The capability determination module 103 is used to determine the protection capability evaluation result of the target security application based on the at least one processing state obtained.

[0224] In an embodiment of the present application, by sending the processing status of the target security application for each target protection test tool to the test device, the test device can determine the protection capability of the target security application for multiple target protection test tools after obtaining the processing status of the target security application for multiple target protection test tools, for example: which target protection test tools have stronger protection capabilities and which target protection test tools have weaker protection capabilities, thereby determining the specific defects of the security application, so as to optimize and update the security application in a targeted manner.

[0225] In a possible embodiment, the device further includes an acquisition module, which is used to obtain the tool library in the following manner:

[0226] Collect multiple security events and extract corresponding protection testing tools from multiple security events;

[0227] Deduplication operations are performed on the extracted multiple protection test tools to obtain multiple different protection test tools;

[0228] Save multiple different protection testing tools to the tool library.

[0229] In a possible embodiment, the application type includes an operating system type and a protection type; the selection module 101 is further used to:

[0230] Based on the operating system type of the target security application, select multiple candidate protection testing tools corresponding to the operating system type from the tool library;

[0231] Based on the protection type of the target security application, at least one target protection testing tool corresponding to the protection type is selected from a plurality of candidate protection testing tools.

[0232] In a possible embodiment, if the target security application is used to intercept the target protection test tool, then the processing state corresponding to each target protection test tool includes at least one of the following:

[0233] Interception state, detection state, perception state, and no perception state;

[0234] The capability determination module 103 is also used for:

[0235] Determining, based on at least one of the obtained processing states, a number of target protection test tools intercepted by the target security application, a number of target protection test tools detected, a number of target protection test tools perceived, and a number of target protection test tools not perceived;

[0236] Based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived, the protection capability evaluation result of the target security application is determined.

[0237] In a possible embodiment, if the target security application is used to detect the target protection test tool, then the processing state corresponding to each target protection test tool includes at least one of the following:

[0238] Detection state, perception state, no perception state;

[0239] The capability determination module 103 is also used for:

[0240] Determining, based on the obtained at least one processing state, a number of target protection test tools detected by the target security application, a number of target protection test tools perceived, and a number of target protection test tools not perceived;

[0241] Based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools, a protection capability assessment result of the target security application is determined.

[0242] Based on the same inventive concept as the above method embodiments of the present application, a security application evaluation device is also provided in the embodiments of the present application. The principle of solving the problem by the device is similar to that of the method in the above embodiment. Therefore, the implementation of the device can refer to the implementation of the above method, and the repeated parts will not be repeated.

[0243] See also Fig.11 As shown, an embodiment of the present application provides a security application assessment device 110, including a processing module 111 and a sending module 112.

[0244] The processing module 111 is used to process the test information of the target protection test tool sent by the test device through the target security application to obtain a corresponding processing state; wherein the target protection test tool is selected by the test device from a tool library including a plurality of different protection test tools based on the application type of the target security application;

[0245] The sending module 112 is used to send the corresponding processing status to the test device.

[0246] In an embodiment of the present application, by sending the processing status of the target security application for each target protection test tool to the test device, the test device can determine the protection capability of the target security application for multiple target protection test tools after obtaining the processing status of the target security application for multiple target protection test tools, for example: which target protection test tools have stronger protection capabilities and which target protection test tools have weaker protection capabilities, thereby determining the specific defects of the security application, so as to optimize and update the security application in a targeted manner.

[0247] In a possible embodiment, if the target security application is used to intercept the target protection test tool, the processing module 111 is further used to:

[0248] The test information is intercepted and processed by the target security application to obtain the interception status; or,

[0249] Detect and process the test information through the target security application to obtain the detection status; or,

[0250] The test information is sensed and processed through the target security application to obtain a sensed state or a non-sense state.

[0251] In a possible embodiment, if the target security application is used to detect the target protection test tool, the processing module 111 is further used to:

[0252] Detect and process the test information through the target security application to obtain the detection status; or,

[0253] The test information is sensed and processed through the target security application to obtain a sensed state or a non-sense state.

[0254] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0255] After introducing the security application evaluation method and device according to the exemplary embodiment of the present application, next, a security application evaluation device according to another exemplary embodiment of the present application is introduced.

[0256] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0257] In some possible implementations, the security application evaluation device according to the present application may include at least a processor and a memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the security application evaluation method according to various exemplary implementations of the present application described in this specification. For example, the processor may execute the following steps: Figure 6 or Fig. 9 Follow the steps shown in .

[0258] After introducing the security application evaluation method and device according to the exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0259] Based on the same inventive concept as the above method embodiments of the present application, an electronic device is also provided in the embodiments of the present application. The principle of solving the problem by the electronic device is similar to the method of the above embodiment. Therefore, the implementation of the electronic device can refer to the implementation of the above method, and the repeated parts will not be repeated.

[0260] See also Fig.12 As shown, the electronic device 120 may include at least a processor 121 and a memory 122. The memory 122 stores program codes, and when the program codes are executed by the processor 121, the processor 121 executes the steps in any one of the above-mentioned security application assessment methods.

[0261] In some possible implementations, the electronic device according to the present application may include at least one processor and at least one memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the security application assessment method according to various exemplary implementations of the present application described above in this specification. For example, the processor may execute Figure 6 or Fig. 9 Follow the steps shown in .

[0262] In an exemplary embodiment, the present application further provides a storage medium including a program code, such as a memory 122 including a program code, and the program code can be executed by a processor 121 of an electronic device 120 to complete the above-mentioned security application assessment method. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0263] Refer to the following Fig.13 An electronic device 130 according to an embodiment of the present application is described. Fig.13 The electronic device 130 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0264] like Fig.13 The electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: at least one processing unit 131, at least one storage unit 132, and a bus 133 connecting different system components (including the storage unit 132 and the processing unit 131).

[0265] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0266] The storage unit 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory unit 1322 , and may further include a read-only memory (ROM) 1323 .

[0267] The storage unit 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0268] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 136. As shown, the network adapter 136 communicates with other modules for the electronic device 130 via a bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0269] In some possible implementations, various aspects of the security application evaluation method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the security application evaluation method according to various exemplary implementations of the present application described above in this specification. For example, the electronic device may execute the following steps: Figure 6 or Fig. 9 Follow the steps shown in .

[0270] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0271] The program product of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a computing device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, device or device.

[0272] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0273] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0274] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0275] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0276] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0277] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0278] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for evaluating a secure application, characterized in that: include: Based on the application type of the target security application to be evaluated, selecting a plurality of target protection test tools corresponding to the application type from a tool library, wherein the tool library includes a plurality of different protection test tools; For the multiple target protection test tools, the following operations are performed respectively: run a target protection test tool to perform a protective test on the device installed with the target security application, and receive the processing status returned by the device after the target security application processes the test information of the target protection test tool; wherein, when the target security application is used to intercept the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: interception status, detection status, perception status, and unaware status; when the target security application is used to detect the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: detection status, perception status, and unaware status; Based on the multiple processing states obtained, determine the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived, and determine the protection capability evaluation result of the target security application based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived; or Based on the multiple processing states obtained, the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools are determined, and based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools, the protection capability evaluation result of the target security application is determined.

2. The method according to claim 1, characterized in that The tool library is obtained in the following way: Collect multiple security events, and extract corresponding protection test tools from the multiple security events respectively; Deduplication operations are performed on the extracted multiple protection test tools to obtain multiple different protection test tools; The plurality of different protection testing tools are saved in the tool library.

3. The method according to claim 1, characterized in that The application type includes an operating system type and a protection type; The step of selecting a plurality of target protection test tools corresponding to the protection type of the target security application to be evaluated from a tool library includes: Based on the operating system type of the target security application, selecting a plurality of candidate protection test tools corresponding to the operating system type from a tool library; Based on the protection type of the target security application, a plurality of target protection testing tools corresponding to the protection type are selected from the plurality of candidate protection testing tools.

4. A method for evaluating a secure application, characterized in that: include: Each time test information of a target protection test tool sent by the test device is received, the test information is processed by the target security application to obtain a corresponding processing status; wherein the target protection test tool is selected by the test device from a tool library including a plurality of different protection test tools based on the application type of the target security application; when the target security application is used to intercept the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: interception status, detection status, perception status, and unaware status; when the target security application is used to detect the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: detection status, perception status, and unaware status; Sending the corresponding processing status to the test device so that the test device determines the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived based on the obtained multiple processing statuses, and determines the protection capability evaluation result of the target security application based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived; or Based on the multiple processing states obtained, the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools are determined, and based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools, the protection capability evaluation result of the target security application is determined.

5. The method according to claim 4, characterized in that If the target security application is used to intercept the target protection test tool, the processing of the test information by the target security application to obtain a corresponding processing status includes: The target security application intercepts the test information to obtain an interception status; or, The target security application detects and processes the test information to obtain a detection status; or, The test information is sensed and processed by the target security application to obtain a sensed state or a non-sense state.

6. The method according to claim 4, characterized in that If the target security application is used to detect the target protection test tool, the test information is processed by the target security application to obtain a corresponding processing status, including: The target security application detects and processes the test information to obtain a detection status; or, The test information is sensed and processed by the target security application to obtain a sensed state or a non-sense state.

7. A security application evaluation device, characterized in that: include: A selection module, for selecting, based on the application type of the target security application to be evaluated, a plurality of target protection test tools corresponding to the application type from a tool library, wherein the tool library includes a plurality of different protection test tools; An attack module is used to perform the following operations respectively for the multiple target protection test tools: run a target protection test tool to perform a protective test on the device installed with the target security application, and receive a processing status returned by the device after the target security application processes the test information of the target protection test tool; wherein, when the target security application is used to intercept the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: interception status, detection status, perception status, and unaware status; when the target security application is used to detect the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: detection status, perception status, and unaware status; A capability determination module is used to determine, based on the multiple processing states obtained, the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived, and to determine a protection capability evaluation result of the target security application based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived; or Based on the multiple processing states obtained, the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools are determined, and based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools, the protection capability evaluation result of the target security application is determined.

8. A security application evaluation device, characterized in that: include: A processing module, for receiving test information of a target protection test tool sent by a test device, processing the test information through a target security application to obtain a corresponding processing status; wherein the target protection test tool is selected by the test device from a tool library including a plurality of different protection test tools based on the application type of the target security application; when the target security application is used to intercept the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: interception status, detection status, perception status, and unaware status; when the target security application is used to detect the target protection test tool, the processing status corresponding to each target protection test tool includes at least one of the following: detection status, perception status, and unaware status; A sending module, used to send the corresponding processing status to a test device, so that the test device determines the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived based on the obtained multiple processing statuses, and determines the protection capability evaluation result of the target security application based on the number of target protection test tools intercepted by the target security application, the number of target protection test tools detected, the number of target protection test tools perceived, and the number of target protection test tools not perceived; or Based on the multiple processing states obtained, the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools are determined, and based on the number of target protection test tools detected by the target security application, the number of perceived target protection test tools, and the number of unaware target protection test tools, the protection capability evaluation result of the target security application is determined.

9. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 3 and 4 to 6.

10. A computer-readable storage medium, characterized in that: It includes program code, and when the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of any one of the methods described in claims 1 to 3 and 4 to 6.

11. A computer program product, characterized in that It includes computer instructions, which are stored in a computer-readable storage medium; when a processor of an electronic device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, so that the electronic device performs the steps of any method described in claims 1 to 3 and 4 to 6.

Images