A method and device for automatically constructing an evaluation operation template

Abstract

The present invention proposes a method and device for automatically constructing an evaluation operation template to improve the efficiency and accuracy of evaluating computer security. It is used to solve the problem that manual verification and operation configuration of computer system security in the prior art require a lot of time and energy, and the efficiency is often very low. At the same time, it solves the technical problem that experience and methods cannot be reused and the verification efficiency is low. The present invention proposes a method for automatically constructing an evaluation operation template. By automatically constructing an evaluation operation template, the test results are objectively classified, counted and analyzed. When the evaluation template itself can realize automatic judgment, or the evaluation template that can realize automatic judgment after manual correction is added to the template library, the existing experience can be used to improve the efficiency and accuracy of computer system evaluation, and automatic construction and detection can be realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer security technology, and in particular to a method and device for automatically constructing an evaluation operation template. Background Art

[0002] With the rapid development of network information technology, computer systems have been widely used in daily life and in the systems of important national functional departments such as government, banks, and the military. It is precisely because computer security has received attention from the government and enterprises that the demand for computer security assessment has emerged in the existing technology. For example, it is often necessary to test whether all computers in the company or unit are correctly configured by users during use, and whether some computers have modified their configurations during user use, resulting in security vulnerabilities. Such tests generally cannot be unified into one template to configure all computers, because different units have different security requirements. For example, the public security system requires stronger system security than ordinary foreign trade companies. However, there are tens of millions of computer configurations. If you want to manually verify and configure the security of multiple computer configurations one by one, it will take a lot of time and energy, and the efficiency is often very low.

[0003] In addition, since manual verification is performed on a single computer system, its experience and methods cannot be reused, which is one of the reasons for the low efficiency of manual verification. In view of the above technical problems, the existing technology does not provide a feasible solution. Summary of the invention

[0004] In view of this, the present invention proposes a method and device for automatically constructing an evaluation operation template to improve the efficiency and accuracy of evaluating computer security.

[0005] In the first aspect, the embodiment of the present invention provides a method for automatically constructing an evaluation operation template, which can automatically construct the evaluation process, form an objective evaluation standard, improve the correctness of the evaluation results, and achieve better evaluation results. The method for automatically constructing an evaluation operation template includes:

[0006] Step 1: Build the evaluation code according to the content to be tested to form an evaluation template;

[0007] Step 2: Apply the evaluation template to different computer systems and conduct tests;

[0008] Step 3: Obtain the test results of the evaluation template, and perform statistics and classification;

[0009] Step 4: Obtain incorrect difference items in the test results and construct difference features;

[0010] Step 5: When the difference feature can be automatically identified through the evaluation template, the correct configuration item in the computer system is added to the template library as a new template library content, and the configuration of the computer system with the difference is automatically corrected according to the correct configuration formed;

[0011] Step 6: When the difference characteristics cannot be automatically identified by the evaluation template, manual correction is performed; and the manually corrected configuration is added to the template library. The test script is pre-built by computer and used as a security evaluation template for automatic testing.

[0012] In a second aspect, an embodiment of the present invention provides an automatic evaluation operation template construction device, which adopts the above-mentioned automatic evaluation operation template construction method. The automatic evaluation operation template construction device includes a generation module, an evaluation template, a classification module, a training module, and a storage module, wherein:

[0013] The generation module is used to build the evaluation code according to the content to be tested;

[0014] The evaluation module performs tests based on the evaluation code of the generation module and provides feedback on the evaluation results;

[0015] The classification module is used to receive the evaluation results and to perform statistics and classification based on the evaluation results;

[0016] The training module receives incorrect difference items in the test results and constructs difference features;

[0017] The storage module is used to store the evaluation templates that can be judged by themselves and the evaluation templates that have been manually trimmed.

[0018] In a third aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, and a computer program is stored on the non-transitory computer-readable storage medium to implement the method for automatically constructing an evaluation operation template of the present application.

[0019] The beneficial effects of the present invention are as follows: the present invention provides a method, device and storage medium for automatically constructing an evaluation operation template. By automatically constructing an evaluation operation template, the test results are objectively classified, counted and analyzed. When the evaluation template itself can realize automatic judgment, or the evaluation template that can realize automatic judgment after manual correction is added to the template library, the existing experience can be utilized to improve the efficiency and accuracy of computer system evaluation and realize automatic construction and detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the method for automatically constructing an evaluation operation template provided by the present invention;

[0021] Figure 2 It is a structural diagram of the automatic construction device of the evaluation operation template provided by the present invention. DETAILED DESCRIPTION

[0022] The embodiments of the present invention provide a method, device and storage medium for automatically constructing an evaluation operation template, which are used to solve technical problems such as the existing computer security system detection requires a lot of manpower and time, cannot perform detection efficiently and accurately, and the detection experience cannot be reused.

[0023] In order to make the purpose, features and advantages of the invention more obvious and easy to understand, the technical solutions in the embodiments of the invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the invention. Obviously, the embodiments described below are only part of the embodiments of the invention, not all of the embodiments. Based on the embodiments of the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the invention. The technical solutions of the invention are further described in detail below in conjunction with the drawings.

[0024] like Figure 1 As shown in the figure, the automatic construction method of the evaluation operation template provided mainly includes:

[0025] Build the evaluation code according to the content to be tested, and use shell scripts or selenium tools to write the test code.

[0026] Preferably, the test content of the evaluation code constructed in step 1 is determined manually in advance, and a shell script command or selenium simulation software is written according to the content to be tested to automatically open the system and test according to the program content. This embodiment uses a shell script, and different shell script command sets are used for systems of different company units. For example, if the company unit is an ubuntu16 system, a specific command set of the system is used. If it is an ubuntu18 system, a unique shell command set of ubuntu18 is used, so that it can be targeted to speed up and accurately test, improve efficiency, greatly optimize the evaluation of computer system security, and achieve the stability of the detection process.

[0027] When the test is conducted by browser testing, the selenium tool is used for testing. Selenium is a tool for automated testing of Web applications. Selenium testing runs directly in the browser and can be used to simulate user operations. Preferably, selenium can be selected as the simulation software to evaluate the configuration of the computer system.

[0028] Step 2: Build different evaluation code templates according to different unit requirements and national standards;

[0029] The construction of the test template is mainly to test the security level that needs to be guaranteed for the system of this type and the corresponding test items mentioned in the national standard document in steps. The national standards mentioned are the standards provided by the state to standardize the technology of this industry. These national standards are described in text form and can be parsed into text for processing through PDF parsing tools. For example, the national standard stipulates that when setting a password in the public security system, the password must contain letters, numbers, special symbols, and at least 8 digits. At this time, the NLP tool can automatically parse the content of this text, read the text of the corresponding paragraph in the national standard through text understanding, and match the text expression of the password specification for the public security system in the national standard through the syntactic analysis technology of NLP technology. Among them, a text expression such as "password specification" may be matched, but it is not the only way to express the literal meaning of "password specification". Therefore, it can be mapped to the word2vec word vector space to generalize various expressions of the text of the "password specification". For example, after generalization, it may be possible to obtain another paragraph marked as "password compliance". Then it can be located in the national standard. What kind of security specification does such a unit system need to correspond to? The corresponding specification is parsed through regular expressions, such as letters, special symbols, at least a few, etc., and their test content is extracted. Through manual code writing, it is constructed as a shell or seleium code template. For example, to test whether the system password contains special symbols, you can use the shell command: sudo passwd command, enter the password, and see if the content of the special symbol is set. If it is possible, the system security is insufficient. Such a command corresponds to a text description rule, such as "test password security". The rules or labels corresponding to the content to be tested can be matched to different test code templates through model training. Because the security standards of relevant national systems are limited, the test content and code are also fixed and limited. It is a machine learning classification problem. Therefore, you can write shell scripts, i.e. code templates, in advance, and set the rules corresponding to the code templates, such as "test password security", as the test rules corresponding to this code, and also as the annotation value for machine learning. The text description "test password security" corresponding to the code is used as the feature of the training corpus. Through the Bayesian classifier, the next time you encounter a rule that needs to be tested, you can enter the text description, call the model, and automatically obtain the template of the test code under the corresponding rule.

[0030] Step 3: Define the test content and rules of the content to be tested by generating template tags.

[0031] However, another difficulty in step 2 is that the matching rules are difficult to define. That is, the regular expression mentioned above is used to parse letters, special symbols, at least a few digits, etc., and then converted into corresponding test content. People cannot describe this test content as "testing the security of passwords". Therefore, it is necessary to define the content to be tested and the matching rules through the expression of tags to solve the above problem.

[0032] On the other hand, due to the large number of systems, code templates cannot be fully reused in different systems or systems with different security levels. Therefore, it is necessary to set code templates with different labels. Code templates with different labels will correspond to different systems, so each code template needs to be set with relevant label information. If the template label information is set completely manually, the workload is too large. For example, the label "test password security" we defined above is manually defined. Therefore, it is also possible to use the national standards to require specific items in the system configuration that need to be tested as annotations.

[0033] For example, in password testing, the encryption algorithm for password testing of computer systems in all public security departments and administrative agencies is required to use advanced encryption algorithms such as RSA. Then the label of the code template can be automatically generated as "system category + test content + security level", for example: "The public security system tests the password security with a high security level", this label is the label for defining the matching rule to be solved above. The extraction method adopts LTP to parse the standard name of the first line in the national standard, and identifies the name therein as the first item. The content in the sub-title to be extracted is parsed by LTP syntax analysis as the second item, and then the text of "security level" mentioned in the minimum title item is parsed to extract the content therein as the third item. In this way, we can automatically generate different template labels, define the relevant rules that this system should test, and this rule also corresponds to the code paragraph that needs to be tested. Therefore, the security of this function point of the system can be tested through this code template. The evaluation label and evaluation template can be fully constructed. Through the corresponding different labels above, we can also distinguish the code template for the password test of the public security system from the code template for the password test of the hospital system. The purpose of different systems corresponding to different labels and templates is achieved.

[0034] Step 4: Apply the evaluation template to different computer systems and conduct tests;

[0035] After obtaining the computer to be tested, first define and analyze what type of system the content to be tested belongs to. According to the system type and the content to be tested, match and obtain the template label. And use the code template with this label to perform automatic testing. The testing method is to first automatically run the program through the computer, and automatically run or click the commands and buttons to be tested. Get the results after running or clicking, and get the running results by obtaining the windows API interface. Store the running results.

[0036] Step 5: Obtain the test results of the evaluation template, and perform statistics and classification;

[0037] In different departments such as banks, public security, and institutions, there are computers of different types and configurations. Therefore, when using the evaluation template for evaluation, the test results are different, and the security judgment standards for different types of computers will also be different. However, computers of the same type and function in the same unit should have the same configuration.

[0038] Therefore, when it is found during the test that the security configuration of some computers is different from that of other computers of the same type, it is determined that this group of computers may have incorrect configuration problems.

[0039] Therefore, when testing a computer, you first need to extract and match the corresponding label in step 3, and then test it according to the test content and code template corresponding to the label. And count which computers have differences and which have no problems among these configurations.

[0040] For example, the security of 100 computers in the hospital registration system needs to be tested. Therefore, by searching and matching the corresponding tag in the tag library, "Hospital system test password security high security level", this tag is used. After entering the trained model in step 1, a piece of code is obtained. This code is a code template, which is specially used to test the security of a certain type of system. By copying this code template to 100 computers and running it automatically, the running results can be obtained. The content of the test results can be obtained through the API provided by Windows and stored. It was found that the test results of 92 of the 100 computers were consistent. For example, the values ​​of GatewayPorts in the tested project were all yes, while 5 were empty values ​​and the values ​​of the other 3 were no. Then it can be known that 8 computers may be different from others. It is only necessary to conduct further key tests on these eight computers to know whether there are problems with these computers, without having to conduct detailed tests on all 100 computers one by one, which consumes a lot of time and energy.

[0041] Step 6: Get incorrect difference items in the test results and construct difference features; group the difference features, automatically label them through security levels, and automatically obtain the most important groups. When there are local differences in high-level groups, it is recognized that the system has differences.

[0042] In the automatic test, it was also found that when computer configuration A and computer configuration B were different, the same test results were obtained, while computer configuration C and computer configuration D were different, and different test results were obtained. That is to say, not all differences in configuration content will lead to different results. Because some different test results will not lead to differences in the final security. However, some different configurations will lead to differences in the final security. For example, in the hospital registration system, incorrect filling of expenses is a major error, which will lead to errors in expense identification and economic losses. However, in the system, when only the year and month are filled in but not the day, the date of birth will not lead to serious consequences. Therefore, during the test process, some items are important test points, some are not important, and not all features are important. Therefore, it is necessary to construct an importance feature. We use the Trusted Computer System Evaluation Standard TCSEC, which divides the system into four categories and seven security levels of ABCD. Level D is the lowest security level, category C is the autonomous protection level; category B is the mandatory protection level; category A is the verification protection class. The security level requirements of level D are very low, and each level corresponds to a set of system security specifications, and the standards of the original system that have not been manually modified are used as the original standards for evaluation. For example, when the test content of a project belongs to the lowest level, we will automatically mark the content as "unimportant", that is, if it is a level D project, even if this project fails the test, we do not think that there is a problem with the system. However, as long as the remaining test items meet other standards above level C, they are all important standards, and alarm processing will be carried out if there is a problem.

[0043] For example, in the C1 level, the system audits the events that occur and writes them into the log, such as when the computer is turned on, which user logged in at what time and from where, etc., so that by checking the log, traces of intrusion can be found. If it is found that the system lacks the function of recording past historical passwords during the password setting process, after matching the standards, it is found that the C-level standard is violated, so an alarm is required. Because this feature of the computer cannot be ignored, it needs to be consistent with other computers. But sometimes, such as accessing a picture on a web page, sometimes you don’t need a password to access it. If there is no password defined, it is not an important standard. Therefore, some systems require passwords, some do not, and these are not the content to be alarmed. Such unimportant features can be ignored, and only important system security features are extracted. Achieve the purpose of accurately obtaining features to make the function of alarming different computer systems more accurate.

[0044] Therefore, it is necessary to clarify which configuration parameters cause the evaluation results to be different from expectations. The configuration parameters need to be extracted to determine which different configurations will lead to different results and which different configurations will result in the same results.

[0045] Step 7: Another method for evaluating system differences also includes extracting configuration parameters that lead to different results in the difference items, using the configuration parameters in the difference items as features, using whether the system security is different as a label value, performing deep learning training, and obtaining a training model for judging difference features. There are many configuration features with difference values, but there are only two results, that is, whether the security levels of the two computers are the same or different in the end. It is a binary classification problem. According to the difference features in the difference items, the model is trained and a binary classification test is performed. When encountering different difference configurations, you can input the difference features to determine whether the system security meets the standards.

[0046] Step 8: When the difference feature can be automatically identified through the evaluation template, the correct configuration item in the computer system is added to the template library as a new template library content, and the configuration of the computer system with the difference is automatically corrected according to the correct configuration formed;

[0047] Because when some computers are identified to be different from other computers, the differences can be repaired. The purpose of modifying the configuration can be achieved by calling the difference content of the project in other computers and automatically running the shell script that changes the difference according to the difference content. Therefore, batch computer configuration self-repair can be achieved.

[0048] Step 9: When the difference features cannot be automatically identified by the evaluation template, manual correction is performed; and the manually corrected configuration is added to the template library.

[0049] When the evaluation template itself can achieve automatic judgment, or the evaluation template that can achieve automatic judgment after manual correction is added to the template library, it is possible to utilize existing experience, overcome the low efficiency of manual verification, improve the efficiency and accuracy of computer system evaluation, and achieve automatic construction and detection.

[0050] like Figure 2 As shown, a structural diagram of an automatic evaluation operation template construction device is provided, including a generation module, an evaluation template, a classification module, a training module, and a storage module, wherein:

[0051] The generation module is used to build the evaluation code according to the content to be tested;

[0052] The evaluation module performs tests based on the evaluation code of the generation module and provides feedback on the evaluation results;

[0053] The classification module is used to receive the evaluation results and to perform statistics and classification based on the evaluation results;

[0054] The training module receives incorrect difference items in the test results and constructs difference features;

[0055] The storage module is used to store the evaluation templates that can be judged by themselves and the evaluation templates that have been manually trimmed.

[0056] An embodiment of the present invention further provides a non-transitory computer-readable storage medium, which stores a computer program to implement the method for automatically constructing an evaluation operation template in the present application.

[0057] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a method or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0058] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0059] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for automatically constructing an evaluation operation template, applied to testing computer security configuration, characterized in that: The method for automatically constructing the evaluation operation template includes: Step 1: Based on the content to be tested and the requirements of the company to be tested, use the national standards related to the company to build an evaluation operation template, write shell script commands or use selenium simulation software to automatically open the system, and perform automatic testing according to the program content; Step 2: Use syntax analysis to automatically generate labels for the evaluation operation template, which are used to define the content and rules to be tested, and apply the evaluation operation template with labels to different computer systems and conduct tests; Step 3: Obtain the test results of the evaluation operation template, and perform statistics and classification; Step 4: Obtain incorrect difference items in the test results and construct difference features; group the difference features to obtain important difference feature groups and exclude unimportant difference groups; and train a binary classification model through machine learning to automatically determine whether the configuration difference will eventually lead to system security differences; Step 5: When the difference feature can be automatically identified by the evaluation operation template, the correct configuration item in the computer system is added to the template library as a new template library content, and the configuration of the computer system with the difference is automatically corrected according to the correct configuration formed; Step 6: When the difference feature cannot be automatically identified by the evaluation operation template, it is manually corrected; and the manually corrected configuration is added to the template library; in the step 4, when a computer system test result different from other computer system test results is found during the test, the difference features are grouped according to the trusted computer system evaluation standard TCSEC to obtain important difference feature groups and exclude unimportant difference groups; The computer systems with different test results are judged as incorrect difference items; The binary classification model is trained through machine learning, including extracting the configuration parameters that lead to different results in the difference items, taking the configuration parameters in the difference items as features, taking whether the system security is different as a label value, performing deep learning training, obtaining a training model for judging the difference features, and performing binary classification.

2. The method for automatically constructing an evaluation operation template according to claim 1, characterized in that: The evaluation operation template constructed in step 1 is to automatically open the system through a shell script command or simulation software, and perform testing according to the program content; the content that needs to be tested is automatically tested by obtaining national standards and automatically testing the security level that needs to be guaranteed for the corresponding type of system mentioned in the national standard document.

3. The method for automatically constructing an evaluation operation template according to claim 1, characterized in that: The automatic test is to write the shell script or the evaluation operation template in advance, and use the content of the evaluation operation template as the annotation value of machine learning; interpret the text description corresponding to the evaluation operation template as the training corpus and features; train through the Bayesian classifier, when encountering the test content that needs to be tested, input the text description, match the model that needs to be called, and automatically obtain the evaluation operation template under the corresponding text description.

4. The method for automatically constructing an evaluation operation template according to claim 3, characterized in that: The text description corresponding to the evaluation operation template is explained, and the text description is automatically extracted into "system category + test content + security level" through syntactic analysis by the LTP natural language processing tool.

5. An automatic evaluation operation template construction device, used to implement the automatic evaluation operation template construction method according to any one of claims 1 to 4, characterized in that: The automatic construction device for evaluation operation templates includes a generation module, an evaluation module, a classification module, a training module, and a storage module, wherein: The generation module is used to construct the evaluation code according to the content to be tested; The evaluation module performs testing according to the evaluation code of the generation module and feeds back the evaluation result; The classification module is used to receive the evaluation results, and perform statistics and classification according to the evaluation results; The training module receives incorrect difference items in the test results and constructs difference features; The storage module is used to store the evaluation operation templates that can be identified automatically and the evaluation operation templates that have been manually trimmed.

6. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for automatically constructing an evaluation operation template according to any one of claims 1 to 4.

Images