A method for constructing an information security scoring system based on artificial intelligence

Abstract

The present invention discloses a method for constructing an information security scoring system based on artificial intelligence, comprising: crawling standard texts related to information security on the industry standard network; determining suspected standard texts to be updated; determining industry standard texts that need to be updated; performing structured extraction on the industry standard texts that need to be updated through preset text matching rules to obtain structured standard data; based on the industry standard texts that need to be updated, evaluating the differences between new and old standard texts and the differences between new and old scoring standard texts; obtaining the results of manual evaluation by an automatic testing system and assessors; processing the updated standard texts based on the comparison results, thereby more accurately solving the problem of updating the standard texts.

Description

[Technical field]

[0001] The present invention relates to the field of information security scoring, and in particular to a method for constructing an information security scoring system based on artificial intelligence. [Background technology]

[0002] With the continuous development of technology, information security is becoming more and more important. When an information system is designed, a corresponding security assessment system will also be designed, the purpose of which is to test the security of the current information system. The most common is the password complexity test. Password complexity means that the password set by the user must meet a certain complexity. If it does not meet the requirements, a corresponding prompt will be given. The security scoring system is a system that includes various functional tests. If it is found that a function of the current information system does not meet the corresponding industry standards, points will be deducted accordingly.

[0003] The current safety scoring system relies on manual customization, requiring practitioners to first learn the corresponding industry standards and then manually design the corresponding scoring system. This method relies more on human subjective judgment and requires energy to read a large number of relevant industry standard details, which is inefficient. If a system can be designed to intelligently learn existing industry standards and scoring rules, when new industry standards emerge, the scoring rules can be intelligently updated and recommended to practitioners. [Summary of the invention]

[0004] The purpose of the present invention is to solve the defects of the above-mentioned prior art and provide a method for constructing an information security scoring system based on artificial intelligence;

[0005] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0006] A method for constructing an information security scoring system based on artificial intelligence, characterized by comprising:

[0007] Crawl industry standard online information security related standard texts;

[0008] According to the crawled standard text, one of the standard text data is obtained as the first standard text data, and all other standard text data are traversed. During the traversal, one of the standard text data is used as the second standard text data, and the directory containing the word "requirement" in the directory of the first standard text data is obtained as the first directory, and the directory containing the word "requirement" in the directory of the second standard text data is obtained as the second directory, and the edit distance of each row of directories in the first directory and the second directory is traversed by the python-Levenshtein toolkit, and the ratio of the number of sub-directories with an edit distance less than 2 to the number of sub-directories of the first directory is calculated as the directory overlap ratio. When the directory overlap ratio exceeds a preset value, the second standard text data is used as the suspected updated standard text of the first standard text data;

[0009] Based on the suspected updated standard text, a hierarchical similarity calculation method is used to determine the industry standard text that needs to be updated;

[0010] Performing structured extraction on the industry standard text that needs to be updated through preset text matching rules to obtain structured standard data;

[0011] According to the structured standard data, manual classification is performed and a binary classifier is trained by a Bayesian classification method;

[0012] According to the industry standard text that needs to be updated, by evaluating the difference between the new and old standard texts and the difference between the new and old scoring standard texts, a mapping relationship between the two differences is mined;

[0013] Through the automatic testing system, the network environment currently required to be tested is automatically evaluated according to the structured standard data of the industry standard text that needs to be updated, and the evaluation score p1 is obtained; the reviewer manually evaluates the network environment currently required to be tested, and obtains the evaluation score p2;

[0014] Compare the results between p1 and p2 to see if they are consistent, and process the updated standard text based on the comparison results.

[0015] Preferably, crawling industry standard online information security-related standard texts includes:

[0016] According to the seed link, extract the target link and put it into the queue to be crawled; parse and download standard text data from the page, and use the Jsoup component through webmagics to parse the HTML page; store the extracted standard text data in text file format or in a database.

[0017] Preferably, the step of determining the industry standard text that needs to be updated by using a hierarchical similarity calculation method based on the suspected update standard text includes:

[0018] The corresponding multi-level directories are subjected to hierarchical similarity calculation. In the process of hierarchical similarity calculation, the directories containing specific keywords are extracted, and the directories not containing specific keywords are filtered out. A standard text directory tree is constructed through a text analysis method. The sum of the similarities between the first standard text data and the content texts under each subdirectory in the first standard text data is calculated through a bm25 similarity algorithm to determine whether it exceeds a threshold. When the similarity is higher than the threshold, the standard file with a later release date is used as the industry standard text that needs to be updated for the file with an earlier release date. When the similarity is not higher than the threshold, it is difficult to determine whether it should be updated. The industry standard text that needs to be updated is determined by fusing the two file data.

[0019] Preferably, the industry standard text to be updated is structured extracted by preset text matching rules to obtain structured standard data, including: the structured standard data includes industry standard field names and their corresponding scoring standard contents, and the structured standard data is manually classified into two types: non-network security type and network security type that can be automatically tested. The unified preset rules of the structured standard are expressed as:

[0020] {[Industry standard field name],[Corresponding scoring standard content]};

[0021] The non-network security type is a type that cannot pass the automatic test security score test;

[0022] The automatically testable network security type refers to a type whose security score can be automatically tested by a testing system.

[0023] Preferably, according to the industry standard text that needs to be updated, by evaluating the differences between the new and old standard texts and the differences between the new and old scoring standard texts, the mapping relationship between the two differences is excavated; including: the differences are subdivided into three aspects: numerical values, degree adverbs and quantity difference changes.

[0024] Preferably, the automatic testing system automatically evaluates the network environment that currently needs to be tested according to the structured standard data of the industry standard text that needs to be updated to obtain the evaluation score p1; the reviewer manually evaluates the network environment that currently needs to be tested to obtain the evaluation score p2; including: the automatic testing system has a preset software automatic testing tool, which can automatically evaluate the network environment according to the industry standard field name in the structured standard of the standard text or the corresponding scoring standard content to obtain the evaluation score;

[0025] The reviewer manually evaluates the network environment that currently needs to be tested, and pushes the structured standard data of the updated standard text and the differences between the new and old scoring standard texts, and digs out the mapping relationship between the two differences to the reviewer, to assist the reviewer in manually evaluating the network environment.

[0026] Preferably, comparing the results of p1 and p2 to see if they are consistent, and processing the updated standard text according to the comparison result, including:

[0027] If they are consistent, the updated standard text is judged to be accurate, and the updated standard text and the corresponding structured standard data are stored in the database;

[0028] If the results are inconsistent, it is determined by default that the updated standard text field is updated incorrectly, resulting in a score difference, and the reviewer is asked to list the updated incorrect fields and the reasons for the errors; based on the updated incorrect fields and the reasons for the errors listed by the reviewer, the structured text matching rules of the standard text are updated;

[0029] If the reviewer does not list the specific field update error, it is judged that the updated standard document contains non-network security fields, and the score cannot be automatically measured by the automatic testing system; obtain the industry standard field name that causes the score difference and the corresponding scoring standard content, mark it as a non-network security type, and retrain the binary classifier in step 5.

Brief Description of the Drawings

[0030] Figure 1 It is a flowchart of a method for constructing an information security scoring system based on artificial intelligence according to the present invention.

[0031] Figure 2 It is another schematic diagram of the method for constructing an information security scoring system based on artificial intelligence of the present invention, namely, a directory tree of text information of the industry standard "Guidelines for Evaluation of Cybersecurity Level Protection in the Financial Industry".

[0032] Figure 3 It is another schematic diagram of a method for constructing an information security scoring system based on artificial intelligence of the present invention, namely, a directory tree of text information of the industry standard "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements". [Specific implementation method]

[0033] S1: Use the crawler framework webmagic to crawl industry-standard online information security-related standard texts as raw data. The specific implementation is as follows: first, extract the target link based on the seed link and put it into the queue to be crawled; then parse and download the standard text data from the page, webmagics uses the Jsoup component to parse the HTML page; finally, store the extracted standard text data in a text file format or store it in a database.

[0034] S2: According to the standard text data crawled in S1, one of the standard text data is obtained as the first standard text data, and all other standard text data are traversed. During the traversal, one of the standard text data is used as the second standard text data, and the directory containing the word "requirement" in the directory of the first standard text data is obtained as the first directory, and the directory containing the word "requirement" in the directory of the second standard text data is obtained as the second directory, and the edit distance of each row of directories in the first directory and the second directory is traversed by the python-Levenshtein toolkit, and the ratio of the number of subdirectories with an edit distance less than 2 to the number of subdirectories of the first directory is calculated as the directory overlap ratio. When the directory overlap ratio exceeds a preset value, the second standard text data is used as the suspected update standard of the first standard text data, for example:

[0035] First, the first standard text data crawled by the crawler is: "Financial Industry Cybersecurity Level Protection Evaluation Guide" and the second standard text data "Financial Industry Cybersecurity Level Protection Implementation Guide Part 2: Basic Requirements" two industry standard documents. Assume that the corresponding old industry standard is "Financial Industry Cybersecurity Level Protection Evaluation Guide";

[0036] For each industry standard document, extract the text information whose name contains the keyword "requirements" in the second-level directory of the file; and use the python-Levenshtein toolkit to traverse the edit distance of each row of the first directory and the second directory, and calculate the sub-directories with an edit distance less than 2. For example, the text information extracted from the "Financial Industry Cybersecurity Level Protection Assessment Guide" includes: 6 Second-level Assessment Requirements, 7 Third-level Assessment Requirements, 8 Fourth-level Assessment Requirements All text information under the three second-level directories; and the text information extracted from the "Financial Industry Cybersecurity Level Protection Implementation Guide Part 2: Basic Requirements" includes: 7 Second-level security requirements, 8 Third-level security requirements, 9 Fourth-level security requirements All text information under the three second-level directories.

[0037] According to the "Second Level Requirements", there are four points below. For the "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements", they are: general security requirements, cloud computing security extension requirements, mobile Internet security extension requirements, and Internet of Things security extension requirements; and for the "Financial Industry Cybersecurity Level Protection Assessment Guide", they are: general security assessment requirements, cloud computing security assessment extension requirements, mobile Internet security assessment extension requirements, and Internet of Things security assessment extension requirements;

[0038] According to the catalog under “General Security Requirements”, it includes: secure physical environment, secure communication network, secure area boundaries, etc.;

[0039] Based on the secure physical environment, two industry-standard physical location text messages are as follows:

[0040] 7.1.1.1 Physical location selection

[0041] This requirement includes:

[0042] a) The computer room should be located in a building that is earthquake-proof, wind-proof and rain-proof.

[0043] b) The computer room should not be located on the top floor or basement of a building, otherwise waterproofing and moisture-proof measures should be strengthened.

[0044] 6.1.1.1 Physical location selection

[0045] Assessment Unit (L2-PES1-01)

[0046] This assessment unit includes the following requirements:

[0047] a) Evaluation indicators: The computer room should be located in a building that is earthquake-proof, wind-proof, and rain-proof.

[0048] b) Evaluation objects: record form documents and computer rooms.

[0049] c) The implementation of the assessment includes the following:

[0050] 1) Check whether the building has the building seismic fortification approval document.

[0051] 2) Check whether there is any rainwater leakage in the computer room.

[0052] 3) Check that the doors and windows of the computer room are free of severe dust caused by wind.

[0053] 1) Check whether the roof, walls, doors, windows and floors are damaged or cracked.

[0054] d) Unit judgment: If 1) to 4) are all affirmative, then the unit meets the requirements of this assessment unit; otherwise, it does not meet or partially meets the requirements of this assessment unit.

[0055] Assessment Unit (L2-PES1-02)

[0056] This assessment unit includes the following requirements:

[0057] a) Evaluation indicators: The computer room should not be located on the top floor or basement of a building, otherwise waterproofing and moisture-proof measures should be strengthened.

[0058] b) Evaluation object: computer room.

[0059] c) Assessment implementation: It should be verified whether the computer room is not located on the top floor or basement of the building. If not, it should be verified whether waterproof and moisture-proof measures have been taken in the computer room.

[0060] d) Unit judgment: If the above assessment implementation content is positive, it meets the requirements of this assessment unit indicator; otherwise, it does not meet the requirements of this assessment unit indicator.

[0061] The requirements and standards in the "Financial Industry Cybersecurity Level Protection Evaluation Guide" include all the contents in the "Financial Industry Cybersecurity Level Protection Implementation Guide Part 2: Basic Requirements". The "Financial Industry Cybersecurity Level Protection Evaluation Guide" contains all the requirements of the "Financial Industry Cybersecurity Level Protection Implementation Guide Part 2: Basic Requirements", and the directory overlap ratio exceeds the preset value of 80%. As a result, the system determines that the two industry standards are too similar. Therefore, they cannot be distinguished. If no distinction is made, the system will parse the "Financial Industry Cybersecurity Level Protection Implementation Guide Part 2: Basic Requirements" and extract the text information to update the industry standard operation. This will directly lead to update errors.

[0062] S3: According to step S2, the suspected updated standard text is obtained, and the hierarchical similarity calculation method is used to determine the industry standard text that needs to be updated. Specifically, it includes: performing hierarchical similarity calculation on the corresponding multi-level directories. In the process of hierarchical similarity calculation, the directories containing specific keywords are extracted, and the directories that do not contain specific keywords are filtered out. The standard text directory tree is constructed through the text analysis method; the sum of the similarities between the content texts under each subdirectory of the two standard files is calculated through the bm25 similarity algorithm to determine whether it exceeds the threshold. When the similarity is higher than the threshold, the standard file with a later release date is used as the industry standard text that needs to be updated for the file with an earlier release date. When the similarity is not higher than the threshold, it is difficult to determine whether it should be updated. The industry standard text that needs to be updated is determined by the method of fusion of the two file data, which specifically includes:

[0063] Through text analysis methods, a standard text directory tree is constructed. For example, based on the two text data of the industry standard "Guidelines for the Evaluation of Cybersecurity Level Protection in the Financial Industry" and the industry standard "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements", through text analysis methods, the text information of the industry standard "Guidelines for the Evaluation of Cybersecurity Level Protection in the Financial Industry" is regarded as a directory tree, and the text information of the industry standard "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements" is regarded as a directory tree. Figure 2 and Figure 3 shown.

[0064] (b) Calculate the sum of the similarities between the content texts in each subdirectory of the two standard files through the bm25 similarity algorithm to determine whether it exceeds the threshold. When the similarity is higher than the threshold, the file with a later release date is used as the industry standard file for updating the file with an earlier release date. When the similarity is not higher than the threshold, it is difficult to determine whether it should be updated. The industry standard file is updated by fusing the data of the two files. For example: Calculate the distance between the content texts in each subdirectory of the industry standard "Guidelines for the Evaluation of Cybersecurity Level Protection in the Financial Industry" and the industry standard "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements". Compare the similarity between the text information of the industry standard "Guidelines for the Evaluation of Cybersecurity Level Protection in the Financial Industry" and the text information of the industry standard "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements". To calculate the similarity between two text information, the bm25 similarity algorithm is used to calculate the sum of the similarities between all directory data points of the two text information. The larger the sum of the similarities, the higher the similarity between the two standard data. When the similarity is higher than the threshold, the file with a later release date replaces the file with an earlier release date as the updated industry standard file. When the similarity is not higher than the threshold, it is difficult to determine whether it should be updated. The industry standard file is updated by fusing the two file data.

[0065] (c) Update industry standard documents by fusing the data of two documents, for example: Figure 2 , each secondary topic under each level topic in the two directories is regarded as a data point. Figure 2 It can be seen that the data points under the "Second Level Assessment Requirements (A)" in the first-level topic are: General Requirements for Security Assessment (A2-1),

[0066] Cloud computing security assessment extension requirements (A2-2), mobile Internet security assessment extension requirements (A2-3), Internet of Things security assessment extension requirements (A2-4). Other secondary topics are divided in the same way. Figure 3The data points in the secondary topics are: general security requirements (D2-1), cloud computing security extension requirements (D2-2), mobile Internet security extension requirements (D2-3), Internet of Things security extension requirements (D2-4), etc. In order to facilitate the calculation of the distance between each data point, a label is given to each system, such as Figure 2 (A2-1), (A2-2), (A2-3), (A2-4), etc. described in (A2-1), (A2-2), (A2-3), (A2-4), etc.

[0067] (d) Calculate the similarity between the data points for the first-level topics corresponding to the first-level topics and the second-level topics corresponding to the second-level topics in the two directories, and merge and update the two data points with the highest similarity. For example, through calculation, it can be known that the similarity between the data point (A2-1) in the industry standard "Guidelines for the Evaluation of Cybersecurity Level Protection in the Financial Industry" and the data point (D2-1) in the industry standard "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements" is the highest. Therefore, merge the data point (A2-1) and the data point (D2-1) into a new data point New_A2-1. Merge the data point (A2-2) and the data point (D2-2) into a new data point New_A2-2. Similarly, the other ones are similar, constantly calculating the distance between data points, and finding the two data points with the smallest distance, and merging them into a new data point.

[0068] (e) According to the principle of corresponding first-level topics and first-level topics described in step (d), the newly combined data points are merged into the corresponding first-level topics. For example, the combined data points New_A2-1, New_A2-2, New_A2-3 and New_A2-4 are merged into a new first-level topic corresponding to the data point New_A. The other combined data points are merged in the same way to form a new first-level topic. Finally, all the newly generated first-level topic data points are merged to generate a new directory tree New_1. The meaning of this new directory tree New_1 is: the original industry standard document "Guidelines for the Evaluation of Cybersecurity Level Protection in the Financial Industry" is combined with the crawled industry standard document "Guidelines for the Implementation of Cybersecurity Level Protection in the Financial Industry Part 2: Basic Requirements" to perform text clustering and update into a new industry standard file New_1, that is, the directory tree New_1.

[0069] (f) using the newly generated industry standard file in step (e) as the industry standard file that needs to be updated.

[0070] S4: Structural extraction is performed on the industry standard file to be updated through preset text matching rules to obtain structured standard data, which includes industry standard field names and their corresponding scoring standard contents, and the structured standard data is manually classified into two types: non-network security type and automatically testable network security type. The unified preset rule of the structured standard is expressed as: {[industry standard field name], [corresponding scoring standard content]}.

[0071] The non-network security type is a type that cannot pass the automatic test security score test.

[0072] The automatically testable network security type refers to a type whose security score can be automatically tested by a testing system.

[0073] For example, based on the industry standard text of "Information Security Technology IPv6-based High-Performance Network Audit System Product Security Technical Requirements", grammatical analysis technology is used to extract field information features from the text information; wherein, the field information features include field type and field content corresponding to the field type. For this industry standard document, the extracted text information is all text information whose title contains the keyword "requirements" in its first-level title. Filter out other unnecessary first-level title text information. For example, the first-level titles are: "Security Function Requirements", "Environmental Adaptability Requirements", "Performance Requirements", and "Safety Assurance Requirements". For each second-level title, extract the title of its corresponding last-level sub-topic as one of the keys in the feature value, and the requirements corresponding to its text information as the value corresponding to this key.

[0074] In the module "Security Function Requirements", you will find that the standard requirement information in this industry standard document includes both non-network security types and automatically testable network security types. The five modules of "Information Collection", "Data Restoration", "Management Control Requirements", "Safety Management", and "Data Storage" all belong to the non-network security type after the text information extracted is vectorized. The remaining four modules of "Audit Record Statistics", "Audit Record Analysis and Processing", "Identification and Authentication", and "Audit Log" all belong to the automatically testable network security type after the text information is vectorized. The details are as follows: For the non-network security type, "Data Storage", according to its last level title, the extracted keys are: "Storage Media", "Data Deletion", "Backup and Recovery", etc.

[0075] The data format after text information standardization is as follows:

[0076] {'Data deletion': [a) should be able to record basic information about the deletion behavior, including time, date, operator, and description of the deleted content; b) should be able to set policies to automatically delete data that exceeds the retention period],

[0077] 'Data deletion scoring criteria': [a) Failure to record basic information about the deletion, including time, date, operator, and description of the deleted content, will result in a deduction of 1 point; b) Failure to set a policy to automatically delete data that has exceeded the retention period, will result in a deduction of 1 point]}

[0078] {'Backup and restore': [IPv6 network audit products should provide audit record backup and restore functions],

[0079] 'Backup and recovery scoring criteria': [IPv6 network audit products that do not provide audit record backup and recovery functions will be deducted 1 point]}

[0080] For the type of network security that can be automatically tested, "Audit Record Analysis and Processing", based on its last level title, the extracted keys are: "Correlation Analysis", "Abnormal Analysis", "Response Warning", etc.

[0081] The data format after text information standardization is as follows:

[0082] {'Abnormal analysis': [a) can predefine abnormal events, can trigger alarms when the number of occurrences in a certain time period or the frequency domain reaches a certain threshold or a certain flow reaches a threshold, b) can define abnormal behavior based on the results of the correlation analysis function, and trigger alarms for abnormal behavior of the system, c) other abnormal situations],

[0083] 'Abnormal analysis scoring criteria': [a) Ability to predefine abnormal events, and trigger alarms when the number of occurrences in a certain time period or the frequency domain reaches a certain threshold or a certain flow reaches a threshold, deduct 1 point, b) Ability to define abnormal behavior based on the results of the correlation analysis function, and trigger alarms for abnormal behavior of the system, deduct 1 point, c) Other abnormal situations, deduct 1 point]}

[0084] {'Correlation analysis': [a) basic information correlation, including information collected based on time, event, source IP address, source port address, destination IP address, service type, network protocol, etc., b) statistical correlation, using data mining algorithms and other methods to perform correlation analysis between different network events],

[0085] 'Correlation analysis scoring criteria': [a) No basic information correlation, including information collected based on time, event, source IP address, source port address, destination IP address, service type, network protocol, etc., deduct 0.5 points; b) No statistical correlation, using data mining algorithms and other methods to perform statistical correlation analysis between different network events, deduct 0.5 points]}

[0086] {'Response to warning': [a) The product should support policy settings to trigger alarms, b) It should be able to record alarms, and the content should include: date, time, event subject, event topic, event level, event description, alarm number, event result, c) The alarm method should support at least one of the following methods: email alarm, SNMPtrap alarm, audio and video alarm, SMS alarm, etc.],

[0087] 'Response to warning scoring criteria': [a) If the product cannot support policy settings to trigger alarms, deduct 1 point; b) If the product cannot record alarms, the content should include: date, time, event subject, event topic, event level, event description, number of alarms, and event results, deduct 1 point; c) If the alarm method does not support at least one of the following methods: email alarm, SNMPtrap alarm, audio and video alarm, SMS alarm, etc., deduct 1 point]}

[0088] S5: Based on the manually classified structured standard data obtained in step S4, a binary classifier is trained using the Bayesian classification method, one class is a non-network security type, and the other class is an automatically testable network security type. The input parameters of the Bayesian classification include the field name of the structured standard data and the text information included in the field content. The prediction result of the Bayesian classification is a non-network security type or an automatically testable network security type.

[0089] S6: According to step S3, the industry standard text that needs to be updated is obtained, and by evaluating the differences between the new and old standard texts and the differences between the new and old scoring standard texts, the mapping relationship between the two differences is mined, and the mapping relationship is effectively modeled. The differences specifically include two aspects: 1. The degree of change of the standard text at different times 2. The degree of change of the scoring standard. In particular, the difference characteristics of the standard text can also be subdivided into three aspects: numerical value, degree adverb and quantity change. For example:

[0090] The numerical description in the standard text has changed: the old standard text: the safety function of the product should cover 80%; the new standard text: the safety function of the product should cover 90%. The change is that the numerical feature changes from "80%" to "90%".

[0091] The degree adverbs in the standard text have changed: the old standard text: partially describes the safety function of the product; the new standard text: completely describes the safety function of the product. The change is that the degree adverb feature changes from "partial" to "completely".

[0092] The quantities in the standard text have changed:

[0093] Old standard text: Developers should provide complete functional specifications, which should meet the following requirements: a) fully describe the product's security functions; b) describe the purpose and usage of all security function interfaces; c) identify and describe all parameters related to each security function interface; d) describe the security function implementation behavior related to the security function interface; e) describe the direct error messages caused by the security function implementation behavior processing; f) describe the traceability of security function requirements to security function interfaces;

[0094] New standard text: Developers should provide complete functional specifications, which should meet the following requirements: a) fully describe the product's security functions; b) describe the purpose and usage of all security function interfaces; c) identify and describe all parameters related to each security function interface; d) describe the security function implementation behavior related to the security function interface; e) describe direct error messages caused by the processing of security function implementation behavior; f) describe the traceability of security function requirements to security function interfaces; g) describe all behaviors related to security function interfaces during the security function implementation process; The change is that the number of standards has increased from "6" to "7".

[0095] S7: Based on the standard text obtained in step S3, the difference evaluation system identifies the updated industry standard content. The difference evaluation system includes a feature recognition model. Specifically, it includes:

[0096] The feature recognition model is essentially an entity recognition model, which aims to identify the entity words in the given text. For example, "Tomorrow is the Mid-Autumn Festival", "Mid-Autumn Festival" is a "festival" entity. Therefore, this step uses the entity recognition model to identify the "feature" entity in the given text. The present invention only involves industry standards and scoring standards in the field of information security. The text involved has a strong regularity, so the rule-based named entity recognition method can achieve better results. First, the entities (features) in the training sample are taken out, a feature word dictionary is established, and then the sequence is annotated, and finally the entity is identified by the regular matching method. Take the identification of the features in "Describing the safety function of the product should cover 90%" as an example: first, the sentence is segmented [Describing the safety function of the product should cover 90%], and then the part-of-speech tagging tool is used for annotation. The annotation rule is that if the word is in the feature word dictionary, it is marked as E, and the rest are annotated according to the part of speech given in the part-of-speech tagging tool. The obtained sequence is [VNDETNADVVE]. For this type of text, regular matching according to the obtained sequence rule can identify the feature entity of this type of text. The model is used to perform feature recognition on the industry standard text and scoring standard text to be processed. Then, the LSTM network is used to encode the text with identified features, and finally the multi-layer perceptron is used to perform nonlinear changes on the LSTM encoded sentences to achieve semantic understanding of the features. The labeled sequence is used as a feature and input into the LSTM network together with the label for training, where the output dimension is 50, the activation function is tanh, and then a 0.5 Dropout layer is added, the activation function is a fully connected layer with softmax, the optimizer uses adam, and the monitoring indicator uses accuracy. Specifically, the feature recognition module recognizes the numerical features in the given industry standard text "The description of the safety function of the product should cover 90%" and the scoring standard text "The description of the safety function of the product should cover if it only reaches 80%, deduct 1 point; the description of the safety function of the product should cover if it only reaches 70%, deduct 2 points; less than 70%, deduct 5 points (full score 5 points)". After identifying the numerical features, the text understanding module is used to perform semantic understanding on the identified features to achieve modeling of the mapping relationship between such industry standard texts and scoring standard texts. For example, there is a correlation between the change in the numerical characteristics of the industry standard and the degree of change in the scoring standard. When the numerical characteristics of the industry standard are adjusted from "80%" to "

[0097] When it reaches "90%", the corresponding numerical feature of the scoring standard also changes from "80%" to "90%", and the characteristic of the change is that the changes of the two features are consistent.

[0098] S8: According to the method of steps S1-S7, the standard text is continuously updated.

[0099] S9: The automatic testing system automatically evaluates the network environment that needs to be tested according to the structured standard data of the updated standard text obtained in steps S1-S8 to obtain a test score p1; the reviewer manually evaluates the network environment that needs to be tested to obtain a test score p2;

[0100] The automatic testing system has a preset software automatic testing tool, which can automatically evaluate the network environment according to the industry standard field name or the corresponding scoring standard content in the structured standard of the standard text to obtain the evaluation score;

[0101] The reviewer manually evaluates the network environment that needs to be tested, including: mining the mapping relationship between the structured standard data of the updated standard text obtained in steps S1-S8 and the difference between the new and old scoring standard texts and pushing them to the reviewer, so as to assist the reviewer in manually evaluating the network environment;

[0102] Compare the results of p1 and p2 to see if they are consistent, and process the updated standard text according to the comparison results;

[0103] If they are consistent, the updated standard text is judged to be accurate, and the updated standard text and the corresponding structured standard data are stored in the database;

[0104] If the results are inconsistent, it is determined by default that the updated standard text field is updated incorrectly, resulting in a score difference, and the reviewer is asked to list the updated incorrect fields and the reasons for the errors. Based on the updated incorrect fields and the reasons for the errors listed by the reviewer, the structured text matching rules of the standard text are updated.

[0105] If the reviewer does not list the specific field update error, it is judged that the updated standard document contains non-network security fields, and the score cannot be automatically measured by the automatic testing system. Obtain the industry standard field name that causes the score difference and the corresponding scoring standard content, mark it as a non-network security type, and retrain the binary classifier in step 5.

Claims

1. A method for constructing an information security scoring system based on artificial intelligence, characterized in that: include: Crawl industry standard online information security related standard texts; According to the crawled standard texts, one of the standard texts is obtained as the first standard text data, and all other standard texts are traversed. During the traversal, one of the standard texts is used as the second standard text data, and the directory containing the word "requirement" in the directory of the first standard text data is obtained as the first directory, and the directory containing the word "requirement" in the directory of the second standard text data is obtained as the second directory, and the edit distance of each row of directories in the first directory and the second directory is traversed by the python-Levenshtein toolkit, and the ratio of the number of subdirectories with an edit distance less than 2 to the number of subdirectories of the first directory is calculated as the directory overlap ratio. When the directory overlap ratio exceeds a preset value, the second standard text data is used as the suspected updated standard text of the first standard text data; Determine the industry standard text that needs to be updated by using a hierarchical similarity calculation method according to the suspected updated standard text and the first standard text data; Performing structured extraction on the industry standard text that needs to be updated through preset text matching rules to obtain structured standard data; According to the structured standard data, manual classification is performed, and a binary classifier is trained by a Bayesian classification method, wherein the input parameters of the Bayesian classification method include the field name of the structured standard data and the text information included in the field content, and the prediction result of the Bayesian classification method is a non-network security type or a network security type that can be automatically tested; According to the industry standard text that needs to be updated, by evaluating the difference between the new and old standard texts and the difference between the new and old scoring standard texts, a mapping relationship between the two differences is mined; Through the automatic testing system, the network environment that needs to be tested is automatically evaluated according to the structured standard data of the industry standard text that needs to be updated, and the evaluation score p1 is obtained; the reviewer manually evaluates the network environment that needs to be tested, and obtains the evaluation score p2; Compare the results of p1 and p2 to see if they are consistent, and process the updated industry standard text according to the comparison results; The automatic testing system automatically evaluates the network environment that needs to be tested according to the structured standard data of the industry standard text that needs to be updated, and obtains the evaluation score p1; the reviewer manually evaluates the network environment that needs to be tested, and obtains the evaluation score p2; including: the automatic testing system presets the software automatic testing tool, and automatically evaluates the network environment according to the industry standard field name in the structured standard data or the corresponding scoring standard content, and obtains the evaluation score p1; The reviewer manually evaluates the network environment that needs to be tested, evaluates the differences between the new and old standard texts, and the differences between the new and old scoring standard texts, and mines out the mapping relationship between the two differences and pushes it to the reviewer, assisting the reviewer to manually evaluate the network environment to obtain the scoring score p2; The comparison between p1 and p2 to determine whether the results are consistent and the updated industry standard text is processed according to the comparison result include: If they are consistent, the updated industry standard text is judged to be accurate, and the updated industry standard text and the corresponding structured standard data are stored in the database; If the results are inconsistent, it is determined by default that the fields of the updated industry standard text are updated incorrectly, resulting in the difference in scores, and the reviewer is asked to list the fields that were updated incorrectly and the reasons for the errors; based on the fields that were updated incorrectly and the reasons for the errors listed by the reviewer, the structured text matching rules of the industry standard text are updated; If the reviewer does not list the specific field update errors, it is judged that the updated industry standard text contains non-network security fields, and the scores cannot be automatically evaluated by the automatic testing system; obtain the industry standard field names and corresponding scoring standard contents that cause the score differences, mark them as non-network security types, and retrain the binary classifier.

2. The method for constructing an information security scoring system based on artificial intelligence according to claim 1, characterized in that: The crawling industry standard online information security related standard texts include: According to the seed link, extract the target link and put it into the queue to be crawled; parse and download the standard text from the page, and use the Jsoup component through webmagics to parse the HTML page; store the extracted standard text in text file format or in the database.

3. The method for constructing an information security scoring system based on artificial intelligence according to claim 1, characterized in that: The step of determining the industry standard text that needs to be updated by using a hierarchical similarity calculation method according to the suspected update standard text and the first standard text data includes: The corresponding multi-level directory is subjected to hierarchical similarity calculation, wherein the corresponding multi-level directory refers to the multi-level directory of the first standard text data and the second standard text data. In the process of hierarchical similarity calculation, the directory containing specific keywords is extracted, and the directory not containing specific keywords is filtered out. The standard text directory tree is constructed through the text analysis method; the sum of the similarities between the content texts under each subdirectory in the first standard text data and the second standard text data is calculated through the bm25 similarity algorithm to determine whether it exceeds the threshold. When the similarity is higher than the threshold, the standard text with a later publication date is used as the industry standard text that needs to be updated for the file with an earlier publication date. When the similarity is not higher than the threshold, it is difficult to determine whether it should be updated. The industry standard text that needs to be updated is determined by the method of fusing the two standard text data.

4. The method for constructing an information security scoring system based on artificial intelligence according to claim 1, characterized in that: The industry standard text to be updated is structured extracted by using preset text matching rules to obtain structured standard data, including: the structured standard data includes industry standard field names and corresponding scoring standard contents, and the structured standard data is manually classified into two types: non-network security type and automatically testable network security type. The unified preset rules of the structured standard are expressed as: {[industry standard field name], [corresponding scoring standard content]}; The non-network security type is a type that cannot pass the automatic test security score test; The automatically testable network security type refers to a type that automatically tests the security score through a testing system.

5. The method for constructing an information security scoring system based on artificial intelligence according to claim 1, characterized in that: According to the industry standard text that needs to be updated, by evaluating the differences between the new and old standard texts and the differences between the new and old scoring standard texts, the mapping relationship between the two differences is excavated; including: the differences include three aspects: numerical values, degree adverbs and quantity difference changes.

Images