Data trace analysis capability evaluation method and system based on network security
By defining and weighting the key evaluation indicators of data trace analysis, an automated testing environment is built, and the weight value is set using hierarchical analysis method, which solves the shortcomings of the evaluation method of data trace resolution capabilities in the existing technology, achieves more comprehensive, systematic and scientific evaluation, and improves the testing efficiency and practicality of the results.
Patent Information
- Application Number
- CN202411826209.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-25
AI Technical Summary
The existing data trace resolution capability evaluation methods lack systematicity and comprehensiveness, fail to fully consider the specific needs in different business scenarios, the key indicator weight setting is not scientific enough, the testing environment is expensive and flexible, data analysis lacks structured methods, and test case design and execution rely on manual operations, which is inefficient.
Provide a method for evaluating data trace resolution capabilities based on network security, including obtaining data trace resolution standards, defining analysis capability evaluation indicators, building a network testing environment, executing tests through automated test scripts, collecting and analyzing test results, and generating a comprehensive evaluation report. The weight value is set using the analysis method of hierarchy to ensure the scientificity and fairness of the evaluation.
By defining and weighting the key evaluation indicators for data trace analysis, we ensure the comprehensiveness and systematicity of the evaluation, improve the practicality and effectiveness of the evaluation results, reduce the cost of the test environment, improve the testing efficiency and accuracy, ensure the consistency and repeatability of the experiment, and provide in-depth analysis and improvement suggestions through structured reports.
Smart Images
Figure CN119892703A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data trace analysis, and in particular to a method and system for evaluating data trace analysis capability based on network security. Background Art
[0002] With the rapid development and widespread application of network technology, network security issues are becoming more and more complex, and the requirements for data trace analysis capabilities are also increasing. However, the existing data trace analysis capability evaluation methods have many shortcomings and are difficult to meet current needs.
[0003] First, traditional evaluation standards often lack systematicity and comprehensiveness, focusing only on individual aspects and failing to fully reflect the comprehensive performance of security systems. Second, these standards usually do not fully consider the specific needs of different business scenarios, resulting in a disconnect between the evaluation results and actual applications. The relative importance weights between key indicators are not scientific enough, which affects the fairness and rationality of the evaluation results. In terms of the test environment, relying on real hardware or simplified virtual settings is not only costly and inflexible, but also difficult to simulate diverse and realistic network threat scenarios. In addition, the design and execution of test cases mostly rely on manual operations, which are inefficient and prone to errors. Especially when dealing with a large number of complex test cases, manual execution is almost impossible to ensure that each step is carried out correctly. At the same time, due to the low degree of automation, the environmental recovery work before and after each test becomes very difficult, affecting the consistency and repeatability of the experiment. Finally, in the data analysis stage, most existing practices lack a structured method to integrate the performance of various indicators and give clear improvement suggestions. Many reports simply list some data, lack in-depth cause analysis and specific optimization strategies, and are not very helpful in guiding subsequent security improvement work. Therefore, we propose a data trace analysis capability evaluation method based on network security. Summary of the invention
[0004] In view of the above-mentioned problems, the present invention is proposed.
[0005] Therefore, the technical problems solved by the present invention are: the existing data trace analysis capability evaluation methods do not fully consider the specific needs under different business scenarios, the key indicator weight setting is not scientific enough, the test environment is costly and has poor flexibility, and there is a lack of structured methods for data analysis, as well as the technical problems of how to realize the automatic design and execution of test cases.
[0006] To solve the above technical problems, the present invention provides the following technical solutions: a data trace analysis capability evaluation method based on network security, comprising obtaining data trace analysis standards and defining analysis capability evaluation indicators; building a network test environment, testing data traces and collecting test results; analyzing and scoring the data according to the test results, and generating a comprehensive evaluation report.
[0007] As a preferred solution of the data trace analysis capability evaluation method based on network security described in the present invention, wherein: the acquisition of data trace analysis standards includes collecting and studying existing data trace analysis standards and best practices.
[0008] As a preferred solution of the data trace analysis capability evaluation method based on network security described in the present invention, wherein: the definition of the analysis capability evaluation index includes defining core key indicators based on security requirements and business scenarios, and setting weight values for each core key indicator; the setting of weight values includes calculating using the hierarchical analysis method.
[0009] As a preferred solution of the data trace analysis capability evaluation method based on network security described in the present invention, wherein: the construction of the network test environment includes creating a test environment based on network simulation software, setting hardware and software components, deploying applications that generate log records, designing and implementing network threat scenarios, operating the test environment according to the test requirements of target users, planning test input conditions, expected outputs and evaluation criteria according to test scenarios, and executing preset operation sequences according to automated test scripts; after the test is completed, the test environment is restored to its initial state and multiple rounds of testing are performed.
[0010] As a preferred solution of the data trace analysis capability evaluation method based on network security described in the present invention, wherein: the testing of data traces and collection of test results include recording the interaction process generated by executing a preset operation sequence according to an automated test script, tracking the response of the system under test during the test by tracking special tools, and backing up and archiving the log files and test data generated during the test; the special tracking tools include Wireshark, Tcpdump and Security Onion.
[0011] As a preferred solution of the data trace analysis capability evaluation method based on network security described in the present invention, wherein: the data analysis and scoring includes collecting and summarizing the test data obtained in each round of testing, generating a structured report document, analyzing the report document according to the industry standards of the target user or the internally set target values, scoring each core key indicator based on the scoring rules of the defined analysis capability evaluation indicators, and performing weighted averaging to obtain the total score; setting a core key indicator score threshold, marking the core key indicators below the score threshold as low-scoring indicators, analyzing the reasons for the generation of the low-scoring indicators, and generating improvement suggestions based on the reasons for the generation.
[0012] As a preferred solution of the data trace analysis capability evaluation method based on network security described in the present invention, wherein: the generation of a comprehensive evaluation report includes evaluating and summarizing the overall test results, generating the test performance of each indicator and the analysis reasons for the test results, using a chart tool to generate a key indicator data change trend chart, and generating a low-scoring indicator optimization plan and evaluation report;
[0013] After passing the internal review, the final version of the evaluation report will be published.
[0014] Another object of the present invention is to provide a data trace analysis capability evaluation system based on network security, which can set weights by adopting hierarchical analysis method, thereby solving the problem that the current network security technology has unscientific key indicator weight setting in data trace analysis capability evaluation.
[0015] As a preferred solution of the data trace analysis capability evaluation system based on network security described in the present invention, it includes: an indicator definition module, a data trace testing module, and an evaluation report generation module; the indicator definition module is used to obtain the data trace analysis standard and define the analysis capability evaluation indicator; the data trace testing module is used to build a network test environment, test the data trace and collect the test results; the evaluation report generation module is used to analyze and score the data according to the test results, and generate a comprehensive evaluation report.
[0016] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for evaluating data trace analysis capability based on network security.
[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for evaluating data trace analysis capability based on network security.
[0018] The beneficial effects of the present invention are as follows: the data trace analysis capability evaluation method based on network security provided by the present invention ensures the comprehensiveness and systematicness of the evaluation by defining and weighting the key evaluation indicators of data trace analysis, and more accurately reflects the comprehensive performance of the security system; by considering the specific needs under different business scenarios, the evaluation results are closer to actual applications, and the practicability and effectiveness of the evaluation results are improved; by setting the weight value by adopting the hierarchical analysis method, not only the scientificity and rationality of the key indicator weights are ensured, but also the evaluation results are made more fair and reasonable; by using network simulation tools to create a virtual network, the cost of the test environment is reduced, the flexibility is improved, and the test environment can simulate diverse and realistic network threat scenarios; by automating the test script, manual operations are reduced, and the test efficiency and accuracy are improved; by restoring to the initial state before each test, the consistency and repeatability of the experiment are guaranteed; by forming a structured report document in the data analysis stage, and measuring the performance with reference to industry standards or internal target values, the data is analyzed more deeply, and the problems existing in the data and their possible causes are evaluated more accurately; by generating a comprehensive evaluation report and officially publishing it to the outside after internal review, the professionalism and transparency of the entire evaluation process are ensured. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0020] Figure 1 An overall flow chart of a method for evaluating data trace analysis capability based on network security provided in the first embodiment of the present invention.
[0021] Figure 2 An overall flow chart of a data trace analysis capability evaluation system based on network security provided in the third embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0023] Example 1, reference Figure 1, as an embodiment of the present invention, provides a method for evaluating data trace analysis capability based on network security, comprising:
[0024] S1: Obtain the data trace analysis standard and define the evaluation index of the analysis capability.
[0025] Furthermore, obtaining data trace parsing standards involves collecting and studying existing data trace parsing standards and best practices.
[0026] It should be noted that the definition of analytical capability evaluation indicators includes defining core key indicators based on security requirements and business scenarios, and setting weight values for each core key indicator; setting weight values includes calculation using the hierarchical analysis method.
[0027] It should also be noted that the core key indicators include but are not limited to parsing speed, accuracy, false positive rate and missed negative rate.
[0028] It should also be noted that the analytic hierarchy process is used for calculation. The specific steps are: determine the analytical ability evaluation index, decompose the factors affecting the target into multiple levels, the first level is the analytical ability evaluation target, the second level is the analytical ability evaluation sub-target, and the third level is the specific analytical scheme of the analytical ability evaluation index. Each evaluation element in the same level is compared with each evaluation element in the previous level, and a judgment matrix is constructed. In the judgment matrix, the importance ratio between each two is determined through expert scoring or data analysis. For the judgment matrix, the characteristic vector of the analytical ability evaluation index is calculated. The characteristic vector of the analytical ability evaluation index is the relative weight of each evaluation index. The weight is normalized through standardization so that the sum of the relative weights is 1. The consistency of the judgment matrix is tested, the consistency ratio is calculated, and the allowable range of CR value is set. If the CR value is within the allowable range, the consistency of the judgment matrix is passed, otherwise the error value in the judgment matrix is readjusted. The weight vectors of each level are combined and calculated to obtain the comprehensive weight of each scheme relative to the highest target. The core indicators are ranked according to the calculated comprehensive weight, and the contribution of each indicator in the overall evaluation is ranked.
[0029] S2: Build a network test environment, test data traces and collect test results.
[0030] Furthermore, building a network test environment includes creating a test environment based on network simulation software, setting up hardware and software components, deploying applications that generate log records, designing and implementing network threat scenarios, operating the test environment according to the test requirements of target users, planning test input conditions, expected outputs and evaluation criteria according to test scenarios, and executing preset operation sequences according to automated test scripts; after the test, the test environment is restored to its initial state and multiple rounds of testing are performed.
[0031] It should be noted that testing data traces and collecting test results includes recording the interactive process generated by executing a preset sequence of operations according to an automated test script, tracking the response of the system under test during the test by using special tracking tools, and backing up and archiving the log files and test data generated during the test; special tracking tools include Wireshark, Tcpdump, and Security Onion.
[0032] It should also be noted that hardware and software components include, but are not limited to, servers, firewalls, intrusion detection systems, and intrusion prevention systems.
[0033] S3: Analyze and score the data based on the test results and generate a comprehensive evaluation report.
[0034] Furthermore, data analysis and scoring includes collecting and summarizing the test data obtained from each round of testing, generating structured report documents, analyzing the report documents according to the target user's industry standards or internally set target values, scoring each core key indicator based on the defined scoring rules for the analysis capability evaluation indicators, and performing weighted average to obtain the total score; setting a core key indicator score threshold, marking the core key indicators below the score threshold as low-scoring indicators, analyzing the reasons for the generation of low-scoring indicators, and generating improvement suggestions based on the reasons.
[0035] It should be noted that generating a comprehensive evaluation report includes evaluating and summarizing the overall test results, generating the test performance of each indicator and analyzing the reasons for the test results, using chart tools to generate key indicator data change trend charts, generating low-scoring indicator optimization plans and evaluation reports; after passing the internal review, the final version of the evaluation report will be published.
[0036] Example 2 is an embodiment of the present invention, which provides a method for evaluating data trace analysis capabilities based on network security. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
[0037] First, this example verifies the effectiveness and superiority of the data trace analysis capability evaluation method proposed in the present invention, and compares it with the existing method.
[0038] Among them, the test environment is configured, the network simulation platform uses Mininet, the hardware components include servers, firewalls, intrusion detection systems, software components include firewall software, intrusion detection software, data trace analysis tools, the application is a network application that generates diversified logs, and the network threat scenarios include DDoS attacks, SQL injection attacks, and malicious code injections.
[0039] Experimental steps: collect and study existing data trace analysis standards and best practices; define core key indicators based on security requirements and business scenarios; define core key indicators including analysis speed, accuracy, false alarm rate, and missed alarm rate; use hierarchical analysis method to set weight values; use Mininet to create a virtual network; configure servers, firewalls, and intrusion detection systems; deploy network applications that generate diverse logs; design and implement network threat scenarios; determine network intrusion detection scenarios; input attack data; the expected output is the attack type and characteristics; the evaluation criteria are recognition accuracy; develop automated test scripts, execute preset operation sequences, and record the interaction process; confirm that the test cases run correctly in the constructed environment; restore to the initial state before each test to ensure the consistency of the test; use Wireshark, Tcpdump, and Security Onion to track the response of the tested system; back up all relevant log files and data; perform data analysis and scoring; summarize the overall evaluation situation; list the specific performance and cause analysis of each indicator; use charts to present the trend of key data changes; provide professional opinions on optimization plans or new measures; and publish the final version of the evaluation report after internal review.
[0040] Table 1 Comparison of test data
[0041] Test subjects Parsing speed (ms) accuracy(%) False alarm rate (%) Missing rate (%) Total score Existing tools 152 84 4.5 9.8 76.1 Method of the present invention 98 92 2.3 1.7 91.6 Existing tools 185 72 7.8 11.3 63.1 Method of the present invention 105 95 1.9 0.8 92.4
[0042] It can be seen from the data in Table 1 that in terms of parsing speed, the method of the present invention is significantly better than existing tools in parsing speed, with an average speed of more than 40% faster, and can more quickly identify and track network attack behaviors; in terms of accuracy, the accuracy of the method of the present invention is higher than that of existing tools, and can more accurately identify and classify different types of attacks, reducing the false alarm rate and missed alarm rate; in terms of false alarm rate, the method of the present invention has a lower false alarm rate than existing tools, which can effectively reduce the misjudgment of normal network behavior and improve the reliability of the security system.
[0043] In summary, the method of the present invention outperforms existing tools in all indicators, and the total score is significantly improved, indicating that the method has stronger data trace analysis capabilities and can more effectively ensure network security.
[0044] Example 3, reference Figure 2 , as an embodiment of the present invention, provides a data trace analysis capability evaluation system based on network security, including an indicator definition module, a data trace testing module, and an evaluation report generation module.
[0045] The indicator definition module is used to obtain the data trace analysis standard and define the analysis capability evaluation index; the data trace testing module is used to build a network test environment, test the data trace and collect the test results; the evaluation report generation module is used to analyze and score the data according to the test results and generate a comprehensive evaluation report.
[0046] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0047] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0048] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.
[0049] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
[0050] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A data trace analysis capability evaluation method based on network security, characterized in that: include: Obtain data trace analysis standards and define analysis capability evaluation indicators; Build a network test environment, test data traces and collect test results; The data is analyzed and scored based on the test results, and a comprehensive evaluation report is generated.
2. The method for evaluating data trace analysis capability based on network security according to claim 1, characterized in that: The obtaining of data trace analysis standards includes collecting and studying existing data trace analysis standards and best practices.
3. The method for evaluating data trace analysis capability based on network security according to claim 2, characterized in that: Defining the analytical capability evaluation index includes defining core key indicators based on security requirements and business scenarios, and setting weight values for each core key indicator; The setting of the weight value includes calculating by using a hierarchical analysis method.
4. The method for evaluating data trace analysis capability based on network security according to claim 3, characterized in that: The construction of the network test environment includes creating a test environment based on network simulation software, setting up hardware and software components, deploying applications that generate log records, designing and implementing network threat scenarios, operating the test environment according to the test requirements of target users, planning test input conditions, expected outputs and evaluation criteria according to test scenarios, and executing preset operation sequences according to automated test scripts; After the test is completed, the test environment will be restored to its initial state and multiple rounds of testing will be performed.
5. The method for evaluating data trace analysis capability based on network security according to claim 4, characterized in that: The testing of data traces and collection of test results includes recording the interactive process generated by executing a preset operation sequence according to the automated test script, tracking the response of the system under test during the test process through a dedicated tracking tool, and backing up and archiving the log files and test data generated during the test process; The tracing-specific tools include Wireshark, Tcpdump, and Security Onion.
6. The method for evaluating data trace analysis capability based on network security according to claim 5, characterized in that: The data analysis and scoring includes collecting and summarizing the test data obtained in each round of testing, generating a structured report document, analyzing the report document according to the industry standards of the target users or the internally set target values, scoring each core key indicator based on the defined scoring rules of the analytical capability evaluation indicators, and performing a weighted average to obtain a total score; Set a core key indicator score threshold, mark the core key indicators below the score threshold as low-scoring indicators, analyze the reasons for the low-scoring indicators, and generate improvement suggestions based on the reasons.
7. The method for evaluating data trace analysis capability based on network security according to claim 6, characterized in that: Generating a comprehensive evaluation report includes evaluating and summarizing the overall test results, generating the test performance of each indicator and analyzing the reasons for the test results, using chart tools to generate key indicator data change trend charts, and generating low-scoring indicator optimization plans and evaluation reports; After passing the internal review, the final version of the evaluation report will be published.
8. A system using the network security-based data trace analysis capability evaluation method according to any one of claims 1 to 7, characterized in that: Including indicator definition module, data trace testing module, and evaluation report generation module; The indicator definition module is used to obtain data trace analysis standards and define analysis capability evaluation indicators; The data trace test module is used to build a network test environment, test data traces and collect test results; The evaluation report generation module is used to analyze and score the data according to the test results and generate a comprehensive evaluation report.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for evaluating data trace analysis capability based on network security described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data trace analysis capability evaluation method based on network security described in any one of claims 1 to 7 are implemented.