Network security actual combat skill assessment method and system based on artificial intelligence analysis

Abstract

The invention discloses a network security actual combat skill assessment method and system based on artificial intelligence analysis, and relates to the technical field related to network security, and the method comprises the steps: building a calibration test database, and building a first behavior sequence log of a target user; establishing a grade matching coefficient of the test environment; establishing a second behavior sequence log of the target user; executing multi-dimensional feature operation evaluation of mapping result backtracking, and establishing a first skill evaluation result; establishing a second skill evaluation result according to the first behavior sequence log, the second behavior sequence log and the dynamic test database; and fusing the first skill evaluation result and the second skill evaluation result, and outputting a network security actual combat skill evaluation result. The technical problem that in the prior art, actual combat skills of network security personnel in a complex real environment are difficult to accurately evaluate, and consequently the overall network security protection level is poor is solved, accurate network security skill evaluation is achieved, and the technical effect of improving the network security protection level is achieved.

Description

Technical Field

[0001] The present application relates to the technical field related to network security, and specifically to a method and system for evaluating practical network security skills based on artificial intelligence analysis. Background Art

[0002] The importance of cybersecurity is becoming increasingly prominent. As cyberattack methods evolve from traditional malware and phishing to more sophisticated and insidious advanced persistent threats (APTs), existing cybersecurity skills assessment methods are no longer able to meet the demands of today's rapidly changing network environment. Early cybersecurity skills assessments relied heavily on theoretical knowledge assessments, such as tests on basic knowledge of network security protocols and encryption algorithms. These tests failed to effectively assess operational response capabilities in real-world cyberattack scenarios, nor did they effectively assess operators' operational path selection, the temporal rationality of their actions, their understanding of attack semantics, and their resilience and remediation capabilities. Subsequently, skills assessments based on simulated environments emerged. These assessments, which simulated attacks and defenses in specific network scenarios, failed to capture the complex and ever-changing nature of real-world network conditions and failed to reflect the dynamic changes in cyberattack methods in real time. Furthermore, the assessment of the assessee's actions was mostly based on behavioral records, making it difficult to delve into complex characteristics, such as the rationality of operational paths and the logic of temporal behavior. This, in turn, impacted the accuracy and comprehensiveness of cybersecurity assessments.

[0003] Therefore, at the current stage, relevant technologies have the technical problem of difficulty in accurately evaluating the practical skills of network security personnel in complex real environments, resulting in a poor overall level of network security protection. Summary of the Invention

[0004] This application solves the technical problem in the existing technology that it is difficult to accurately evaluate the practical skills of network security personnel in complex real environments, resulting in a poor overall level of network security protection, by providing a method and system for evaluating practical network security skills based on artificial intelligence analysis. It realizes accurate network security skills evaluation and achieves the technical effect of improving the level of network security protection.

[0005] The present application provides a method for evaluating practical network security skills based on artificial intelligence analysis, the method comprising: establishing a calibration test database, which is a general test database, using the calibration test database to conduct target user testing, and establishing a first behavior sequence log of the target user; synchronizing the first behavior sequence log and the calibration test database to a grade evaluation network, and establishing a grade matching coefficient of the test environment; using the grade matching coefficient to establish a dynamic test database, conducting user testing of the target user based on the dynamic test database, and establishing a second behavior sequence log of the target user; after mapping the second behavior sequence log with the dynamic test database, performing a multi-dimensional feature operation evaluation of the mapping result backtracing, and establishing a first skill evaluation result, the multi-dimensional features including operation path features, temporal behavior features, semantic analysis features, and strain repair features; establishing a second skill evaluation result based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database; fusing the first skill evaluation result and the second skill evaluation result to output a practical network security skill evaluation result.

[0006] The present application also provides a network security practical skills assessment system based on artificial intelligence analysis, the system including: a first behavior sequence log establishment module, used to establish a calibration test database, the calibration test database is a general test database, the calibration test database is used to perform target user testing, and establish a first behavior sequence log of the target user; a level matching coefficient establishment module, used to synchronize the first behavior sequence log and the calibration test database to a level evaluation network, and establish a level matching coefficient of the test environment; a second behavior sequence log establishment module, used to establish a dynamic test database using the level matching coefficient, perform user testing of the target user based on the dynamic test database, and establish a second behavior sequence log of the target user; a first skill evaluation result establishment module, used to map the second behavior sequence log with the dynamic test database, perform multi-dimensional feature operation evaluation of the mapping result backtracing, and establish a first skill evaluation result, the multi-dimensional features including operation path features, temporal behavior features, semantic analysis features, and strain repair features; a second skill evaluation result establishment module, used to establish a second skill evaluation result based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database; a skill evaluation result fusion module, used to fuse the first skill evaluation result and the second skill evaluation result, and output a network security practical skills evaluation result.

[0007] The proposed method and system for evaluating cybersecurity practical skills based on artificial intelligence analysis will establish a calibration test database, a first behavior sequence log for the target user, a level matching coefficient for the test environment, a second behavior sequence log for the target user, a multi-dimensional feature operation evaluation based on back-tracing of the mapping results, and a first skill evaluation result. The second skill evaluation result will be established based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database. The first and second skill evaluation results will be integrated to output a cybersecurity practical skills evaluation result. This solves the technical problem in the prior art of making it difficult to accurately evaluate the practical skills of cybersecurity personnel in complex real-world environments, resulting in a poor overall cybersecurity protection level. This allows for accurate cybersecurity skill evaluation and achieves the technical effect of improving cybersecurity protection levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings of the embodiments of the present disclosure are briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in precise order. Instead, various steps may be processed in reverse order or simultaneously as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0009] Figure 1 A flowchart of a method for assessing practical cybersecurity skills based on artificial intelligence analysis is provided in an embodiment of the present application.

[0010] Figure 2 Schematic diagram of the structure of the network security practical skills assessment system based on artificial intelligence analysis provided in an embodiment of the present application.

[0011] Explanation of the reference numerals: first behavior sequence log establishment module 10 , level matching coefficient establishment module 20 , second behavior sequence log establishment module 30 , first skill evaluation result establishment module 40 , second skill evaluation result establishment module 50 , skill evaluation result fusion module 60 . DETAILED DESCRIPTION

[0012] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0013] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0014] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict, and the terms “first\second” involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. The terms “including” and “having” and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application only.

[0015] The present application embodiment provides a network security practical skills assessment method based on artificial intelligence analysis, such as Figure 1 As shown, the method includes:

[0016] Step S100 : establishing a calibration test database, which is a general test database. Target user testing is performed using the calibration test database to establish a first behavior sequence log of the target user.

[0017] Preferably, a variety of common and representative network security test scenarios, data types and attack modes are used to conduct preliminary tests on different target users to obtain data with general reference value, integrate and build a calibration test database, that is, a general test database, and then use the calibration database to test the target users (that is, people whose network security practical skills need to be evaluated). Specifically, the target users are tested in a simulated network security environment, including performing corresponding operations according to the test scenarios and tasks set in the calibration database, such as responding to different types of network attacks, executing security configurations, etc.; and the target users' operational behaviors are recorded and sequenced according to their operational order, that is, a first behavioral sequence log, which records in detail the specific operations and sequence of the target users when facing various test situations in the calibration test database, which is helpful for subsequent in-depth analysis of the users' operating habits, etc. Assuming that a scenario simulating hacker intrusion is set up in the calibration test database, the target users may perform multiple operations such as detecting signs of intrusion, taking defensive measures, and attempting to track attackers during the test, which are recorded to form the first behavioral sequence log.

[0018] Step S200: Synchronize the first behavior sequence log and the calibration test database to a level evaluation network to establish a level matching coefficient of the test environment.

[0019] Preferably, the grade evaluation network is an artificial intelligence model that can convert the input relevant information into a quantitative indicator of the degree of match between the test environment and the target user's ability, namely the grade matching coefficient. Specifically, the first behavior sequence log generated by the target user in the calibration test database environment, as well as various information contained in the calibration test database itself (such as the difficulty and type of the test scenario), are input into the grade evaluation network for analysis to evaluate the target user's actual operation behavior and the specific conditions of the test environment. That is, the grade evaluation network analyzes and processes the input first behavior sequence log and the calibration test database information, and compares and matches the target user's behavior during the test with factors such as the difficulty and requirements of the test environment. If the target user can quickly and accurately perform effective operations when facing a more complex test scenario, it means that the user's ability is more matched with the difficulty level of the test environment, and a higher grade matching coefficient is given; conversely, if the user's operation has many errors or cannot effectively cope with the test scenario, the grade evaluation network gives a lower grade matching coefficient. This more accurately measures the target user's performance in a specific test environment and its adaptation to the difficulty level of the environment.

[0020] Furthermore, step S200 also includes step S210, calling the test completion unit of the grade evaluation network to perform completion evaluation of the first behavior sequence log and establish a first-level influence factor; step S220, calling the completion speed unit of the grade evaluation network to perform completion speed evaluation of the first behavior sequence log and establish a second-level influence factor, wherein the second-level influence factor is configured with an influence coefficient of the first-level influence factor; step S230, establishing the grade matching coefficient based on the first-level influence factor, the second-level influence factor, and the influence coefficient.

[0021] Preferably, the test completion unit is a functional module in the grade evaluation network, which is used to evaluate the degree of completion of the target user's tasks during the test process. Specifically, the test completion unit analyzes the first behavior sequence log based on the test tasks and goals pre-set in the calibration test database, determines whether the user has completed all necessary operation steps, and evaluates the quality of completion. For example, in a test simulating network attack defense, the target user may be required to complete attack detection, isolation of infected devices, repair of system vulnerabilities, etc., and then generates a quantitative indicator based on the completion evaluation result, namely the first-level impact factor, wherein the higher the completion degree, the larger the first-level impact factor, indicating that the user performs better in completing the test task, thereby positively affecting the grade matching coefficient.

[0022] Preferably, the completion speed unit is also a functional module of the graded evaluation network, which is used to evaluate the operation speed of the target user during the test. Specifically, the completion speed unit extracts the time information of the operation from the first behavior sequence log, calculates the total time taken by the user to complete each test task, and then compares this time with the pre-set standard time to evaluate the user's completion speed. For example, to complete the network configuration operation within the specified time, the completion speed unit determines whether the target user completes the operation ahead of schedule, on time, or out of time based on the first behavior sequence log, and then generates a second-level impact factor based on the completion speed evaluation result. The shorter the time to complete the operation, the larger the second-level impact factor, indicating that the user performs well in terms of operation speed, and the second-level impact factor is configured with the influence coefficient of the first-level impact factor, that is, the size of the second-level impact factor depends not only on the completion speed itself, but also on the first-level impact factor (completion degree), reflecting the correlation between the two factors.

[0023] Preferably, a level matching coefficient is established based on the first-level influencing factor, the second-level influencing factor and the influence coefficient. Specifically, the second-level influencing factor is first adjusted according to the influence coefficient. That is, if the user's test completion is high, the importance of completion speed will increase accordingly according to the influence coefficient to more comprehensively reflect the user's comprehensive performance. Then, weights are allocated according to the evaluation needs and the degree of importance attached to completion and completion speed. The target user's test completion and completion speed are comprehensively evaluated through weighted summation to calculate the level matching coefficient, which is used to accurately reflect the degree of matching between the user's skill level and the difficulty of the test environment.

[0024] Step S300 : establishing a dynamic test database using the level matching coefficient, performing a user test on a target user based on the dynamic test database, and establishing a second behavior sequence log of the target user.

[0025] Preferably, if the level matching coefficient is high, it means that the user performs well in the general test and may need a more challenging test environment to further evaluate his or her actual skill level. If the level matching coefficient is low, the difficulty of the test environment may need to be reduced to more accurately evaluate the user's actual practical skill level; then the test database is dynamically adjusted according to the level matching coefficient to establish a dynamic test database. Specifically, for users with high level matching coefficients, the complexity, quantity or difficulty level of the test tasks in the database are increased, such as in network security testing, more advanced hacker attack scenario simulations, advanced persistent threat (APT) simulated attacks, or users are required to complete more security defense operations in a shorter time; for users with low level matching coefficients, the test tasks are simplified, or more prompts and guidance information are provided to make the test environment more suitable for their actual ability level; and then a test database that can dynamically change according to the user's actual performance is established, that is, a dynamic test database.

[0026] Preferably, a dynamic test database is used to conduct user testing on the target user, that is, the target user operates according to the test tasks and scenarios set in the dynamic test database in a simulated network security environment, wherein the test tasks may include vulnerability detection, intrusion prevention, emergency response, etc. For example, the user needs to detect potential vulnerabilities within a specified time and take corresponding repair measures; or when suffering a network attack, quickly activate the emergency response mechanism to block the attack and restore the normal operation of the system; at the same time, during the test process, the user's operation behavior is recorded in detail, including the specific action of the operation (such as the command executed, the button clicked, etc.), the timestamp of the operation, the target object of the operation (such as a specific network device, file, etc.), and the result of the operation. For example, when the user executes a vulnerability scanning command, the specific content of the command, the execution time, the target host scanned, and the scanning results (the number and type of vulnerabilities found, etc.) are recorded; then these operation records are combined to form a second behavior sequence log of the target user, so as to fully present the user's operation process and behavior pattern in the dynamic test database environment, and compared with the first behavior sequence log, it can better reflect the user's ability and adaptability when adapting to tests of different difficulty levels.

[0027] Step S400: After mapping the second behavior sequence log with the dynamic test database, perform a multi-dimensional feature operation evaluation based on backtracing of the mapping result to establish a first skill evaluation result. The multi-dimensional features include operation path features, temporal behavior features, semantic analysis features, and strain repair features.

[0028] Preferably, each behavior record in the second behavior sequence log is mapped and associated with the relevant test tasks, scenarios and expected results in the dynamic test database, and the specific test content and goals corresponding to each behavior of the user in a specific test environment are clarified. Then, a multi-dimensional feature (including operation path features, timing behavior features, semantic analysis features, and strain repair features) operation evaluation of the mapping result backtracking is performed. Specifically, the operation path feature evaluation refers to analyzing the operation path of the user in the process of completing the test task, that is, evaluating whether the user directly adopts the optimal path to solve the problem or completes the test goal after multiple attempts, such as completing the vulnerability repair task, evaluating whether the user can quickly locate the vulnerability and choose the appropriate repair method, or determine the solution after performing a large number of meaningless operations.

[0029] Preferably, the temporal behavior feature evaluation refers to analyzing the time sequence and time interval of the user's operation behavior, judging whether the user follows a reasonable time sequence when performing the relevant operations of the test task, and whether the time interval between each operation step is reasonable. For example, in an emergency response scenario, whether the user can quickly take key measures within the specified time (such as promptly cutting off the network connection to prevent the spread of the attack), and whether the repair operation is performed at a reasonable time interval to avoid excessive pauses or delays. The semantic analysis feature evaluation refers to evaluating the user's understanding of the professional terms, operation instructions and problems in the test task, that is, analyzing whether the user's operation complies with network security specifications and semantic requirements. For example, when performing a network attack simulation task, whether the attack instructions entered by the user are accurate, whether the meaning of various parameters can be correctly understood and reasonably configured, and whether the attack strategy can be accurately adjusted according to the feedback information to reflect a deep understanding of network attacks.

[0030] Preferably, the contingency and repair feature evaluation is primarily used to assess whether users can respond effectively and perform repairs in a timely manner when encountering unexpected situations or changes in the test environment. This means observing whether users can quickly adjust their strategies and take effective repair measures to restore the system to normal when faced with unexpected problems during the test process, such as additional vulnerabilities being triggered or changes in attack methods. For example, when a new vulnerability appears after fixing one, can users quickly analyze the relationship between the two and take appropriate repair measures, rather than being unable to cope. Finally, the multi-dimensional feature evaluation results are combined to comprehensively evaluate the target user's network security practical skills, obtaining a first skill evaluation result that describes the user's level of operation path selection, temporal behavior rationality, semantic understanding accuracy, and contingency and repair capabilities. For example, if a user selects an efficient operation path, has a reasonable temporal behavior, accurate semantic analysis, and performs well in contingency and repair, and can quickly respond to various emergencies and effectively solve problems, the first skill evaluation result is excellent. Conversely, if the user has obvious deficiencies in multiple dimensions, including confusing operation paths, frequent time delays, misunderstanding of professional terminology, and inability to cope with emergencies, the evaluation result is poor.

[0031] Furthermore, step S400 also includes step S411, extracting the target user's operation behavior according to the second behavior sequence log and establishing an operation path diagram; step S412, using the mapping result to backtrack and call the skill strategy of the dynamic test database; step S413, performing node coverage analysis of the operation path diagram and the skill strategy, and establishing a node matching rate feature; step S414, performing the longest common subpath identification of the operation path diagram and the skill strategy, and establishing an operation sequence similarity feature; step S415, performing meaningless command and skip command identification of the operation path diagram, and generating a strategy anomaly coefficient feature according to the identification result; step S416, establishing an operation path feature evaluation according to the node matching rate feature, the operation sequence similarity feature, and the strategy anomaly coefficient feature.

[0032] Preferably, each operation behavior of the target user and its related information (such as operation time, operation object, etc.) are extracted from the second behavior sequence log, and the operation behavior is used as a node, connected with directed edges in the order of the operation to form an operation path diagram, which represents the user operation process and intuitively shows the specific operation steps and paths taken by the user in completing the test task; then, through the mapping relationship between the second behavior sequence log and the dynamic test database, the test tasks and scenarios corresponding to the user operations are found, and the skill strategies preset in the dynamic test database include recommended operation steps for completing the test tasks, best practices, and handling methods for various situations; then, a node coverage analysis of the operation path diagram and the skill strategy is performed, that is, the nodes in the operation path diagram are compared with the nodes in the skill strategy (that is, the preset key operation steps), and the number of nodes in the operation path diagram that can match the nodes in the skill strategy is counted, and then the proportion of the nodes in the total number of nodes in the skill strategy is calculated, that is, a node matching rate feature is established, wherein the higher the node matching rate, the closer the user's operation path is to the preset skill strategy, reflecting that the user's execution of the test task is more in line with the specifications and best practices.

[0033] Preferably, the longest common sub-path in the operation path diagram and the skill strategy is found, that is, the longest path in the user's actual operation that is exactly the same as the preset skill strategy in operation sequence is found, and the operation sequence similarity feature is obtained by calculating the ratio of the length of the longest common sub-path to the total length of the skill strategy, which is used to measure the similarity between the user's operation sequence and the standard skill strategy, so as to reflect whether the user follows a reasonable operation process and logical sequence when performing the test task; then the commands in the operation path diagram are analyzed to identify meaningless commands that are of no practical help in completing the test task and skip commands that do not conform to the normal operation route and skip necessary intermediate steps, and then calculate and generate a strategy anomaly coefficient feature based on the number and frequency of occurrence of meaningless commands and skip commands, among which the higher the strategy anomaly coefficient, the more abnormal behaviors that do not conform to the conventional skill strategy are in the user's operation, reflecting that the user has an inaccurate understanding of the test task or is not proficient in operation. Finally, the node matching rate characteristics, operation sequence similarity characteristics and strategy anomaly coefficient characteristics are comprehensively considered to evaluate the user's operation path characteristics. For example, if the node matching rate is high, the operation sequence similarity characteristics are high and the strategy anomaly coefficient is low, it means that the user's operation path is both in line with the specifications and has high rationality and fluency, and the operation path characteristics are evaluated as excellent; on the contrary, if at least one of the three characteristics performs poorly, the user's operation path problems are analyzed according to the specific situation, such as insufficient mastery of skill strategies, unclear operation logic, etc., so as to further improve the user's practical skills level.

[0034] Furthermore, step S400 also includes step S421, constructing an operation interval sequence according to the second behavior sequence log, and performing task segmentation identification of the operation interval sequence through the mapping result; step S422, after using the task segmentation identification to segment the operation interval sequence, configuring the time window, counting the operation frequency in each time window, and establishing a density vector; step S423, performing spectrum conversion of the density vector through fast Fourier transform, extracting the main rhythm frequency, high-frequency component, and low-frequency component, and establishing an operation rhythm feature; step S424, performing pause window recognition of the operation time series, and establishing a pause window; step S425, performing pause anomaly recognition according to the mapping result corresponding to the pause window, and establishing a pause anomaly feature; step S426, using the operation rhythm feature and the pause anomaly feature to establish a timing behavior feature evaluation.

[0035] Preferably, the timestamp of each target user's operation is extracted from the second behavior sequence log, and the time interval between adjacent operations is calculated to obtain an operation interval sequence that reflects the change in the time interval between user operations. Then, the specific test task or subtask to which each operation belongs is determined by mapping the second behavior sequence log with the dynamic test database. The operation interval sequence is marked according to this task information, and the test task stage corresponding to each time interval is clearly defined, so that different test task stages can be analyzed separately. Then, according to the task segmentation identifier, the operation interval sequence is segmented according to different tasks to obtain an operation interval subsequence corresponding to each test task, so as to analyze the user's operation time pattern for different tasks separately. A fixed-length time window is configured on the operation interval subsequence corresponding to each task, and the number of user operations in each time window, i.e., the operation frequency, is counted to observe the density of user operations in different time intervals. The operation frequency in each time window is used as an element to form a vector, i.e., a density vector, which reflects the frequency distribution of user operations in different time windows, and indirectly reflects the changes in user operation rhythm and activity in the process of completing the task.

[0036] Preferably, the density vector is processed using a Fast Fourier Transform (FFT) to convert it from the time domain to the frequency domain. The Fast Fourier Transform (FFT) is used to decompose the time series into a combination of sine and cosine waves of different frequencies, thereby obtaining its spectrum information. Specifically, the main rhythm frequency in the spectrum is identified, representing the most important rhythm or cycle in the user's operation; at the same time, high-frequency components and low-frequency components are separated. The high-frequency components correspond to rapid changes or short-term fluctuations in the operation, reflecting the user's handling of emergency situations or details; the low-frequency components are related to long-term trends and slow changes in the operation, such as the overall progress of the task. By analyzing and quantifying the main rhythm frequency, high-frequency components, and low-frequency components, an operation rhythm feature is established to help understand the user's operation rhythm patterns in the process of completing the task, such as whether the user tends to operate quickly and frequently or is more accustomed to slow and steady operations, and how the operation rhythm changes at different task stages.

[0037] Preferably, in the operation time sequence, the time period in which the user has not performed any operation for a long time, i.e., the pause window, is identified and determined, wherein the pause may be when the user is thinking or encountering difficulties, and the start time and end time of each pause window are determined, thereby establishing a pause window, the length and position of the pause window can reflect the pause situation of the user during the operation; then, combined with the mapping results corresponding to the pause window, the test tasks, scenarios and expected operation processes corresponding to the pause window are analyzed to determine whether the pause conforms to the normal task execution logic and expected situation. For example, in a certain task stage, a long pause should not normally occur, but the user has it, or the length of the pause exceeds a reasonable range, which is regarded as a pause abnormality. According to the situation of the pause abnormality, such as the number of pauses, duration, task stage in which it occurs, etc., a pause abnormality feature is established to describe the abnormal degree of the user's pause behavior during the operation.

[0038] Preferably, the user's timing behavior characteristics are evaluated by comprehensively considering the operation rhythm characteristics and pause abnormality characteristics. If the user's operation rhythm is stable, the main rhythm frequency is obvious, the high-frequency and low-frequency components are reasonable, and there are few pause abnormalities, it means that the user's timing behavior is relatively standardized and stable, and the task can be completed at a reasonable rhythm. There are no obvious abnormal pauses during the operation, and the timing behavior characteristics are evaluated as good. On the contrary, if the user's operation rhythm is chaotic, there is no obvious main rhythm frequency, the high-frequency or low-frequency components are abnormal, or there are many pause abnormalities, such as frequent long pauses, pauses at critical task stages, etc., it means that there are problems with the user's timing behavior, which may affect the efficiency and quality of task completion, and then analyze the causes of the abnormality, including the user's familiarity with the task, the degree of concentration, etc., to put forward improvement suggestions.

[0039] Furthermore, step S425 also includes step A1, using the mapping result to obtain the semantic context of the pause window; step A2, identifying the pause types of thinking pauses and technical pauses based on the semantic context; step A3, completing pause abnormality identification based on the pause type identification result and duration, and establishing pause abnormality features.

[0040] Preferably, the semantic context of the pause window is determined through the mapping result of the second behavior sequence log and the dynamic test database, that is, the relevant information corresponding to the pause window in the test database, including the operation task, specific scenario, related commands or operation descriptions, etc. For example, the pause window appears in a certain task step, and the semantic context includes the task description of the step and the surrounding operation information; then the thinking pause is identified and judged based on the semantic context, that is, whether the pause is caused by user thinking, and then the technical jam is identified and judged based on the semantic context, wherein the technical jam is a jam caused by the system, software, hardware, etc. that makes the user operation unable to continue, that is, check whether the semantic context has clues related to technical problems, such as system error information, prompts of failed operation execution, network connection problems, etc.

[0041] Preferably, the type and duration of the pause are comprehensively considered to determine whether the pause is abnormal. Specifically, for thinking pauses, if their duration is within a reasonable range, that is, consistent with the complexity of the task and the time required for normal thinking, it is defined as normal and not abnormal. If the thinking pause is too long, exceeding the reasonable time range determined by task difficulty and experience, it may be considered abnormal. For technical freezes, even if the freeze is short, it may be considered abnormal, especially when technical freezes occur frequently. Finally, based on the abnormal pause situation, such as the number of abnormal pauses, the distribution of different types of abnormal pauses, the duration of each abnormal pause, etc., a pause abnormality feature is established to help describe in detail the abnormal pause behavior during the user operation, ensuring accurate subsequent judgment of user operation behavior and skill level assessment.

[0042] Furthermore, step S426 also includes step B1, executing macro-operation identification of the second behavior sequence log to establish an erroneous operation branch; step B2, extracting the pause duration of the branch node according to the erroneous operation branch, and establishing an abnormal pause thinking feature using the pause duration extraction result and the branch length of the erroneous operation branch; step B3, adding the abnormal pause thinking feature to the timing behavior feature evaluation.

[0043] Preferably, the second behavior sequence log is executed for macro-operation identification, that is, the macro-operations performed by the user are identified from the overall level, including the overall operation set. For example, in network security testing, a complete vulnerability scan and a deployment of defense measures against specific attacks are macro-operations. By analyzing and judging the macro-operations, it is determined that there are erroneous steps, such as entering incorrect commands, selecting incorrect defense strategies, missing key steps, etc., and then the erroneous operations are identified and extracted, and erroneous operation branches are established to clearly show the impact of the erroneous operations and how the user responds after discovering the error. Then, in the erroneous operation branch, the pause duration between each branch node (i.e., each operation) is extracted to reflect the user's pause in the process of performing the erroneous operation. The pause duration extraction result and the branch length of the erroneous operation branch are then comprehensively considered to establish the abnormal pause thinking feature, where the branch length refers to the number of operations from the start of the erroneous operation to the end of the branch. Specifically, if the pause duration of the node in the erroneous operation branch is too long, and the location and frequency of the long pause are unreasonable in combination with the branch length, it is considered that there is an abnormal pause thinking situation. Finally, features describing the abnormal pause thinking situation are established, such as the average abnormal pause duration, the ratio of the number of abnormal pauses to the branch length, etc., and the abnormal pause thinking feature is added to the temporal behavior feature evaluation to more comprehensively reflect the user's temporal behavior performance when an erroneous operation occurs.

[0044] Furthermore, step S400 also includes step S431, classifying the target user's operation into a semantic action label according to the second behavior sequence log; step S432, extracting the semantic action label and establishing a semantic behavior chain according to the time series; step S433, performing feature extraction of semantic fluency features, semantic integrity features, and semantic deviation features through the semantic behavior chain, and establishing a semantic analysis feature evaluation based on the feature extraction results.

[0045] Preferably, the operations of the target user in the second behavior sequence log are analyzed and classified into different semantic action labels according to the meaning of the operations. For example, in a network security practical scenario, operations such as "enter login password", "click submit button", and "view system log" are respectively marked as semantic action labels such as "identity authentication", "data submission", and "system monitoring" to convert specific operations into labels with semantic meanings; then, from the classified semantic action labels, they are extracted in sequence according to the chronological order of the operations to form an ordered sequence, namely a semantic behavior chain, which reflects the operation process and behavior pattern of the target user, and at the same time embodies the logical relationship and semantic coherence between the operations.

[0046] Preferably, semantic fluency features, semantic integrity features, and semantic deviation features are extracted through the semantic behavior chain. Specifically, the semantic fluency feature mainly examines whether the conversion between each semantic action label in the semantic behavior chain is natural and smooth. It can be extracted by analyzing the conversion frequency between semantic action labels, the semantic correlation between adjacent labels, etc., and then evaluate the coherence and rationality of the user operation behavior at the semantic level; the semantic integrity feature focuses on whether the semantic behavior chain contains all key semantic action labels. By comparing the standard task process, it is determined whether important semantic action labels are missing in the semantic behavior chain, thereby extracting the semantic integrity feature, and then measuring whether the user operation completely and comprehensively covers all key links of the task.

[0047] Preferably, the semantic deviation feature refers to the degree of difference between the semantic behavior chain and the expected standard behavior pattern. By comparing the user's semantic behavior chain with the predefined standard behavior pattern, the degree of difference between the two is calculated, thereby extracting the semantic deviation feature, and then evaluating whether the user operation conforms to the conventional operation logic and specifications; finally, the extraction results of the semantic fluency feature, the semantic integrity feature and the semantic deviation feature are combined to conduct a comprehensive semantic analysis and evaluation of the target user's operation behavior to understand the user's semantic understanding and expression ability during the operation process, the degree of mastery of the task process and whether the conventional operation specifications are followed. If the user's semantic fluency is high, the semantic integrity is good and the semantic deviation is low, it means that the user has a strong understanding and execution ability of network security practical tasks, and the operation behavior conforms to the specifications and has good logic; on the contrary, if a certain feature performs poorly, it is possible to analyze the user's deficiencies in a targeted manner and further improve the user's skill level.

[0048] Step S500: Establish a second skill evaluation result based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database.

[0049] Step S500 further includes step S510, establishing growth indicators, stability indicators, and adaptability indicators; step S520, using the growth indicators, stability indicators, and adaptability indicators to call the dynamic test database to perform indicator evaluation of the first behavior sequence log and the second behavior sequence log, and establishing the second skill evaluation result based on the indicator evaluation results.

[0050] Preferably, by comparing the user's operating behavior, task completion status, etc. in the first behavior sequence log and the second behavior sequence log, the user's skill improvement at different stages is analyzed, and a growth indicator is established, wherein the growth indicator is used to measure the target user's progress and development in skill application; the consistency of the user's operation, the fluctuation of the error rate, etc. are analyzed from the behavior sequence log, and a stability index is established, wherein the stability index is used to evaluate the stability of the user's skill performance in the process of executing the task; combined with various test scenarios and condition changes in the dynamic test database, how the user responds to these changes in the first behavior sequence log and the second behavior sequence log is observed, and an adaptability index is established, wherein the adaptability index is used to examine the user's adaptability to different task scenarios, environmental changes and new requirements.

[0051] Preferably, the growth index is applied to the dynamic test database, and the user's skill growth level is evaluated by comparing the test data of the user at different stages recorded in the database with the relevant information in the first behavior sequence log and the second behavior sequence log, such as the growth rate and degree in specific skill areas (such as vulnerability mining, emergency response, etc.), to determine the user's skill growth level; with the help of multiple rounds of test data in the dynamic test database and the details of user operations in the behavior sequence log, the user's actual skill stability in different test scenarios is analyzed, such as observing the accuracy of the detection results and the fluctuation range of the detection time in the user's multiple consecutive network security vulnerability detection tasks. If the user can maintain a similar detection accuracy and stable detection time in different rounds of tests, it means that the stability is good. On the contrary, if the data fluctuates greatly, it indicates that the stability needs to be improved.

[0052] Preferably, the user's adaptability is evaluated based on the various test environments and task requirements set in the dynamic test database, as well as the user's actual operational performance in the first and second behavior sequence logs. For example, when the test database simulates different types of network attack scenarios (such as DDoS attacks and SQL injection attacks), the user's response in the behavior sequence log is observed to determine whether the user can quickly adjust the strategy and effectively defend against it, thereby judging the user's ability to cope with the complex and ever-changing actual network security situations. Finally, the evaluation results of the growth index, stability index, and adaptability index are combined to generate a second skill evaluation result, which more comprehensively and objectively reflects the user's skill status in actual network security operations, thereby providing more targeted guidance for the user's skill improvement and development.

[0053] Step S600: integrate the first skill evaluation result and the second skill evaluation result to output a network security practical skill evaluation result.

[0054] Preferably, the weights of the first skill evaluation result and the second skill evaluation result in the final network security practical skill evaluation result are determined according to different evaluation purposes and actual needs. For example, if it is believed that the adaptability and growth potential in actual combat are more important, the second skill evaluation result is given a higher weight, such as 60%, while the first skill evaluation result accounts for 40%; and the first skill evaluation result and the second skill evaluation result are standardized, and then weightedly fused according to the determined weights to obtain the network security practical skill evaluation result, including the user's performance in each skill dimension, and improvement suggestions for the user's deficiencies, so as to comprehensively and accurately evaluate their own network security practical skill level, thereby improving the level of network security protection.

[0055] In the above, refer to Figure 1 The network security practical skills assessment method based on artificial intelligence analysis according to an embodiment of the present invention is described in detail. Figure 2 The present invention describes a network security practical skills assessment system based on artificial intelligence analysis according to an embodiment of the present invention.

[0056] The network security practical skills assessment system based on artificial intelligence analysis according to the embodiment of the present invention is used to solve the technical problem that it is difficult to accurately assess the practical skills of network security personnel in complex real environments, resulting in a poor overall network security protection level. It realizes accurate network security skills assessment and achieves the technical effect of improving the level of network security protection. Figure 2 As shown, the network security practical skills assessment system based on artificial intelligence analysis includes: a first behavior sequence log establishment module 10, a level matching coefficient establishment module 20, a second behavior sequence log establishment module 30, a first skill evaluation result establishment module 40, a second skill evaluation result establishment module 50, and a skill evaluation result fusion module 60.

[0057] A first behavior sequence log establishment module 10 is used to establish a calibration test database, which is a general test database. The calibration test database is used to perform target user testing and establish a first behavior sequence log of the target user; a level matching coefficient establishment module 20 is used to synchronize the first behavior sequence log and the calibration test database to the level evaluation network and establish a level matching coefficient of the test environment; a second behavior sequence log establishment module 30 is used to establish a dynamic test database using the level matching coefficient, perform user testing of the target user based on the dynamic test database, and establish a second behavior sequence log of the target user; a first skill evaluation result establishment module 40 is used to map the second behavior sequence log with the dynamic test database, perform multi-dimensional feature operation evaluation of the mapping result backtracking, and establish a first skill evaluation result, wherein the multi-dimensional features include operation path features, temporal behavior features, semantic analysis features, and strain repair features; a second skill evaluation result establishment module 50 is used to establish a second skill evaluation result based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database; a skill evaluation result fusion module 60 is used to fuse the first skill evaluation result and the second skill evaluation result to output a network security practical skill evaluation result.

[0058] The specific configuration of the first skill evaluation result establishment module 40 will be described in detail below. The first skill evaluation result establishment module 40 further includes: extracting the target user's operation behavior based on the second behavior sequence log and establishing an operation path diagram; using the mapping result to backtrack and call the skill strategy of the dynamic test database; performing node coverage analysis of the operation path diagram and the skill strategy to establish a node matching rate feature; performing the longest common subpath identification of the operation path diagram and the skill strategy to establish an operation sequence similarity feature; performing the meaningless command and skip command identification of the operation path diagram, and generating a strategy anomaly coefficient feature based on the identification result; and establishing an operation path feature evaluation based on the node matching rate feature, the operation sequence similarity feature, and the strategy anomaly coefficient feature.

[0059] The specific configuration of the first skill evaluation result establishment module 40 will be described in detail below. The first skill evaluation result establishment module 40 further includes: constructing an operation interval sequence based on the second behavior sequence log, and performing task segmentation identification of the operation interval sequence through the mapping result; after using the task segmentation identification to segment the operation interval sequence, configuring a time window, counting the operation frequency in each time window, and establishing a density vector; performing spectrum conversion of the density vector through fast Fourier transform, extracting the main rhythm frequency, high-frequency component, and low-frequency component, and establishing an operation rhythm feature; performing pause window recognition of the operation time series and establishing a pause window; performing pause anomaly recognition according to the mapping result corresponding to the pause window and establishing a pause anomaly feature; and establishing a temporal behavior feature evaluation using the operation rhythm feature and the pause anomaly feature.

[0060] The specific configuration of the first skill evaluation result establishment module 40 will be described in detail below. The first skill evaluation result establishment module 40 further includes: performing macro-operation identification on the second behavior sequence log to establish an erroneous operation branch; extracting the pause duration of branch nodes based on the erroneous operation branch; establishing an abnormal pause thinking feature using the pause duration extraction result and the branch length of the erroneous operation branch; and adding the abnormal pause thinking feature to the temporal behavior feature evaluation.

[0061] The specific configuration of the first skill evaluation result establishment module 40 will be described in detail below. The first skill evaluation result establishment module 40 further includes: obtaining the semantic context of the pause window using the mapping results; identifying pause types such as thinking pauses and technical pauses based on the semantic context; and identifying pause anomalies based on the pause type identification results and duration, and establishing pause anomaly features.

[0062] The specific configuration of the first skill evaluation result establishment module 40 will be described in detail below. The first skill evaluation result establishment module 40 further includes: classifying the target user's operations into semantic action labels based on the second behavior sequence log; extracting the semantic action labels and establishing a semantic behavior chain according to the time series; extracting semantic fluency features, semantic integrity features, and semantic deviation features from the semantic behavior chain; and establishing a semantic analysis feature evaluation based on the feature extraction results.

[0063] The specific configuration of the level matching coefficient establishment module 20 will be described in detail below. The level matching coefficient establishment module 20 further includes: calling the test completion unit of the level evaluation network to perform a completion evaluation on the first behavior sequence log and establish a first level impact factor; calling the completion speed unit of the level evaluation network to perform a completion speed evaluation on the first behavior sequence log and establish a second level impact factor, wherein the second level impact factor is configured with an impact coefficient of the first level impact factor; and establishing the level matching coefficient based on the first level impact factor, the second level impact factor, and the impact coefficient.

[0064] The specific configuration of the second skill evaluation result establishment module 50 will be described in detail below. The second skill evaluation result establishment module 50 further includes: establishing a growth index, a stability index, and an adaptability index; utilizing the growth index, stability index, and adaptability index to perform an index evaluation of the first and second behavior sequence logs in the dynamic test database; and establishing the second skill evaluation result based on the index evaluation results.

[0065] The following describes in detail the specific configuration of the skill evaluation result fusion module 60. The skill evaluation result fusion module 60 further includes: constructing a user profile of the target user based on the cybersecurity practical skill evaluation results and generating a weakness identifier; and managing the target user using the user profile and the weakness identifier.

[0066] The network security practical skills assessment system based on artificial intelligence analysis provided by the embodiment of the present invention can execute the network security practical skills assessment method based on artificial intelligence analysis provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0067] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention.

[0068] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A cybersecurity practical skills assessment method based on artificial intelligence analysis, characterized by: The method comprises: Establishing a calibration test database, which is a general test database, using the calibration test database to perform target user testing and establish a first behavior sequence log of the target user; Synchronizing the first behavior sequence log and the calibration test database to a grade evaluation network to establish a grade matching coefficient for the test environment; establishing a dynamic test database using the grade matching coefficient, conducting a user test on a target user based on the dynamic test database, and establishing a second behavior sequence log of the target user; After mapping the second behavior sequence log with the dynamic test database, performing a multi-dimensional feature operation evaluation based on backtracing of the mapping result to establish a first skill evaluation result, wherein the multi-dimensional features include operation path features, temporal behavior features, semantic analysis features, and strain repair features; Establishing a second skill evaluation result based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database; The first skill evaluation result and the second skill evaluation result are integrated to output the network security practical skill evaluation result.

2. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 1, wherein: The multi-dimensional feature operation evaluation of the execution mapping result backtracking to establish the first skill evaluation result includes: extracting the target user's operation behavior according to the second behavior sequence log and establishing an operation path map; Use the mapping results to backtrack and call the skill strategy of the dynamic test database; Performing node coverage analysis of the operation path graph and the skill strategy to establish a node matching rate feature; Identify the longest common subpath between the operation path graph and the skill strategy to establish an operation sequence similarity feature; Execute the recognition of meaningless commands and skip commands in the operation path diagram, and generate a strategy abnormality coefficient feature according to the recognition result; An operation path feature evaluation is established based on the node matching rate feature, the operation sequence similarity feature, and the strategy anomaly coefficient feature.

3. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 1, wherein: The multi-dimensional feature operation evaluation of the backtracking of the execution mapping result to establish the first skill evaluation result further includes: Constructing an operation interval sequence according to the second behavior sequence log, and performing task segmentation identification on the operation interval sequence through the mapping result; After segmenting the operation interval sequence using the task segmentation identifier, configuring a time window, counting the operation frequency within each time window, and establishing a density vector; The density vector is converted into a spectrum through fast Fourier transform to extract the main rhythm frequency, high-frequency components, and low-frequency components, and establish the operation rhythm characteristics; Identify pause windows in the execution time series of operations and establish pause windows; Identify pause anomalies based on the mapping results corresponding to the pause windows and establish pause anomaly features; The operation rhythm characteristics and the pause abnormality characteristics are used to establish a timing behavior characteristic evaluation.

4. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 3, wherein: The establishing of a timing behavior feature evaluation using the operation rhythm feature and the pause abnormality feature includes: Execute the second behavior to identify macro operations of the sequence log and establish an error operation branch; Extracting the pause duration of the branch node according to the erroneous operation branch, and establishing an abnormal pause thinking feature using the pause duration extraction result and the branch length of the erroneous operation branch; The abnormal pause-to-think feature is added to the temporal behavior feature evaluation.

5. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 3, wherein: The process of identifying pause anomalies based on the mapping results corresponding to the pause windows and establishing pause anomaly features includes: Use the mapping results to obtain the semantic context of the pause window; Identify pause types such as thinking pauses and technical pauses based on the semantic context; Complete pause anomaly identification based on pause type identification results and duration, and establish pause anomaly features.

6. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 1, wherein: The multi-dimensional feature operation evaluation of the backtracking of the execution mapping result to establish the first skill evaluation result further includes: classifying the target user's operation into a semantic action label according to the second behavior sequence log; Extract semantic action labels and build semantic behavior chains according to time series; The semantic behavior chain is used to extract semantic fluency features, semantic integrity features, and semantic deviation features, and a semantic analysis feature evaluation is established based on the feature extraction results.

7. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 1, wherein: The step of synchronizing the first behavior sequence log and the calibration test database to a level evaluation network to establish a level matching coefficient of the test environment includes: Calling the test completion unit of the grade evaluation network to perform a completion evaluation on the first behavior sequence log and establish a first grade impact factor; Calling the completion speed unit of the level evaluation network to perform completion speed evaluation on the first behavior sequence log, and establishing a second level impact factor, wherein the second level impact factor is configured with an impact coefficient of the first level impact factor; The level matching coefficient is established according to the first level impact factor, the second level impact factor, and the impact coefficient.

8. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 1, wherein: The establishing of a second skill evaluation result according to the first behavior sequence log, the second behavior sequence log, and the dynamic test database includes: Establish growth indicators, stability indicators, and adaptability indicators; The dynamic test database is called to perform indicator evaluation of the first behavior sequence log and the second behavior sequence log using the growth indicator, stability indicator, and adaptability indicator respectively, and the second skill evaluation result is established according to the indicator evaluation results.

9. The method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to claim 1, wherein: After outputting the cybersecurity practical skills evaluation results, the method includes: Build a user profile of the target user based on the cybersecurity practical skills evaluation results and generate a vulnerability indicator; Target user management is performed using the user portrait and the weak identifier.

10. A cybersecurity practical skills assessment system based on artificial intelligence analysis, characterized by: The system is used to implement the method for evaluating cybersecurity practical skills based on artificial intelligence analysis according to any one of claims 1 to 9, and the system comprises: A first behavior sequence log establishment module is used to establish a calibration test database, which is a universal test database. The calibration test database is used to perform target user testing and establish a first behavior sequence log for the target user. a grade matching coefficient establishing module, configured to synchronize the first behavior sequence log and the calibration test database to a grade evaluation network, and establish a grade matching coefficient for the test environment; A second behavior sequence log establishment module is configured to establish a dynamic test database using the level matching coefficient, perform a user test on a target user based on the dynamic test database, and establish a second behavior sequence log for the target user; A first skill evaluation result establishment module is configured to map the second behavior sequence log with the dynamic test database, perform a multi-dimensional feature operation evaluation based on the backtracking of the mapping result, and establish a first skill evaluation result, wherein the multi-dimensional features include operation path features, temporal behavior features, semantic analysis features, and strain repair features; A second skill evaluation result establishing module, configured to establish a second skill evaluation result based on the first behavior sequence log, the second behavior sequence log, and the dynamic test database; The skill evaluation result fusion module is used to fuse the first skill evaluation result and the second skill evaluation result to output the network security practical skill evaluation result.

Images