AI intrusion detection automatic test method and system based on linux server operating system

Abstract

The invention discloses an AI intrusion detection automatic test method and system based on a linux server operating system, and the method comprises the steps: checking a dependent component operated by an AI intrusion detection program at a target drone end, installing a missing component, carrying out the authority verification, and waiting for a user to adjust the authority; starting a necessary service or port of the target drone end by using the permission meeting the requirement, and closing a firewall of the target drone end; a dependent component of the attack script is checked at the attacker end, a missing component is installed, and then a missing attack tool is checked and installed; acquiring an attack parameter configuration item; according to the attack parameter configuration item, controlling the attacker end to carry out multi-round attacks on the target drone end; and after each attack, collecting and marking a corresponding alarm log, and generating a structured test report based on the attack and alarm data. According to the method, the technical threshold of testing of testers can be reduced, the time consumption for configuring the testing environment is reduced, the testing efficiency is improved, and a foundation is laid for efficient evaluation of AI intrusion detection.

Description

Technical Field

[0001] This invention relates to automated testing technology, specifically to an AI-based automated intrusion detection testing method and system based on a Linux server operating system. Background Technology

[0002] The cybersecurity field increasingly relies on Intrusion Detection Systems (IDS) to defend against various attacks, with AI-driven intrusion detection technology emerging as a new trend due to its efficient threat identification capabilities. However, testing these systems to ensure their reliability and accuracy faces significant bottlenecks. Traditional testing frameworks typically employ a "target-attack machine" dual-machine model: various services (such as FTP servers, web middleware like Apache or Nginx, and database services like MySQL) are manually deployed on the target test machine (target machine), followed by a customized attack (such as SQL injection or buffer overflow) launched on the attack machine. Finally, the IDS is verified to ensure timely detection of the attack and generation of specified alerts. This model was effective in the early stages, but with the increasing complexity of network attacks and the integration of AI models, its inefficiency and manpower-intensive nature have become amplified. More specifically, AI intrusion detection utilizes deep learning or machine learning models to perform real-time inference on network traffic, automatically identifying attack patterns and triggering alerts. While this improves the intelligence level of detection, it also introduces new requirements for verifying model generalization and stability.

[0003] While conventional intrusion detection testing is a fundamental means of verifying system functionality, its core pain points stem from its high degree of manual intervention and reliance on tools, leading to low testing efficiency and significant resource waste. These pain points can be summarized into two interrelated dimensions: the cumbersome nature of environment deployment and the skill threshold for using attack tools.

[0004] Compared to conventional IDS, AI intrusion detection uses machine learning models to infer network traffic in real time, enabling the identification of complex attack patterns (such as zero-day attacks). However, this also brings unique testing challenges—the need for high-frequency verification of model accuracy, which exacerbates the time consumption issue. The core issue lies in the uncertainty of AI models: model inference results are affected by training data, parameter initialization, and input variations, potentially leading to false positives (benign traffic misidentified as attacks) or false negatives (attacks not identified). Therefore, testing must involve repeated attacks to statistically determine confidence (e.g., requiring each attack to be executed dozens of times to calculate accuracy). For example, when testing the AI ​​model's ability to detect DDoS attacks, it's necessary to simulate attack traffic multiple times (e.g., 50-100 times) to observe whether the model consistently outputs correct alerts; if the results fluctuate significantly, model parameters or data augmentation strategies need to be adjusted. This process significantly extends the testing cycle: according to an IBM Security case (2024), a single AI model verification takes 2-3 times longer than traditional testing (benchmark testing: traditional attack testing requires 1-2 hours / type, while AI verification requires 3-6 hours / type). Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide an automated testing method and system for AI intrusion detection based on the Linux server operating system, which addresses the above-mentioned problems in the prior art. Based on the hardware and software characteristics of the Linux server operating system platform, it realizes a fully automated testing process of environment construction, tool pool integration, intelligent attack injection, and generation of attack result reports. This can reduce the technical threshold for testers, reduce the time spent configuring the test environment, improve testing efficiency, and lay the foundation for efficient evaluation of AI intrusion detection.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An automated testing method for AI-based intrusion detection based on a Linux server operating system includes the following steps: On the target machine, check the dependent components of the AI ​​intrusion detection program and install the missing components. Then, verify permissions and wait for the user to adjust permissions. Use the required permissions to open the necessary services or ports on the target machine and turn off the firewall on the target machine. On the attacking machine, check the dependencies of the attack script and install any missing components, then check and install any missing attack tools. Obtain attack parameter configuration items; Based on the attack parameter configuration items, the attacking machine is controlled to launch multiple rounds of attacks against the target machine; After each attack, the corresponding alarm logs are collected and marked, and a structured test report is generated based on the attack and alarm data.

[0007] Furthermore, when checking the dependent components of the AI ​​intrusion detection program on the target machine and installing the missing components, and when checking the dependent components of the attack script on the attacker machine and installing the missing components, both include: using dynamic import to check whether the AI ​​intrusion detection or attack script's dependent components are already included in the current environment; if there are missing components, then calling the pipinstall command to install the missing components.

[0008] Furthermore, when checking and installing missing attack tools, the process specifically involves checking whether the corresponding attack tools exist and are executable based on predefined attack items, and then installing the missing attack tools.

[0009] Furthermore, when controlling the attacking machine to launch multiple rounds of attacks against the target machine according to the aforementioned attack parameter configuration items, the following are included: Generate the attack task instructions corresponding to the attack parameter configuration items and add them to the attack queue; Establish a session between the attacking machine and the target machine, the session including an SSH session; Clear the alarm logs on the target machine and obtain the current attack task instructions in the attack queue. Call the corresponding attack tool on the attack machine and launch a specified number of attacks on the target machine according to the attack interval in the corresponding attack parameter configuration item. If the attack times out or the interval ends, terminate the process of the attack tool and reclaim the terminated process. During the execution of the current attack task command, the attacking machine sends a heartbeat detection signal to the target machine to keep the SSH session alive, and after execution, it retrieves alarm logs from the target machine via the SSH connection. If there are unexecuted attack task instructions in the attack queue, proceed to the steps of clearing the alarm log on the target machine and obtaining the current attack task instructions in the attack queue, until there are no unexecuted attack task instructions in the attack queue.

[0010] Furthermore, the session also includes a web session, and after opening the necessary services or ports on the target machine with the required permissions, it also includes: initializing the database of the web target field.

[0011] Furthermore, the attack parameter configuration items include attack type, attack method, timeout time and attack interval, and the test report includes attack type, attack method, attack type, attack method, number of tests, number of alarms triggered by a specified number of attacks and number of alarms that meet expectations.

[0012] Furthermore, the specified number of attacks uses a confidence level statistical lower limit, the attack interval is a critical value to prevent resource overload, and the timeout time is the maximum tolerance time to prevent DoS deadlock.

[0013] The present invention also proposes an AI intrusion detection automatic testing system, including a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the AI ​​intrusion detection automatic testing method based on the Linux server operating system.

[0014] The present invention also proposes a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the AI ​​intrusion detection automatic testing method based on a Linux server operating system.

[0015] The present invention also proposes a computer program product, including a computer program, which, when executed by a processor, implements the steps of the AI ​​intrusion detection automatic testing method based on the Linux server operating system.

[0016] Compared with the prior art, the advantages of the present invention are as follows: This invention achieves fully automated testing, including automated deployment of the target machine environment, automated deployment of the attacker machine environment, automatic testing based on attack parameter configuration items, and generation of test reports. Testers only need to fill in the configuration items and define the attack method, frequency, and timeout parameters to complete the incremental function test configuration. This solves the problem of test failure caused by errors in manual configuration, lowers the technical threshold for testers, and improves testing efficiency.

[0017] This invention satisfies the minimum sample size requirement for AI model confidence statistics through multiple rounds of attacks, avoids target machine resource overload by setting attack intervals, and prevents system freeze caused by DoS attacks by setting timeouts, thus achieving stable testing.

[0018] This invention keeps the SSH session alive during an attack by using heartbeat detection, solving the log loss problem caused by disconnection and reconnection in traditional testing, and ensuring that logs generated in each round of attack are effectively collected. Attached Figure Description

[0019] Figure 1 This is a simplified flowchart of the method according to an embodiment of the present invention.

[0020] Figure 2 This is a detailed flowchart of the method according to an embodiment of the present invention.

[0021] Figure 3 Key code examples for automatically detecting and installing the necessary dependencies for the program to run.

[0022] Figure 4 Key code examples for automatically detecting and enabling necessary services and ports.

[0023] Figure 5 A key code example for initializing the database for a web target environment.

[0024] Figure 6 Key code examples for automatically detecting and disabling the target machine's firewall.

[0025] Figure 7 Key code examples for automatically detecting and installing specified attack tools.

[0026] Figure 8 A diagram illustrating the configuration for attack items.

[0027] Figure 9 This is a key code example for creating a session with the target machine.

[0028] Figure 10 This is a key code example for automatically launching an attack.

[0029] Figure 11 Key code examples for forcibly reclaiming a process.

[0030] Figure 12 This is a schematic diagram of the structured test results. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0032] Before introducing specific embodiments of the present invention, relevant concepts or terms will be explained.

[0033] Intrusion Detection: Also known as IDS, it is a network security device or application that monitors network transmissions in real time and issues alerts or takes proactive measures when suspicious transmissions are detected.

[0034] Target machine: The machine being attacked, usually the object being tested, is a device that simulates a real network environment.

[0035] Attacker machine: The device used to launch an attack; it is the device used to simulate a hacker attack.

[0036] To address the challenges of labor-intensive processes, high barriers to entry, and the repetitiveness and time consumption of model accuracy verification, this embodiment proposes an automated testing method for AI intrusion detection based on a Linux server operating system. This method utilizes automated scripts to achieve environment construction, tool pool integration, and intelligent attack injection, enabling standardized, repeatable, and high-frequency testing. This supports rapid iteration and reliable deployment of AI models and ensures compatibility with Kylin OS through a domestic adaptation layer.

[0037] like Figure 1As shown, the method includes the following steps: S1) Automated deployment of target machine environment: Automatically configure the target machine environment according to the AI ​​intrusion detection requirements. Specifically, check the dependent components of the AI ​​intrusion detection program on the target machine and install the missing components. Then, verify the permissions and wait for the user to adjust the permissions. Then, use the required permissions to open the necessary services or ports on the target machine and close the firewall on the target machine. S2) Automated deployment of attack machine environment: Deploy attack tools based on detection capabilities. Specifically, check the dependent components of the attack script on the attack machine and install the missing components, and then check and install the missing attack tools. S3) Attack Item Configuration: Obtain attack parameter configuration items, including attack method, attack frequency, single attack duration and attack interval time; S4) Automated attack: Parse the configuration parameters from the previous step and automatically launch an attack according to the strategy. Specifically, based on the attack parameter configuration items, control the attacking machine to launch multiple rounds of attacks on the target machine. S5) Alarm Collection: Collect and mark the corresponding alarm logs after each attack; S6) Test Report: A structured test report is generated based on attack and alarm data.

[0038] The following is in conjunction with the appendix Figure 2 Each step is explained in detail.

[0039] This embodiment achieves environment initialization checks and configurations on the target machine during program execution through step S1, such as... Figure 2 As shown, the core process of step S1 includes: Step S101: Check the dependencies of the AI ​​intrusion detection program and install any missing components. Specifically, use dynamic import (__import__) to check if the necessary modules for the AI ​​intrusion detection program (such as requests, paramiko) are already included in the current environment. If any components are missing, call the pip install command to install them. Key code examples are shown below. Figure 3 As shown.

[0040] Step S102: Script permission verification. If the user does not have root privileges, the program will exit and prompt the user to execute the script with higher privileges.

[0041] Step S103: Use root privileges to check if the service or port is open or listening. If not, install the service and open the port. Key code examples are shown below. Figure 4 As shown.

[0042] Step S104: Initialize the web testbed database. Use the request module to initialize the web testbed database. Key code examples are as follows: Figure 5 As shown.

[0043] Step S105: Disable the firewall on the target machine. Key code examples are as follows: Figure 6 As shown.

[0044] Through the above steps, step S1 automatically configures the service port and firewall policy through dynamic dependency detection (__import__) and permission verification (root detection), achieving the following functions: (1) Automatically detect and install the necessary dependencies for the program to run; (2) Automatically detect and enable necessary services and ports; (3) Automatically detect and disable the target machine's firewall.

[0045] This embodiment completes the integration and verification of the attack toolchain through step S2, such as... Figure 2 As shown, step S2 in this embodiment includes: Step S201: Reuse the S101 mechanism to install the attack script dependencies. Specifically, when checking the attack script's dependent components and installing missing components on the attack machine, dynamic import is also used to check whether the attack script's dependent components are already present in the current environment. If missing components exist, the pip install command is called to install the missing components.

[0046] Step S202: Check and install missing attack tools. Specifically, based on predefined attack items (SQL injection / DDoS, etc.), check if the corresponding attack tools (nmap / amap, etc.) exist and are executable. If missing, automatically download, unzip, check, and install the missing attack tools. Key code examples are shown below. Figure 7 As shown.

[0047] Through the above steps, step S2, based on predefined attack items (SQL injection / DDoS, etc.), automatically verifies and installs missing tools (nmap / amap, etc.), achieving the following functionality: (1) Automatically detect and install the necessary dependencies for the program to run; (2) Automatically detect and integrate / install the specified attack tools.

[0048] In step S3 of this embodiment, to prevent DoS attacks from causing system crashes, a timeout period is added to the attack parameter configuration item. The timeout threshold is set to ≤60 seconds, based on the maximum tolerance time for DoS crashes. Furthermore, the attack parameter configuration item also includes detection category, attack sub-category, attack method, and attack interval, such as... Figure 8 As shown. The attack interval is set to 17±2 seconds, based on the critical value to prevent resource overload.

[0049] In this embodiment, step S4 initiates a controlled attack based on the configuration of step S3, such as... Figure 2 As shown, it includes the following steps: Step S401: Configuration item parsing, generating attack task instructions corresponding to the attack parameter configuration items in step S3 and adding them to the attack queue; Step S402: Establish a session between the attacking machine and the target machine. This session includes an SSH session and a web session. Specifically, it involves creating session information for the target machine's web service (using `requests.Session()` to maintain the cookie). A key code example is shown below. Figure 9 As shown in (a), and the establishment of an SSH persistent connection (paramiko.SSHClient), key code examples are as follows. Figure 9 As shown in (b); Step S403: Before the attack, clear the alarm log on the target machine (> / var / log / xxx.log), and obtain the current attack task instruction in the attack queue. Call the corresponding attack tool on the attacking machine and launch a specified number of attacks on the target machine according to the attack interval (17±2 seconds) in the corresponding attack parameter configuration. In this embodiment, the specified number of attacks is set to 10 times according to the lower limit of confidence statistics. Therefore, each attack in the multi-round attack scheduling is executed 10 times (to meet the confidence statistics requirements of the AI ​​model). The default attack interval is 17 seconds. Key code examples are as follows. Figure 10 As shown; Step S404: Forced process reclamation. If the attack times out or the interval ends, a system command is invoked to terminate the remaining process, thereby terminating the process of the attack tool and reclaiming the terminated process. A key code example is shown below. Figure 11 As shown; Step S405: Keep the SSH session alive. During the execution of the current attack task command, control the attacking machine to send a heartbeat detection signal to the target machine to keep the SSH session alive (send echo "alive" before each log collection). Step S406: Targeted log capture. After the current attack task command is executed, the alarm logs are extracted from the target machine by executing the cat command via SSH connection. Step S407: If there are unexecuted attack task instructions in the attack queue, jump to step S403 to clear the alarm log on the target machine and obtain the current attack task instructions in the attack queue, until there are no unexecuted attack task instructions in the attack queue.

[0050] Through the above steps, step S4 implements a multi-round attack scheduling engine: launching attacks according to the configured timeout, attack interval (default 17 seconds), and fixed number of attacks (≥10 times); it also implements alarm quantitative analysis: collecting logs in real time through the SSH long connection keep-alive mechanism (heartbeat detection) and calculating accuracy / missed detection rate.

[0051] This embodiment achieves quantifiable monitoring of attack effects through step S5, specifically by collecting alarms generated by each attack behavior and counting the number of alarms triggered by 10 attack behaviors and the number of alarms that meet expectations.

[0052] This embodiment outputs structured test results through step S6, such as... Figure 12 As shown, the test report includes the attack type, attack method, attack type, attack method, number of tests, number of alarms triggered by a specified number of attacks, and the number of alarms that meet expectations.

[0053] Furthermore, this embodiment also proposes an AI intrusion detection automatic testing system, including a processor and a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program is executed by the processor to implement the steps of the AI ​​intrusion detection automatic testing method based on the Linux server operating system described in this embodiment.

[0054] Furthermore, this embodiment also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the AI ​​intrusion detection automatic testing method based on the Linux server operating system described in this embodiment.

[0055] Furthermore, this embodiment also proposes a computer program product, including a computer program, which, when executed by a processor, implements the steps of the AI ​​intrusion detection automatic testing method based on the Linux server operating system described in this embodiment.

[0056] In summary, this invention proposes an automated testing method and system for AI intrusion detection based on a Linux server operating system. It achieves fully automated testing through automated deployment of the target machine environment, automated deployment of the attacker machine environment, attack item configuration, automated attack execution, alarm collection, and test report generation. Compared with existing technologies, this invention has the following advantages: (1) This invention achieves a paradigm shift in AI intrusion detection testing through deep adaptation to domestic operating systems and full-link automation architecture.

[0057] (2) This invention achieves a leap in efficiency of unmanned operation throughout the entire process (solving the pain point of low efficiency in manual testing). The overall process covers the automated testing process of target machine environment construction, attack machine environment construction, automated testing, and generation of test reports. Compared with manual testing, the overall testing efficiency is improved by no less than 90%.

[0058] (3) This invention realizes an scalable architecture ecosystem. The applicability of the current tool has fully covered the AI ​​intrusion detection of the Kylin server operating system. When adding AI intrusion detection capabilities, you only need to fill in the configuration items, define the attack method, frequency and timeout parameters, and complete the test configuration of the incremental function.

[0059] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0060] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An AI-based automated intrusion detection testing method based on a Linux server operating system, characterized in that, Includes the following steps: On the target machine, check the dependent components of the AI ​​intrusion detection program and install the missing components. Then, verify permissions and wait for the user to adjust permissions. Use the required permissions to open the necessary services or ports on the target machine and turn off the firewall on the target machine. On the attacking machine, check the dependencies of the attack script and install any missing components, then check and install any missing attack tools. Obtain attack parameter configuration items; Based on the attack parameter configuration items, the attacking machine is controlled to launch multiple rounds of attacks against the target machine; After each attack, the corresponding alarm logs are collected and marked, and a structured test report is generated based on the attack and alarm data.

2. The AI-based automated intrusion detection testing method based on a Linux server operating system according to claim 1, characterized in that, When checking the dependent components of the AI ​​intrusion detection program on the target machine and installing missing components, and when checking the dependent components of the attack script on the attacker machine and installing missing components, both include: using dynamic import to check whether the AI ​​intrusion detection or attack script's dependent components are already included in the current environment; if missing components exist, calling the pip install command to install the missing components.

3. The AI-based automatic intrusion detection testing method based on a Linux server operating system according to claim 1, characterized in that, When checking and installing missing attack tools, the process specifically involves checking if the corresponding attack tools exist and are executable based on predefined attack items, and then installing the missing attack tools.

4. The AI-based automatic intrusion detection testing method based on a Linux server operating system according to claim 1, characterized in that, When controlling the attacking machine to launch multiple rounds of attacks against the target machine according to the aforementioned attack parameter configuration items, the following are included: Generate the attack task instructions corresponding to the attack parameter configuration items and add them to the attack queue; Establish a session between the attacking machine and the target machine, the session including an SSH session; Clear the alarm logs on the target machine and obtain the current attack task instructions in the attack queue. Call the corresponding attack tool on the attack machine and launch a specified number of attacks on the target machine according to the attack interval in the corresponding attack parameter configuration item. If the attack times out or the interval ends, terminate the process of the attack tool and reclaim the terminated process. During the execution of the current attack task command, the attacking machine sends a heartbeat detection signal to the target machine to keep the SSH session alive, and after execution, it retrieves alarm logs from the target machine via the SSH connection. If there are unexecuted attack task instructions in the attack queue, proceed to the steps of clearing the alarm log on the target machine and obtaining the current attack task instructions in the attack queue, until there are no unexecuted attack task instructions in the attack queue.

5. The AI-based intrusion detection automatic testing method based on a Linux server operating system according to claim 4, characterized in that, The session also includes a web session, and after enabling the necessary services or ports on the target machine with the required permissions, it also includes initializing the database of the web target environment.

6. The AI-based automatic intrusion detection testing method based on a Linux server operating system according to claim 4, characterized in that, The attack parameter configuration items include attack type, attack method, timeout time and attack interval. The test report includes attack type, attack method, attack type, attack method, number of tests, number of alarms triggered by a specified number of attacks, and number of alarms that meet expectations.

7. The AI-based automatic intrusion detection testing method based on a Linux server operating system according to claim 6, characterized in that, The specified number of attacks uses the lower limit of confidence level statistics, the attack interval is the critical value to prevent resource overload, and the timeout time is the maximum tolerance time to prevent DoS deadlock.

8. An AI-based automated intrusion detection testing system, characterized in that, The device includes a processor and a computer-readable storage medium storing a computer program, which is executed by the processor to implement the steps of the AI ​​intrusion detection automatic testing method based on a Linux server operating system as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the AI ​​intrusion detection automatic testing method based on a Linux server operating system as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the steps of the AI ​​intrusion detection automatic testing method based on a Linux server operating system as described in any one of claims 1 to 7.

Images