A method and system for automatically arranging commands for an image tool
By loading and sorting the command model, generating a running parameter table, and automatically determining the concurrency number, it solves the problem of inefficiency of existing mirror analysis tools, realizes efficient and accurate mirror analysis task processing, and optimizes the utilization of the hardware system.
Patent Information
- Application Number
- CN202411154876.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-22
AI Technical Summary
Existing mirror analysis tools lack the ability to automatically orchestrate multi-command, resulting in inefficiency, easy introduction of artificial errors, and the efficiency of the hardware system has not been maximized.
By loading the command model, sorting and determining the priority of command execution, generating command operation parameter fill tables, determining the number of concurrency based on historical load consumption and server load, realizing automated orchestration and parallel execution of commands, ensuring dependencies and error detection.
It improves command execution efficiency, reduces error incidence, optimizes the utilization rate of the hardware system, and realizes automated mirror analysis task processing.
Smart Images

Figure CN118672645B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an automatic command arrangement method and system for a mirroring tool, belonging to the technical field of information processing and network security. Background Art
[0002] In the field of security research, in-depth analysis of images is a crucial step. Different types of image files, including memory images, hard disk images, and network traffic images, can provide researchers with rich information and evidence. When conducting analysis, researchers usually use a variety of specialized tools to ensure the comprehensiveness and accuracy of the analysis.
[0003] The tool provides capabilities in the form of commands, and each command completes the analysis of a specific indicator of the image. A tool provides several independent commands, and researchers manually combine different commands according to their needs.
[0004] Taking memory image analysis as an example, Volatility is a very popular tool that is widely used in memory forensic analysis. By running the "volatility -f memory.img imageinfo" command, you can determine the operating system version and service pack information of the memory image; by running the "volatility -f memory.img pslist" command, you can list all running processes in the memory.
[0005] The existing image analysis has the following shortcomings: 1. The tool provides the ability to analyze a single indicator of a command, but does not provide the ability to automatically orchestrate multiple commands. The execution of multiple analysis commands needs to be manually orchestrated, which is inefficient; 2. The execution of some commands has dependencies, and manual command execution is prone to human errors, resulting in incorrect execution results; 3. Manual mode command execution relies on expert experience to determine the maximum number of command tasks that can be run in parallel on the server, which is not conducive to maximizing the efficiency of the hardware system. Summary of the invention
[0006] Purpose of the invention: In view of the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide an automated command orchestration method and system for an image tool, so as to improve the efficiency of command orchestration and execution, as well as the efficiency of the hardware system.
[0007] Technical solution: To achieve the above-mentioned purpose, the present invention provides an automatic command arrangement method for an image tool, comprising the following steps:
[0008] Load the command model of the analysis task to be executed, wherein the command model arranges multiple commands, each command is configured with a command name, an execution instruction template, an execution order, an execution result description file, and dependencies between commands; the execution instruction template is provided with framework parameters and / or dynamic parameter placeholders; the execution result description file defines how to parse the output of the command, including fields for extracting the command output content;
[0009] Sort the commands in the command model, determine the command execution priority, and check the execution order based on the dependency relationship;
[0010] Generate a task command running parameter filling table, including framework parameters and dynamic parameters;
[0011] Determine the maximum number of concurrent command executions based on the historical average command load consumption and the theoretical maximum server load.
[0012] Commands are executed concurrently in order from high to low priority. The actual number of concurrent commands started each time is the maximum number of concurrent commands minus the number of commands being run and the number of commands that cannot be started due to dependencies. The execution steps of each command include: obtaining framework parameters and / or dynamic parameters from the command running parameter filling table, filling in the execution instruction template to generate the actual execution instruction; triggering command execution and blocking to wait for return; for commands that fail to execute, all pending commands that depend on this command are marked as failed; for commands that succeed, the output of the command is parsed according to the execution result description file, and if the command output is involved in the command running parameter filling table, the command running parameter filling table is filled.
[0013] Preferably, the framework parameters are automatically filled in the initial stage of command execution, including the output address of the command execution result and the storage address of the image file; the dynamic parameters depend on the execution results of other commands and are filled in during the command execution process; the framework parameters and dynamic parameters are distinguished by placeholders in different forms in the execution instruction template.
[0014] Preferably, the framework parameters are generated according to the task number and command name; the dynamic parameters are generated in a key-value format, the key value is generated according to the task number, command name and parameter name, and the value is filled in after the execution result is dynamically obtained.
[0015] Preferably, the historical statistical average value of command load consumption is obtained by averaging the load consumption of each command in the most recent preset number of successfully executed commands; the command load consumption average value is dynamically updated after each command is successfully executed.
[0016] Preferably, the commands in the command model are verified, including: parsing the commands that define dependent fields one by one, if the execution priority of the dependent command is lower than the current command, prompting that the command priority definition is wrong and stopping the task execution; parsing the commands that contain dynamic parameters in the execution instruction template one by one, if the command does not define dependencies, prompting that the command dependency definition is wrong and stopping the task execution.
[0017] Preferably, the maximum concurrent number of command executions is the theoretical maximum load of the server / the average value of command load consumption.
[0018] Preferably, the actual number of concurrent commands is determined according to the following steps:
[0019] Assume the maximum number of concurrent commands is X, the number of commands in progress is M, then the maximum number of commands that can be started in parallel is XM;
[0020] Remove the dependent commands from the XM commands to be executed, ensuring that the dependent commands are executed first. Suppose P commands are removed;
[0021] Among the XMP commands to be executed, check whether there is a command that depends on the M commands being executed. If there is a dependency, remove the dependent command. Suppose Q commands are removed.
[0022] The maximum number of commands actually executed concurrently is R=XMPQ.
[0023] Preferably, each command in the command model is also configured with a maximum execution time. If the time for waiting for the command execution result to be returned exceeds the maximum execution time, the execution process of the command is closed and the command execution is marked as failed.
[0024] Based on the same inventive concept, the present invention provides an automatic command arrangement system for an image tool, comprising:
[0025] A loading and analysis module is used to load a command model of an analysis task to be executed, wherein the command model arranges multiple commands, each command is configured with a command name, an execution instruction template, an execution order, an execution result description file, and dependencies between commands; the execution instruction template is provided with framework parameters and / or dynamic parameter placeholders; the execution result description file defines how to parse the output of the command, including fields for extracting the output content of the command; and the commands in the command model are sorted, the command execution priority is determined, and the execution order is checked according to the dependencies;
[0026] The parameter filling module is used to generate the command running parameter filling table of the task, including framework parameters and dynamic parameters;
[0027] A concurrent number determination module is used to determine the maximum concurrent number of command executions based on the historical statistical average command load consumption and the theoretical maximum load of the server;
[0028] And a concurrent execution module, which is used to concurrently execute commands in order from high to low priority. The actual concurrency number of each command started is the maximum concurrency number minus the number of commands being run and the number of commands that cannot be started due to dependencies. The execution steps of each command include: obtaining framework parameters and / or dynamic parameters from the command running parameter filling table, filling in the execution instruction template to generate the actual execution instruction; triggering command execution and blocking and waiting for return; for commands that fail to execute, all pending commands that depend on this command are marked as failed to execute; for commands that succeed, the output of the command is parsed according to the execution result description file, and if the command output is involved in the command running parameter filling table, the command running parameter filling table is filled.
[0029] Based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which implements the steps of the automatic command arrangement method of the mirroring tool when executed by a processor.
[0030] Beneficial effects: Compared with the prior art, the present invention has the following advantages:
[0031] 1. The present invention uses an abstract tool command model to allow users to freely arrange instructions based on the model. The arranged instructions can be automatically executed in a predetermined order, thereby achieving dynamic acquisition of parameters and improving the efficiency of command execution.
[0032] 2. The present invention defines the dependency relationship of commands, based on which necessary detection of parameter rationality can be performed, thereby reducing errors caused by human negligence and lowering the error rate of command execution.
[0033] 3. When executing commands, the present invention can automatically count the load consumption during the command execution process, automatically determine the number of parallel commands according to the maximum load that the hardware level can withstand, and improve the utilization rate of the hardware. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is an overall flow chart of an embodiment of the present invention.
[0035] Figure 2 4 is an execution flow chart of each command in an embodiment of the present invention.
[0036] Figure 3 Detailed flowchart of an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments.
[0038] like Figure 1 As shown, an automated command orchestration method for a mirroring tool disclosed in an embodiment of the present invention first loads a pre-arranged command model of an analysis task to be executed, wherein a plurality of commands are orchestrated in the command model, and each command is configured with a command name, an execution instruction template (provided with framework parameters and / or dynamic parameter placeholders), an execution order, an execution result description file, and dependencies between commands; then, the commands in the command model are sorted, the command execution priority is determined, and the execution order is checked according to the dependencies; then, a command running parameter filling table for the task is generated, and the maximum concurrent number of command executions is determined according to the historical statistical average command load consumption and the theoretical maximum load of the server; and then, commands are executed concurrently in order from high to low priority, and the actual concurrent number of each command execution is the maximum concurrent number minus the number of commands being executed and the number of commands that cannot be started due to dependencies. Figure 2 As shown, the execution steps of each command include: obtaining framework parameters and / or dynamic parameters from the command running parameter filling table, filling in the execution instruction template to generate the actual execution instruction; triggering the command execution and blocking and waiting for return; for the command that failed to execute, all the pending commands that depend on this command are marked as failed to execute; for the command that was successfully executed, parsing the output of the command according to the execution result description file, and filling the command running parameter filling table if the command output is involved in the command running parameter filling table.
[0039] The embodiment of the present invention abstracts the command model of the tool. The user can freely arrange instructions based on the model. The arranged instructions are uniformly triggered by the framework and automatically executed in a predetermined order. During the execution process, the execution results can be dynamically extracted and dynamic parameters can be set, thereby improving the efficiency of command execution. Necessary parameter rationality detection can be performed based on the dependency relationship of the commands, thereby reducing errors caused by human negligence and reducing the error rate of command execution. At the same time, when executing commands, the framework automatically counts the load consumption during the command execution process, and automatically determines the number of parallel commands based on the maximum load that the hardware level can withstand, thereby improving the utilization rate of the hardware.
[0040] The command model of the embodiment of the present invention is exemplarily described below.
[0041] Example definition of a command model:
[0042] linux:
[0043] - name: linux.imageinfo
[0044] command: / usr / bin / python3 / home / kali / volatility3 / vol.py -f ${imageInput} -r json imageinfo>${outputDir}
[0045] order: 1
[0046] execTimeout: 100
[0047] schemaPath: / linux / imageinfo_desc.json
[0048] levelTag: System Analysis / System Information / Metadata
[0049] - name: linux.pstree.PsTree
[0050] command: / usr / bin / python3 / home / kali / volatility3 / vol.py -f $[Banner] -r json linux.pstree.PsTree>${outputDir}
[0051] order: 2
[0052] execTimeout: 100
[0053] schemaPath: / linux / pstree_desc.json
[0054] dependency: linux.imageinfo
[0055] levelTag: System Analysis / Process Analysis / Process List
[0056] The above command model includes multiple commands, each of which contains:
[0057] Command name (name), command execution instruction template (command), execution order (order), maximum execution time (execTimeout), execution result description file (the file corresponding to schemaPath), and command dependencies (dependency), etc.
[0058] The execution result description file defines how the framework parses the output of the command. The execution result description file is defined in json format, as shown below: [
[0060] {
[0061] "field": "Banner",
[0062] "desc": "Banner information"
[0063] },
[0064] {
[0065] "field": "Offset",
[0066] "desc": "Memory offset address"
[0067] } ]
[0069] The field field represents the command output content that the framework needs to detect. The framework will automatically parse and extract the Banner and Offset fields according to the result of the banner plug-in execution.
[0070] The command execution instruction template supports multiple parameter formats:
[0071] 1. Framework parameters: expressed in the format of ${param}, this type of parameters can be automatically filled by the framework at the initial stage of command execution. Typical examples include the output address of each command execution result, the storage address of the image file, etc.
[0072] 2. Dynamic parameters: Parameters that depend on the execution results of other commands, expressed in the $[param] format. When defining this parameter, you also need to define the name of the command it depends on. For example, in the command model example above, the output of the linux.imageinfo command contains the Banner field, which is depended on by the linux.pstree.PsTree command.
[0073] The global variable AvgCost set by the system records the average value of historical command load consumption. The framework counts the load consumption C of each command in the most recent N executed commands. The most recent N execution consumption Cost is recorded as [C1, C2, …, C N ], the average consumption AvgCost = (C1 + C2 + ... + C N ) / N. Each command is a separate process, and the load consumption can be counted through the statistical process command, such as top, pidstat, etc. to get the CPU utilization.
[0074] Based on the above command model and the definition of system global variables, a complete exemplary process of an automatic command arrangement method for an image tool according to an embodiment of the present invention is as follows: Figure 3The detailed steps are as follows:
[0075] Step 1: Receive the analysis instruction of the image and generate the analysis task. Each analysis task to be executed corresponds to a unique number and a pre-arranged command model (commands are arranged according to different operating system types). Load the corresponding command model according to the image operating system type to be analyzed.
[0076] Step 2: According to the execution order in the command model, set the execution priority for all commands that need to be executed in the current task. The smaller the order number, the higher the priority. If there is a command with no defined priority, set it to the lowest priority (999). Sort all commands from high to low in order of priority.
[0077] Step 3: Check the rationality of the command, including:
[0078] Parse the commands that define the dependency field one by one. If the execution priority of the dependent command is lower than the current command, it means that the command priority definition is wrong, and the task execution is stopped;
[0079] Parse the commands containing dynamic parameters one by one. If the command does not define dependencies, it means that the command dependency definition is incorrect and the task execution is stopped.
[0080] Step 4: Generate the current task command running parameter filling table, including framework parameters and dynamic parameters. The framework parameters are generated based on the task number + command dimension. For example, when generating the output address of the command, the directory can be defined in the form of " / task number / command name"; for dynamic parameters, generate a dynamic parameter filling table, the key value is generated based on the task number, command name and parameter name, such as task number + command name + parameter name, and the value is filled after the dependent command is executed.
[0081] Step 5: According to the historical load consumption of the plug-in execution, determine the maximum concurrent number of task execution X, X = TotaoCpu / AvgCost. Where TotaoCpu represents the total number of CPUs on the server, which is equal to the theoretical maximum load of the system.
[0082] Step 6: Execute all commands in descending order of priority to determine the actual number of concurrent commands executed each time, including:
[0083] Step 6.1: Assume that the number of commands currently running is M, and the maximum number of commands that can be started in parallel is XM.
[0084] Step 6.2: Remove dependent commands from the XM commands. If command X depends on command Y, remove command X to ensure that Y is executed before X. Assuming that P commands are removed during this process, the maximum number of concurrently running commands is XMP.
[0085] Step 6.3: Check whether any command among the XMP commands is dependent on the M commands being executed. If so, remove the dependent command. Assume that Q commands are removed in this process.
[0086] Step 6.4: Finally, determine that the maximum number of concurrently executed commands is R=XMPQ.
[0087] Step 7: For R concurrently executed commands, each command is executed as follows:
[0088] Step 7.1, obtain the dynamic parameters required by the current command from the dynamic parameter filling table;
[0089] Step 7.2: Use framework parameters and dynamic parameters to fill in the command of the current command and generate the actual execution instructions;
[0090] Step 7.3: The framework triggers the command execution and blocks to wait for the return. The maximum waiting time is the command execution timeout. If the command is not executed after the timeout, the command execution process is closed and the command execution is marked as failed.
[0091] Step 8: For the command that failed to execute, check the list of commands to be executed, mark all the commands that depend on this command as failed to execute, and delete them from the list of commands to be executed. Repeat this process until the list of commands to be executed does not change. Go to step 6.
[0092] Step 9: For commands that are successfully executed, update the load consumption Cost of the command to dynamically update AvgCost.
[0093] Step 10: According to the execution result description file, parse and store the return result of the command. If the dynamic parameter table involves the result of the current command, fill in the corresponding dynamic parameter table at the same time.
[0094] Step 11: Repeat steps 6 to 10 until all commands are executed.
[0095] Based on the same inventive concept, an automatic command arrangement system for an image tool disclosed in an embodiment of the present invention includes: a loading and analysis module, which is used to load a command model of an analysis task to be executed, wherein a plurality of commands are arranged in the command model, and each command is configured with a command name, an execution instruction template, an execution order, an execution result description file, and dependencies between commands; and sorting the commands in the command model, determining the command execution priority, and checking the execution order according to the dependencies; a parameter filling module, which is used to generate a command running parameter filling table for the task, including framework parameters and dynamic parameters; a concurrent number determination module, which is used to determine the maximum number of times to execute the command according to the historical statistics of the command load consumption average value and the theoretical maximum load of the server. A maximum concurrency number; and a concurrent execution module, which is used to concurrently execute commands in order from high to low priority. The actual concurrency number of each command started is the maximum concurrency number minus the number of commands being run and the number of commands that cannot be started due to dependencies. The execution steps of each command include: obtaining framework parameters and / or dynamic parameters from the command running parameter filling table, filling in the execution instruction template to generate the actual execution instruction; triggering command execution and blocking and waiting for return; for commands that fail to execute, all pending commands that depend on this command are marked as failed to execute; for commands that are successfully executed, the output of the command is parsed according to the execution result description file, and if the command output is involved in the command running parameter filling table, the command running parameter filling table is filled.
[0096] Based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which implements the steps of the automatic command arrangement method of the mirroring tool when executed by a processor.
Claims
1. An automated command arrangement method for an image tool, characterized in that: The steps include: Load the command model of the analysis task to be executed, wherein the command model arranges multiple commands, each command is configured with a command name, an execution instruction template, an execution order, an execution result description file, and dependencies between commands; the execution instruction template is provided with framework parameters and / or dynamic parameter placeholders; the execution result description file defines how to parse the output of the command, including fields for extracting the command output content; the framework parameters are automatically filled in the initial stage of command execution, including the output address of the command execution result and the storage address of the image file; the dynamic parameters depend on the execution results of other commands and are filled in during the command execution process; the framework parameters and dynamic parameters are distinguished by placeholders of different forms in the execution instruction template; Sort the commands in the command model, determine the command execution priority, and check the execution order based on the dependency relationship; Generate a task command running parameter filling table, including framework parameters and dynamic parameters; The maximum number of concurrent execution commands is determined based on the historical statistical average value of command load consumption and the theoretical maximum load of the server; the maximum number of concurrent execution commands is the theoretical maximum load of the server / the average value of command load consumption; Execute commands concurrently in descending order of priority. The actual number of concurrent commands started each time is the maximum number of concurrent commands minus the number of commands being run and the number of commands that cannot be started due to dependencies. The actual number of concurrent commands is determined according to the following steps: Let the maximum number of concurrent commands be X, the number of commands being run be M, then the maximum number of commands that can be started in parallel is XM; Remove the dependent commands from the XM commands to be executed, ensuring that the dependent commands are executed first. Suppose P commands are removed; Among the XMP commands to be executed, check whether there is a command that depends on the M commands being executed. If there is a dependency, remove the dependent command. Suppose Q commands are removed. The maximum number of commands actually executed concurrently is R=XMPQ; The execution steps of each command include: obtaining framework parameters and / or dynamic parameters from the command running parameter filling table, filling in the execution instruction template to generate the actual execution instructions; triggering command execution and blocking to wait for return; for commands that failed to execute, all pending commands that depend on this command are marked as failed to execute; for commands that were successfully executed, parsing the output of the command according to the execution result description file, and filling in the command running parameter filling table if the output of the command is involved.
2. The automatic command arrangement method of the mirroring tool according to claim 1, characterized in that: The framework parameters are generated according to the task number and command name; the dynamic parameters are generated in a key-value format, where the key value is generated according to the task number, command name and parameter name, and the value is filled in after the execution result is dynamically obtained.
3. The automatic command arrangement method of the mirroring tool according to claim 1, characterized in that: The historical statistical command load consumption average value is obtained by averaging the load consumption of each command in the most recent preset number of successfully executed commands; the command load consumption average value is dynamically updated after each command is successfully executed.
4. The automatic command arrangement method of the mirroring tool according to claim 1, characterized in that: Verify the commands in the command model, including: parsing the commands that define dependent fields one by one. If the execution priority of the dependent command is lower than the current command, it will prompt that the command priority definition is wrong and stop the task execution; parse the commands that contain dynamic parameters in the execution instruction template one by one. If the command does not define dependencies, it will prompt that the command dependency definition is wrong and stop the task execution.
5. The automatic command arrangement method of the mirroring tool according to claim 1, characterized in that: Each command in the command model is also configured with a maximum execution time. If the time waiting for the command execution result to be returned exceeds the maximum execution time, the command execution process is closed and the command execution is marked as failed.
6. An automated command arrangement system for an image tool, characterized in that: include: The loading and analysis module is used to load the command model of the analysis task to be executed, wherein the command model arranges multiple commands, each command is configured with a command name, an execution instruction template, an execution order, an execution result description file, and dependencies between commands; the execution instruction template is provided with framework parameters and / or dynamic parameter placeholders; the execution result description file defines how to parse the output of the command, including fields for extracting the command output content; And sort the commands in the command model, determine the command execution priority, and check the execution order according to the dependency relationship; the framework parameters are automatically filled in the initial stage of command execution, including the output address of the command execution result and the storage address of the image file; the dynamic parameters depend on the execution results of other commands and are filled in during the command execution process; the framework parameters and dynamic parameters are distinguished by different forms of placeholders in the execution instruction template; The parameter filling module is used to generate the command running parameter filling table of the task, including framework parameters and dynamic parameters; A concurrent number determination module is used to determine the maximum concurrent number of command executions based on the historical statistical average value of command load consumption and the theoretical maximum load of the server; the maximum concurrent number of command executions is the theoretical maximum load of the server / the average value of command load consumption; and a concurrent execution module, which is used to concurrently execute commands in descending order of priority. The actual concurrent number of commands started each time is the maximum concurrent number minus the number of commands being run and the number of commands that cannot be started due to dependencies. The actual concurrent number of commands is determined according to the following steps: assuming that the maximum concurrent number is X, the number of commands being run is M, and the maximum number of commands that can be started in parallel is XM; Remove the dependent commands from the XM commands to be executed, ensuring that the dependent commands are executed first. Suppose P commands are removed; Among the XMP commands to be executed, check whether there is a command that depends on the M commands being executed. If there is a dependency, remove the dependent command. Suppose Q commands are removed. The maximum number of commands actually executed concurrently is R=XMPQ; The execution steps of each command include: obtaining framework parameters and / or dynamic parameters from the command running parameter filling table, filling in the execution instruction template to generate the actual execution instructions; triggering command execution and blocking to wait for return; for commands that failed to execute, all pending commands that depend on this command are marked as failed to execute; for commands that were successfully executed, parsing the output of the command according to the execution result description file, and filling in the command running parameter filling table if the output of the command is involved.
7. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the automated command arrangement method for the mirroring tool according to any one of claims 1 to 5 are implemented.
