AI-based secure IT operations driven by expert population knowledge

Abstract

A computing device (1200) for secure command execution is described, the computing device (1200) comprising one or more processors (1201) and a memory (1202) that stores, in a non-transitory form, data defining program code executable by the one or more processors (1201), where the program code is executable by the one or more processors (1201), and where the program code is not executable by the one or more processors (1201). Causing the computing device (1200) to: obtain an input command (831) from an operator (701); evaluating the input command (831) to determine whether the input command (831) is secure through a group-based knowledge tag (832); the input command (831) is executed in accordance with the input command (831) being secure. In this way, unsafe command execution may be prevented using knowledge of the expert reviewer (821).

Description

Technical Field

[0001] The present invention relates to an apparatus and method for executing security commands based on group-based knowledge tags. Background Art

[0002] Growing security threats are driving organizations to develop effective security solutions to protect digital applications, data, and resources. Consequently, several solutions exist to prevent malicious attacks. In large-scale IT systems, operations and maintenance (O&M) personnel can access production systems through remote terminals to perform operations. Incorrect or malicious operations can lead to significant system failures and data loss, necessitating solutions to prevent potentially risky operations. Figure 1 An example of such an operation classification is shown in . Figure 1 It is a first table 100 of the prior art classification operation. Table 100 includes a column for commands 101 and a corresponding column for risk classification 102. In this case, risk classification 102 includes a "risky" or "safe" classification.

[0003] A common approach to command classification is for experts to generate if-then-else rules to detect and prevent risky operations. However, this approach may not scale well to large computing environments. This is because large computing environments are often highly dynamic, and static rules generated by humans must be constantly maintained and updated to reflect the dynamic nature of the environment.

[0004] Another approach to command classification uses machine learning techniques to automatically generate generalizable rules to prevent risky actions. However, this approach requires predefined examples of actions and their associated labels as input. Generating labeled examples for this task is often challenging, as the person labeling the actions must be intimately familiar with the computing environment and the applications within it. Therefore, generating high-level, accurate labels requires a high level of expertise. Furthermore, the required expertise is often distributed among the experts operating various applications within the computing environment. Therefore, labeling is a distributed expert task that requires organization and synchronization.

[0005] Turning to existing technology:

[0006] Figure 2A The first exemplary security command execution device 200 of the prior art is schematically shown. Specifically, Figure 2A The Google tool, Security Agent, is shown. Security Agent is a framework that enables authorized personnel to access or modify the state of a physical server, virtual machine, or specific application. Security Agent can be used to review, approve, and run risky commands without establishing a direct SSH connection to the end system. Figure 2AA security agent, called a tool agent, authenticates commands and forwards them to one or more target machines. All commands used by production systems are subject to rules in a so-called strict schema. This schema allows for parameter restrictions, user restrictions, and mandatory reviewer confirmation. Figure 2B An example of this pattern is shown in . Figure 2B yes Figure 2A An exemplary policy definition 201 of a first exemplary security command execution device 200 in the prior art.

[0007] Figure 2A The Google tool agents shown in [1] can rely on predefined rules, which requires a language to specify the patterns of these rules. The language may not be as simple as regular expressions (Linux program arguments are order-independent, so the language cannot be regular or context-independent). Different tools use different styles of arguments (POSIX, GNU long options, Java style, old). From a practical point of view, creating a universal pattern may be difficult. An expert may be needed to generalize command instances into rules. Ideally, the expert should have domain knowledge of the commands and be familiar with the rule language. The expert can be involved every time a new tool is added to the system or when an existing command is executed with parameters that were not previously available.

[0008] Figure 3 A second exemplary flow chart 300 of a security command execution device of the prior art is schematically shown. Specifically, Figure 3 A technique for risk validation using a protected shell is shown. The device is able to capture commands before they are executed and validate the commands based on a set of rules. The solution is able to validate commands by considering the command's target (e.g., working directory). Each command (and its parameter set) is assigned to a category (GO, NOT SURE, NO GO). Commands in the GO category are accepted. Commands in the NOT SURE category are further checked for compliance with policy. Commands in the NO GO category are rejected. Commands are assigned to categories, and policy definition for NOT SURE commands is completed on a per-scenario basis.

[0009] Figure 4 This is the second table 400 of the prior art classification operation. Figure 4The security of the shell command is determined by the shell command security level table shown in . As a result of the check, if the security value of the shell command is determined to be "safe", the shell command is executed, and if the security value of the shell command is checked to be "unsafe", the user is required to enter the password level and password (open key value) according to the relevant level. The additional hacker check part checks whether the shell command is in the hacker command list, and checks whether the file that the user wants to access using the shell command is a system security file. The execution of unauthorized shell commands (or processes) entered through a shell with administrator authorization can be detected, and the execution can be restricted based on whether the password is correct. Similar prior art systems also propose to activate a command protection shell as a login shell to limit executable commands. The command requested from the user terminal is checked according to a configuration file, which defines the available commands and can be described in XML format. If the input command is not available, the user terminal will be notified.

[0010] Figure 3 and Figure 4 The prior art shown in is based on predefined policies which may limit the options for building and maintaining policies. Figure 3 and Figure 4 The prior art techniques shown in are also implemented on an OS shell running on the host machine, which may require changes on the target machine.

[0011] Figure 5 A third exemplary security command execution device 500 in the prior art is schematically shown. Figure 5 The present invention illustrates how to provide a user with access rights to execute commands on network-accessible computing resources. A service provider environment can provide a user with a command execution component having a shell-like interface. A shell aggregator executes on one or more computing systems. A request from a user indicates that a command is to be executed by each of a plurality of computing nodes provided by the network-accessible service. Operation of the user command execution interface component can be initiated, for example, by a user of a client device, who provides one or more commands to a shell aggregator module of the user command execution interface component to be executed on information about one or more computing-related resources for executing the one or more commands. The shell aggregator module determines whether the user is authorized to execute the one or more commands on the one or more computing-related resources, for example, by querying a permission proxy module of the user command execution interface component to determine whether the user is authorized to perform such actions. A permission repository maintains permissions for one or more users, wherein each user's permissions identify which commands the user is authorized (or not authorized) to execute and on which computing nodes.

[0012] Figure 6 A fourth exemplary security command execution device 600 in the prior art is schematically shown. Figure 6 The architecture of a convolutional neural network (CNN) model for PowerShell command classification is shown. The classification model combines traditional NLP-based classifiers with deep CNNs to learn to classify PowerShell commands as malicious or non-malicious. Traditional NLP models use character-level 3-grams and a bag-of-words model, while the CNN model uses one-hot encoding at the character level, with a maximum command length of 1024 characters.

[0013] Tokenization and one-hot encoding may not account for relationships between tokens. Furthermore, CNNs may not consider the sequential nature of commands. Training may also require a large number of labeled commands. Outsourcing labeling to services like Mechanic Turk (which lacks domain expertise) or requiring expert labeling (which is prohibitively expensive) is also not feasible.

[0014] To prevent large-scale IT system failures caused by operators executing risky commands on the command line interface, high-risk commands should be automatically flagged. UNIX security features may not be sufficient to run large-scale cloud environments because:

[0015] (i) They may not provide fine-grained control over arguments, options, and flags. The effect of a command on the target system depends on its arguments and flags. For example, fdisk -l is safe because it only lists storage devices, but any other call to fdisk is potentially risky.

[0016] (ii) They may need to access and modify the end host on which the command is executed. Therefore, hosts in a cloud environment may need to be micro-managed to perform policy updates (who can execute what). This may not be feasible in a cloud environment, as a large number of servers (e.g., millions) may need to be updated at a time.

[0017] (iii) They are difficult to change based on the type of session. For example, while a scheduled maintenance session typically has strict permissions, a session used to handle critical incidents may have elevated permissions, enabling the user to execute all commands.

[0018] One of the challenges in solving this task is that, although shell commands may have different components such as arguments, options, and flags, there is no unified syntax to specify flags and arguments. For example, some tools do not use a hyphen as a flag (tarxz), some use a single hyphen (rm -rf), and some use double or multiple hyphens (ls --human-readable).

[0019] Another challenge is that while the order of subcommands, flags, and arguments is not important in some cases, it can be crucial in other cases. For example, the commands rm -rf {PATH}, rm -fr {PATH}, and rm -rf {PATH} are all the same command, but cp {FILE_1} {FILE_2} and cp {FILE_2} {FILE_1} have very different effects. Furthermore, arguments, options, and flags have meaning in the context of the executed command. For example, considering the rm and tar applications, the flag -f means "force" (potentially risky) and archive file name (no risk), respectively.

[0020] For another example, consider two commands: (1) xargs kill -9 12345; and (2) echo "kill -9 12345". The former is a potentially dangerous command that can terminate important running processes, while the latter is safe because it just prints some text to the terminal.

[0021] Another important challenge is the dimensionality of the problem to be solved and the generalizability of the solution. Existing rule-based approaches may only capture what is known to the operator, with the operator explicitly configuring the if-then-else rules based on their prior knowledge of the system. However, a drawback of this approach is that not all commands and their combinations of arguments, options, and flags are known to the operator. Therefore, by only seeing a limited number of command cases and their corresponding risk levels, the solution should be able to generalize to unseen cases.

[0022] Labeling commands (e.g., bash commands) can be a non-trivial task and may require expertise across many domains and applications. Therefore, obtaining accurate labels for training AI models can be challenging. Known options include rule-based approaches or Linux security features that work with review systems. This can be costly because many rules must be generated by experts. Furthermore, executed commands and rules may need to be reviewed regularly by a team of experts. Rule-based approaches may not be as accurate as they lack generalizability to new commands. Therefore, in some cases, executing risky commands may be unavoidable. Consequently, cloud providers may incur additional remediation costs. The global security operations center (SOC) market was valued at $5.39 billion in 2021 and is expected to grow at a CAGR of 10.2% over the forecast period. SOCs can be responsible for preventing, detecting, investigating, and responding to cyber threats, and if the solution is sold locally, can inspect code.

[0023] It would be desirable to develop devices and methods that overcome the above-mentioned problems. Summary of the Invention

[0024] According to a first aspect, a computing device for secure command execution is provided, the computing device comprising one or more processors and a memory storing data defining program code executable by the one or more processors in a non-transitory form, wherein the program code is executable by the one or more processors such that the computing device is configured to: obtain an input command from an operator; evaluate the input command to determine whether the input command is secure using a population-based knowledge label; and execute the input command based on the input command being secure. In this way, the knowledge of an expert reviewer can be used to prevent unsafe command execution.

[0025] In some implementations, a computing device can be configured to evaluate the input command to determine whether the input command is safe using a classifier model, wherein one or more population-based knowledge labels are incorporated into the classifier model. In this way, the knowledge base in the classifier model can be used to predict whether the input command is safe.

[0026] In some implementations, the computing device may be configured to determine the population-based knowledge label by: providing the input command to an expert reviewer; having the expert reviewer evaluate the input command; and determining the population-based knowledge label based on the expert reviewer's evaluation. In this manner, the expert reviewer's expertise may be used to directly determine whether the input command is safe.

[0027] In some implementations, the computing device can be configured to update the classifier model using the population-based knowledge labels determined from the expert reviewer evaluations, so that each time an expert reviewer is asked to evaluate a command, the output can be used in the future by updating the classifier model.

[0028] In some implementations, the computing device can be configured to store the population-based knowledge labels determined from the expert reviewer evaluations, so that if the same or a very similar command is entered in the future, the precise labels may already be stored and used to determine the safety of the entered command.

[0029] In some implementations, the computing device can be configured to determine the population-based knowledge label based solely on population-based knowledge labels for which the classifier model cannot identify the input command within a certain level. This way, if expert knowledge is not required, the device is not slowed down by waiting for expert knowledge.

[0030] In some implementations, the computing device may be configured to select the expert reviewer based on the expert reviewer's knowledge of the input command, so that the best expert reviewer for reviewing a particular type of input command may be selected.

[0031] In some implementations, the computing device may be configured to determine the population-based knowledge label by: providing the input command to multiple expert reviewers; each expert reviewer evaluating the input command; aggregating the corresponding expert reviewer evaluations; and determining the population-based knowledge label based on the aggregated expert reviewer evaluations. In this way, knowledge from more than one expert reviewer may be aggregated to assess whether the input command is safe.

[0032] In some implementations, the computing device can be configured to aggregate the evaluations of the respective expert reviewers by weighting each expert reviewer's evaluations, the weighting being dependent on the expert reviewer's knowledge of the input command. In this way, aggregating knowledge from one or more expert reviewers can account for differing levels of expert reviewer knowledge of a particular type of input command.

[0033] In some implementations, the computing device may be configured to: divide the input command into multiple atomic operations and corresponding parameters; evaluate each of the atomic operations and corresponding parameters to determine whether the atomic operations and corresponding parameters are safe using a population-based knowledge label; and execute the input command based on whether each of the atomic operations and corresponding parameters is safe. In this way, the input command can be broken down into smaller commands, which makes it easier and more efficient to analyze their security.

[0034] In some implementations, the computing device may be configured to divide the input command into multiple atomic operations and corresponding parameters based on the mutual relationships between the pairs of atomic operations and corresponding parameters. In this way, the mutual relationships between the parts of the input command can be used to check whether the combination of the parts is generally safe.

[0035] In some implementations, the computing device may be configured to evaluate each of the atomic operations and corresponding parameters to determine whether the atomic operations and corresponding parameters are safe based on the context of the atomic operations and corresponding parameters. In this way, the safety of a command may be determined using the context of each portion of the command that may indicate safety.

[0036] According to a second aspect, there is provided a cloud infrastructure comprising any of the above-stated computing apparatuses, so that the apparatus can be implemented across a wide range of computing devices.

[0037] In some implementations, the cloud infrastructure may include an operator, an agent, and a target host. The agent may be configured to receive input commands from the operator and evaluate the input commands, and the target host may be configured to execute the input commands. This allows the input commands to be executed remotely from the operator, which may increase operator safety.

[0038] According to a third aspect, a method for secure command execution is provided, comprising the steps of: obtaining an input command from an operator; evaluating the input command to determine whether the input command is secure using a community-based knowledge tag; and executing the input command based on the input command being secure. In this way, the knowledge of expert reviewers can be used to prevent unsafe command execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will now be described by way of examples with reference to the accompanying drawings. In the accompanying drawings:

[0040] Figure 1 It is the first table of classification operation in the prior art.

[0041] Figure 2A A first exemplary security command execution device in the prior art is schematically shown.

[0042] Figure 2B yes Figure 2A An exemplary policy definition of a first exemplary security command execution device in the prior art.

[0043] Figure 3 A flow chart of a second exemplary security command execution device in the prior art is schematically shown.

[0044] Figure 4 It is the second table of the classification operation of the prior art.

[0045] Figure 5 A third exemplary security command execution device in the prior art is schematically shown.

[0046] Figure 6 A fourth exemplary security command execution device in the prior art is schematically shown.

[0047] Figure 7 The top-level view of an exemplary security command execution device of the present application is schematically shown.

[0048] Figure 8 A detailed view of an exemplary security command execution device of the present application is schematically shown.

[0049] Figure 9 Schematically shows Figure 8A sequence diagram of an exemplary security command execution device of the present application.

[0050] Figure 10 An exemplary byte pair encoding algorithm pseudocode is shown.

[0051] Figure 11 An example of a computer-implemented method for secure command execution is shown.

[0052] Figure 12 An example of an apparatus for implementing the methods described herein is shown. DETAILED DESCRIPTION

[0053] Apparatus and methods described herein relate to secure command execution based on crowd-based knowledge tags.

[0054] Embodiments of the present invention can solve one or more of the aforementioned problems by evaluating input commands to determine whether the input commands are safe through group-based knowledge tags. In this way, the knowledge of expert reviewers can be used to prevent unsafe command execution.

[0055] This system prevents the execution of risky commands in cloud infrastructure, allowing only safe commands to execute. This system incorporates AI and crowd-based tagging to detect risky commands. Crowd-based tagging can label commands using the distributed knowledge of experts. These experts can be employees of the company implementing this safe command execution system.

[0056] AI-based classifiers can be used to overcome the limitations of existing rule-based methods in their generalizability to unseen commands. Specifically, AI-based classifiers can employ byte-pair segmentation and encoding specifically for the bash language. AI-based classifiers can consider relationships between words that frequently appear together, thereby improving classification results. AI-based classifiers can employ the bidirectional encoder representations from transformers (BERT) model specifically for the bash language. The BERT model can preserve the sequential nature of executed commands, thereby preserving the context of parameters, options, and flags. AI-based classifiers do not impose any special structure on their input. Active learning methods that leverage community-based knowledge can be used for labeling tasks. Community-based labeling allows a large group of experts to contribute their knowledge in a distributed manner to train AI-based classifiers. Community-based labeling reduces labeling costs by only hiring experts to label commands when the classifier lacks knowledge. Experts can be selected for labeling based on their areas of expertise. The opinions of various experts can be synthesized into ground truth.

[0057] Figure 7 The top-level view of an exemplary secure command execution device 700 of the present application is schematically shown.

[0058] Device 700 may include an operation agent 703. Operation agent 703 may communicate with operator 701. Operator 701 may be a user of device 700. The user may wish to execute a command. Operation agent 703 may communicate with target host 702. Target host 702 may be configured to execute the command. Operator 701, operation agent 703, and target host 702 may communicate. Communication may occur via a wired or wireless connection (e.g., a network). Alternatively, operator 701, operation agent 703, and target host 702 may form part of the same computer system.

[0059] The apparatus 700 may include an operation classifier 704. The apparatus may include a population tagging module 705. The operation agent 703, the operation classifier 704, and the population tagging module 705 may communicate. The communication may be via a wired or wireless connection (e.g., a network). Alternatively, the operation agent 703, the operation classifier 704, and the population tagging module 705 may form part of the same computer system.

[0060] Operator 701 may interact with the target host through operation agent 703. Operation agent 703 may intercept operations to be performed on target host 702 and forward them to operation classifier 704 where the risk of the operations is assessed. Operation classifier 704 may utilize the knowledge of an expert group through labels provided by the expert group for previously performed operations. Operation classifier 704 may be included to learn to distinguish between safe operations and risky operations. If the operation request is safe, it may be forwarded to target host 702 and the results of the operation may be forwarded back to operator 701. However, if the requested operation is risky, the operation may be blocked and operator 701 may be notified that the requested operation is not allowed.

[0061] Specifically, operator 701 may input a command to operation agent 703. Operator 701 may wish the command to be executed. Operation agent 703 may receive the command from operator 701. Operation agent 703 may output the command to operation classifier 704. Operation classifier 704 may classify the command into one of a plurality of categories. For example, operation classifier 703 may classify the command as safe or risky. Based on the classification, operation classifier 704 may output the classification to operation agent 703. Operation classifier 704 may determine the classification using community-based knowledge labels. Community-based knowledge labels may be based on the knowledge of command type experts.

[0062] If the operation classifier 704 cannot classify the command as safe or risky, the command may be classified as unknown. If there are no relevant community-based knowledge tags to provide information to classify the command, the command may be classified as unknown. If the command is classified as unknown, the command may be output to the community tagging module 705. The community tagging module 705 may determine the classification of the command. The community tagging module 705 may use knowledge from command type experts to determine the classification of the command. The community tagging module 705 may output the classification of the command back to the operation classifier 704.

[0063] If the classification is safe, the operation agent 703 may allow the command to be executed. The operation agent 703 may output the safe command to the target host 702. The target host 702 may execute the received safe command. The target host 702 may output the result to the operation agent 703. The operation agent 703 may output the result back to the operator 701. If the classification is risky, the operation agent 703 may not allow the command to be executed. If the classification is risky, the operation agent 703 may notify the operator that the command may not be executed because it has been classified as risky.

[0064] Figure 8 A detailed view of an exemplary security command execution device 700 of the present application is schematically shown. Figure 7 Each of the operation agent 703, the operation classifier 704 and the group labeling module 705 is Figure 8 . The operation agent 703, the operation classifier 704, and the group labeling module 705 each include multiple submodules. It should be understood that more or fewer submodules may also be suitable for performing the steps described herein. In other words, a submodule may be described herein as performing a particular step, but different submodules or the apparatus 700 (as a whole) may be suitable for performing those steps.

[0065] The apparatus 700 may be implemented on a cloud infrastructure.

[0066] Figure 9 Schematically shows Figure 8 In other words, Figure 9 Steps that may be implemented on the apparatus 700 are shown.

[0067] Device 700 can be used to obtain input command 831. Input command 831 can be input by operator 701. Operator 701 may desire input command 831 to be executed. Alternatively, device 700 can be used to request receipt of input command 831. Operator 701 may be an engineer. Operator 701 may be performing an O&M task. Operator 701 can perform actions via terminal 806. Terminal 806 can provide an interface to a shell environment.

[0068] Input commands 831 may include: Figure 1 The structure shown in the first row 101 of Table 100 in FIG. Input command 831 may include one or more atomic operations and corresponding parameters. Input command 831 may include instructions to be executed. Input command 831 may include instructions to be executed by target host 702. In addition to the command, input command 831 may also include information. For example, input command 831 may include information about the operator, information about the target host, and / or instructions about the command. Input command 831 may be implemented using the bash language.

[0069] Input command 831 can be received by terminal 806. In other words, terminal 806 can provide an entry point for operator 701 to interact with target host 702. Operator 701 can enter operations into the command line input (CLI). By pressing the Enter key, input command 831 can be intercepted and sent through other components of the system. Terminal 806 can operate in different compatibility modes. For example, terminal 806 can use Bash, but the same technology can also be applied to other compatibility modes, such as database interfaces. Terminal 806 can remove other information from input command 831. Terminal 806 can only output commands. Terminal 806 can output input command 831 to parser 807.

[0070] Typically, the operation in the input command 831 can contain multiple components. For example, a bash operation can be a program that includes commands, variables and / or conditions. The parser 807 can process the operation according to the syntax of the given shell environment and extract fragments (subcommands) for risk assessment. Only fragments that can change the system state are considered. These can form a list of commands 834. In addition, the parser 807 can be able to determine whether the operation is invalid (for example, contains a syntax error) and then notify the operator 701 through the terminal 806.

[0071] The parser 807 may be configured to divide an input command 831 into atomic operations and corresponding parameters 834. Depending on the content of the input command 831, the input command 831 may be divided into multiple atomic operations and corresponding parameters 834. The multiple atomic operations and corresponding parameters 834 may provide a list of commands 834.

[0072] The parser 807 can be used to divide the input command 831 into multiple atomic operations and corresponding parameters 834 based on the relationship between the pairs of atomic operations and corresponding parameters 834. In other words, the pairs of atomic operations and corresponding parameters 834 can be related to each other in some way, and the parser 807 can be used to divide the input command 831 based on the relationship between the pairs. For example, if it is found that these pairs are usually together in the input command, the parser 807 may or may not be used to decompose these pairs. Similarly, if it is found that these pairs are not usually together in the input command 831, the parser 807 may or may not be used to decompose these pairs. The context of the atomic operations and corresponding parameters 834 can be used to determine whether a pair exists.

[0073] The parser 807 may be configured to output a list of commands 834, including atomic operations and corresponding parameters 834, to the risk assessor 809. The risk assessor 809 may invoke the classifier 810 with each individual command in the list of commands 834. The classifier may return a label 832 (safe or risky) for each given command in the list of commands 834. The risk assessor 809 may aggregate each of the labels 832 to form an aggregated label 836. The aggregated label 836 may then be safe or risky. The aggregated label 836 may be based on an average of the labels 832. Alternatively, the aggregated label 836 may be based on a weighted average of the labels 832. The weighting may depend on the relative riskiness of the command types in the list of commands 834. In the case of a single command in the list of commands 834, a single label 832 may be generated, and the aggregated label 836 may be the single label 832.

[0074] The classifier 810 may be AI-based. When the input command 831 is implemented on bash, the classifier 810 may be implemented using BERT, GPT2, or GPT3 models.

[0075] The classifier 810 may take as input a command from the list of commands 834 and may be used to infer a label 832 for the command 834. The label 832 may be classified into one or three categories: safe, risky, and unknown. The safe label 832 may indicate that the command 834 is safe to execute. The risky label 832 may indicate that the command 834 is not safe to execute. The unknown label 832 may indicate that the command 834 cannot be classified. If the classifier 810 cannot identify the label 832 of the command 834, an unknown label 832 may be determined. If the classifier 810 cannot identify the label 832 of the command 834 within a specific level of certainty, an unknown label 832 may be determined. The level of certainty may be predetermined. The level of certainty may vary depending on the security requirements of the device 700.

[0076] The labels 832 may include group-based knowledge labels 832. In other words, the group-based knowledge labels 832 may have been previously determined by the expert reviewer 821. The expert reviewer 821 may be an expert in the particular type of command 834. For example, the expert reviewer 821 may be an engineer from the O&M department with expertise in a particular field or subset of operations. Alternatively or additionally, the current command 834 may be sent to the expert reviewer 821 for review. In this case, the group-based knowledge labels 832 may be determined directly from the current command 834. In either case, the group-based knowledge labels 832 have been determined by the expert reviewer 821, either previously or currently.

[0077] The classifier 810 may be used to determine whether the input command 831 is safe using one or more population-based knowledge labels 832. In other words, one or each of the one or more population-based knowledge labels 832 may be used to determine an aggregate label 836, and the aggregate label 836 indicates whether the input command 831 is safe.

[0078] The classifier 310 can be used to determine whether the input command 831 is safe using a classifier model 833. The classifier model 833 can be provided to the classifier 810. One or more community-based knowledge labels 832 can be incorporated into the classifier model 833. In other words, the classifier model 833 can have been trained using the one or more community-based knowledge labels 832. Since the classifier model 833 is based on the one or more community-based knowledge labels 832, determining whether the input command 831 is safe using the classifier model 833 is part of determining whether the input command 831 is safe using the community-based knowledge labels 832.

[0079] If the label 832 is determined to be unknown for command 834, the classifier 814 can forward the command 834 for review. The classifier 810 can output the command 834 to the reviewer console 813. The reviewer console 813 can forward the command 834 to one or more expert reviewers 821. There can be one or more expert reviewers 821. The expert reviewers 821 can be selected based on their knowledge to be included in the pool of expert reviewers 821. The expert reviewers 821 can evaluate the command 834. The expert reviewers 821 can return their evaluation of the command 834 to the reviewer console 813. The evaluation of the command 834 can include a population-based knowledge label 832. Alternatively, the population-based knowledge label 832 can be determined based on the expert reviewers' 821 evaluation. If there are multiple expert reviewers 821, each expert reviewer 821 can evaluate the command 834. Alternatively, a selected number of expert reviewers 821 can evaluate the command 834.

[0080] Classifier 810 may output a request to select a peer 837. Classifier 810 may output the request to select a peer 837 to peer selector 814. The request to select a peer 837 may include information about command 834. Peer selector 814 may be used to find the best expert reviewer 821 to review command 834. The selection of expert reviewer 821 may be based on a profile model 838. Profile model 838 may be provided to peer selector 814 by profile database 817. Profile 838 may include a list of expert reviewers 821 and their corresponding knowledge of different types of commands 834. Peer selector 814 may use profile model 838 to select an expert reviewer 821 to review command 834. In this way, expert reviewer 821 may be selected based on their knowledge of command 834. Peer selector 814 may output the selected expert reviewer 821 to reviewer console 813. A personal ID 839 may be assigned to the selected expert reviewer 821.

[0081] Each of the selected reviewers 821 may be notified that they have been selected to provide input by the reviewer console 813. The notification may include a command 834. The notification may include a pop-up window on their respective computer screens. The reviewer console 813 may include a user interface to record the expert reviewer's 821 evaluation of the command 834. The notification may include two questions: (1) whether the command 834 should be allowed in the current session; and (2) what is the general risk level of the command 834. The answers to the questions may be returned to the reviewer console 813.

[0082] The answers to the questions, commands 834, and / or personal IDs 839 may be output from the reviewer console 813 to the tag database 815. The tag database 815 may store the questions, commands 834, and / or personal IDs 839. The tag database 815 may store the information in raw data form.

[0083] In addition to or as an alternative to storing information, the tag database 815 can output questions, commands 834, and / or personal IDs 839 to a tag voting inference 816. The tag voting inference 816 can be used to aggregate the corresponding expert reviewer 821 evaluations. Based on the aggregated corresponding expert reviewer 821 evaluations, a group-based knowledge label 832 can be determined. The group-based knowledge label 832 can be based on an average of the expert reviewer 821 evaluations. Alternatively, the group-based knowledge label 832 can be based on a weighted average of the expert reviewer 821 evaluations. The weighting can depend on the corresponding knowledge of each expert reviewer 821. In the case of a single expert reviewer 821, a single group-based knowledge label 832 can be generated, and the single expert reviewer 821 evaluation can be the group-based knowledge label 832.

[0084] Label voting inference 816 may assess the difficulty of assigning command 834 based on the aggregated corresponding expert reviewers 821. For example, if most expert reviewers 821 agree that command 834 has a safe / risky label, then the command may be determined to be easy to evaluate. However, if most expert reviewers 821 disagree that command 834 has a safe / risky label, then the command may be determined to be difficult to evaluate.

[0085] Label voting inference 816 may also assign a knowledge score to each expert reviewer 821 based on the accuracy of the expert reviewer's 821 assessment. For example, if a particular expert reviewer 821 believes that command 834 is safe / risky, but a majority of other expert reviewers believe that command 834 is risky / safe, then the knowledge score of that expert reviewer 821 may be lowered for that type of command 834. Similarly, if a particular expert reviewer 821 believes that command 834 is safe / risky, and a majority of other expert reviewers believe that command 834 is similarly safe / risky, then the knowledge score of that expert reviewer 821 may be lowered for that type of command 834.

[0086] Tag voting inference 816 can output a knowledge score in the form of a personal profile model 840. Personal profile model 840 can be provided to a profile database 817. Profile database 840 can be updated based on personal profile model 840. For example, if the expert reviewer is accurate or inaccurate for a particular type of command 834, this will be recorded in the profile database. Updating profile database 840 can be performed out-of-band. In other words, profile database 840 can be updated offline. This allows updates to profile database 840 without interrupting the evaluation of the current command 834.

[0087] Label voting inference 816 can output commands 834, group-based knowledge labels 832, and / or the difficulty of commands 834 to a command database 818. Command database 818 can store commands 834, group-based knowledge labels 832, and / or the difficulty of commands 834. Command database 818 can store information in raw data form.

[0088] The reviewer console 813, peer selector 814, and label voting inference 816 are described in more detail. The ML approach proposed by the system may require labeling of commands 834. In a large-scale system running hundreds of services, the number of commands 834 and parameters can be in the millions. Therefore, expert reviewers 821 may need to manually label a large set of instances. A simple approach may be to present operators 701 with commands for labeling before they can execute their own commands 834. Once the commands 834 are labeled by several operators 701, the system can use a "majority vote" heuristic to assign a final label 832 to the commands 834. However, the given labels 832 are often noisy or inaccurate because the operators 701 do not have the time to label them correctly or lack the necessary knowledge. Therefore, the "majority vote" heuristic has two main limitations: (1) it may not take into account the expertise of the operators 701, and (2) some commands 834 are more difficult to understand and label than others.

[0089] Therefore, the system can employ a probabilistic model that considers the expertise of each expert reviewer 821 and the difficulty of each command when calculating the label to be assigned to command 834. The system can automatically infer command difficulty and the expertise of the expert reviewers 821. Benefits of the system can include: (i) higher quality and efficiency of labeling commands 834, (ii) higher efficiency of labeling because fewer votes / labels from expert reviewers 821 are required, (iii) complex commands can be assigned to more experienced expert reviewers 821, and (iv) the experience of expert reviewers 821 is automatically derived from their input to the system.

[0090] The method for automatically inferring command difficulty and expert reviewer 821 expertise is described below. The goal is to find the most likely values ​​of the unobservable variables zj, α, β given the observed labels l.

[0091] Classifier 810—output (command): “rm -rf / home”.

[0092] Peer Selector 814 - estimates difficulty d(command) using a distance metric to existing commands, selects n operators based on (difficulty d, expertise α) and output (one or more personal ids): [John, Maria, ..., Name_n].

[0093] Reviewer console 813 – output (command): “rm -rf / home”, input (labels): [John, risky] and output (personal id, command, labels): [John, “rm -rf / home”, risky].

[0094] Label voting inference 816 – Input (person id, command, label): [John, “rm-rf / home”, risky], observed variables: l = label, unobserved variables: zj = true label, α = expertise, β = difficulty, and α←~0, α<0, β←~0, β<0.

[0095] When it does not converge: –E step: p(zj|l,α,β)∝p(zj)∏p(lij|zj,αi,βj), M step: ∑ j E[lnp(zj)]+∑ ij E[lnp(lij|zj,αi,βj)].

[0096] Output (personal profile model): [John, expertise α].

[0097] output(command, labels): ["rm -rf / home", label argmaxp(zj)].

[0098] Output(command, difficulty): ["rm -rf / home", difficulty β].

[0099] The expectation-maximization (EM) algorithm implemented by the label voting inference 816 component can enable estimation of command complexity and operator expertise when data is incomplete or missing. Directly modeling this data can be difficult because it is generated by a process with hidden variables.

[0100] In addition to or as an alternative to storing information, command database 818 can output commands 834 and / or community-based knowledge labels 832 to classifier model training 812. Classifier model training 812 can use command 834 and community-based knowledge label 832 pairs to train / update classifier model 833. The update of classifier model 833 can be performed in an out-of-band process. In other words, classifier model 833 can be updated offline. This allows updates to classifier model 833 without interrupting the evaluation of current commands 834.

[0101] Once the classifier model 833 has been updated, the updated classifier model 833 can be provided by the classifier model training 812 to the classifier model database 811. The classifier 810 can use the updated classifier model 833 from the classifier model database 811 to evaluate the command 834, as described herein.

[0102] Additionally, command database 818 may output commands 834, community-based knowledge tags 832, and / or the difficulty of commands 834 to command analytics dashboard 819. Command analytics dashboard 819 may use the received information to provide insight into command database 818 to a reviewer.

[0103] Terminal 806 may also output input command 831 to access control unit 808. Access control unit 808 may receive aggregated label 836 from risk assessor 809. If aggregated label 836 indicates risk, input command 831 may not be executed. If aggregated label 836 indicates risk, operator 701 may be notified of the risk of input command 831 via terminal 806. Operator 701 may be notified via an operation block notification. If aggregated label 836 indicates safety, input command 831 may be executed. If aggregated label 836 indicates safety, operator 701 may be notified of the safety of input command 831 via terminal 806.

[0104] If the aggregated tag 836 is secure, the input command 831 can be determined to be secure for execution. In this case, the input command 831 can be output to the target host 702. The target host 702 can execute the input command 831. The target host 702 may include a machine in a production environment that executes the input command 831. There may be multiple target hosts 702. A single target host 702 may be used for execution. Alternatively, multiple target hosts 702 may be used for execution. The input command 831 may include information indicating which target host(s) 702 should be used for execution.

[0105] Figure 10 An exemplary byte pair encoding algorithm pseudo code 1000 is shown. The byte pair encoding algorithm pseudo code 1000 may be used to train the classifier model 833.

[0106] The classifier model 833 may be a bidirectional transformer for language understanding (BERT) model 833. The model 833 may use a byte-pair encoder (BPE) to encode the input. The BERT model 833 may first be pre-trained by employing word masking and next sequence prediction tasks on a large unlabeled bash command corpus. The model 833 may be fine-tuned to perform the classification task of determining risky commands and safe commands.

[0107] BPE training. A large unlabeled command corpus can be collected. The unlabeled commands can be used as input for training the BPE model 833. The pseudo code of the BPE algorithm is as follows Figure 10As shown in Figure 2 . The input parameter to this stage can be the number of combinations of terms k to be stored as a vocabulary. The BPE algorithm can parse the entire unlabeled bash corpus and generate k vocabulary entries. The BPE algorithm can also perform an encoding corresponding to each vocabulary entry. Before any further processing, all inputs to the classifier pre-training and training stages can first be encoded using the vocabulary generated by the BPE algorithm and its encoding.

[0108] Classifier pre-training. During the pre-training phase, two datasets can be generated using a large unlabeled bash command corpus. These two datasets can be generated using word masking and next sequence prediction pre-training tasks. In the word masking task, for a given command in the large bash corpus, a masked command can be generated such that α% of the words are removed. The model can be pre-trained to guess the removed words. In the next sequence prediction task, the large bash corpus can first be divided into word blocks. Then, for β% of all two consecutive blocks, the second block can be swapped with another random block, while (100-β)% of the two consecutive blocks remain unchanged. The model can be pre-trained to learn to determine whether a block pair is unchanged or changed. The output of the pre-training can be a model 833 that is able to capture the important relationships and co-occurring terms of the bash language as well as important grammatical structures. The pre-trained model 833 can then be fine-tuned to perform specific language tasks.

[0109] Classifier fine-tuning. During the fine-tuning phase, labeled commands 834 stored in the command database 818 can be used. Commands 834 and their associated labels 832 can then be provided to a model 833 pre-trained on a large corpus of bash commands. Model 833 can then be trained to distinguish between safe commands 834 and risky commands 834. At this stage, a fully connected classification layer can be added to the pre-trained model 833, and training can be performed using a backpropagation algorithm.

[0110] The system described herein may provide the following advantages:

[0111] Over time, the scheme can improve itself by training on new data samples observed from real-world environments.

[0112] Labeling can be done by many engineers, whose expertise can be incorporated into the labeling process.

[0113] This solution can be extended to other CLI tools such as database or network device management.

[0114] The scheme can provide a language model to classify executed commands as risky or safe.

[0115] The language model can generalize to unseen combinations of commands, subcommands, and flags.

[0116] The language model can preserve the sequential nature of executed commands, taking into account the context in which flags and subcommands are used.

[0117] Existing security solutions, such as Unix security, assume that users are registered with the operating system and that permissions for resources can be managed. In large-scale cloud infrastructure, these two assumptions do not hold true. The operating system may not know the operator and therefore cannot assign permissions to resources. This system can provide a layer above the operating system that manages and enforces security rules for executing commands on resources.

[0118] The system provides a mechanism for dynamically discovering security rules to manage cloud infrastructure. The system can use an in-sourcing mechanism to retrieve, aggregate, and integrate operator opinions based on their expertise and the complexity (difficulty) of the commands.

[0119] The proposed scheme not only scales with the number of operators, but also performs even better when the pool of available operators is large. The literature shows that aggregating information from independent and decentralized groups often produces better decisions than any single member of the group could make.

[0120] Aggregation of independent and dispersed opinions can be performed automatically by using expectation maximization techniques on operators' past opinions on order risk.

[0121] Aggregated opinions can be constructed using machine learning techniques to enable automatic extraction of security rules.

[0122] The apparatus 700 may be implemented in the following systems:

[0123] Desktops and laptops: Companies can deploy Device 700 to prevent employees from running commands that could damage the operating system. Risky commands may be blocked and may require additional permissions to execute.

[0124] ChatOps / DevOps / SRE: ChatOps systems (such as IBM ChatOps, OpsGenie, Mattermost) enable SRE / operators to send commands to the Kubernetes cluster via Slack when an incident occurs. The device 700 can be linked to a Slack channel and prevent the execution of certain commands based on their risk level.

[0125] Home routers and IoT devices: Any IoT device that can be remotely configured using a CLI (such as an e-On smart meter) can be protected using the appliance 700.

[0126] Industrial Planning: Industrial factories have real-time event response solutions that can send commands to machines when they need to be repaired or reconfigured. Device 700 can also be used here.

[0127] The apparatus 700 may have the capability to protect any device / computer / service / system that operates using a CLI.

[0128] Figure 11 An example of a method 1100 for secure command execution is summarized. In step 1101, the method 1100 includes obtaining an input command from an operator. In step 1102, the method 1100 includes evaluating the input command to determine whether the input command is secure using a population-based knowledge tag. In step 1103, the method 1100 includes executing the input command based on the input command being secure.

[0129] Figure 12 An example of an apparatus 1200 for implementing the method 1000 is schematically shown. The computing apparatus 1200 may include the apparatus 700. The apparatus 1200 may be implemented on an electronic device such as a laptop, a tablet computer, a smartphone, or a television.

[0130] The apparatus 1200 includes a processor 1201 for processing a data set in the manner described herein. For example, the processor 1201 may be implemented as a computer program running on a programmable device such as a central processing unit (CPU). The apparatus 1200 includes a memory 1202 arranged to communicate with the processor 1201. The memory 1202 may be a non-volatile memory. The processor 1201 may also include a cache memory ( Figure 12 (not shown), the cache memory can be used to temporarily store data from memory 1202. The apparatus may include more than one processor and more than one memory. The memory may store data executable by the processor. The processor may be configured to operate according to a computer program stored in a non-transitory form on a machine-readable storage medium. The computer program may store instructions for causing the processor to perform its method in the manner described herein.

[0131] Applicants hereby disclose individually each individual feature described herein and any combination of two or more such features. It would be within the ordinary skill of those skilled in the art to be able to implement such features or combinations as a whole based on this specification, without regard to whether such features or combinations of features solve any of the problems disclosed herein, and without limiting the scope of the claims. Applicants indicate that aspects of the present invention may include any such individual features or combinations of features. In view of the foregoing description, it will be apparent to those skilled in the art that various modifications may be made within the scope of the present invention.

Claims

1. A computing device (1200) for secure command execution, characterized in that: The computing device (1200) comprises one or more processors (1201) and a memory (1202), the memory (1202) storing data defining program code executable by the one or more processors (1201) in a non-transitory form, wherein the program code is executable by the one or more processors (1201) such that the computing device (1200) is configured to: Obtaining input commands (831) from an operator (701); evaluating the input command (831) to determine whether the input command (831) is safe using a population-based knowledge tag (832); According to the input command (831) being safe, the input command (831) is executed.

2. The computing device (1200) according to claim 1, characterized in that The apparatus (1200) is configured to evaluate the input command (831) to determine whether the input command (831) is safe using a classifier model (833), one or more community-based knowledge labels (832) being incorporated into the classifier model (833).

3. The computing device (1200) according to claim 1 or 2, characterized in that The apparatus (1200) is configured to determine the group-based knowledge tag (832) by performing the following operations: providing the input command (831) to the expert reviewer (821); The expert reviewer (821) evaluates the input command (831); Based on the expert reviewer (821) evaluation, the population-based knowledge tag (832) is determined.

4. The computing device (1200) according to claim 3, characterized in that The apparatus (1200) is configured to update the classifier model (833) using the population-based knowledge labels (832) determined from the expert reviewer (821) evaluations.

5. The computing device (1200) according to claim 3 or 4, characterized in that The apparatus (1200) is configured to store the community-based knowledge tags (832) determined from the expert reviewer (821) evaluation.

6. The computing device (1200) according to any one of claims 3 to 5, characterized in that: The apparatus (1200) is configured to determine the population-based knowledge label (832) solely based on the population-based knowledge label (832) for which the classifier model (833) is unable to identify the input command (831) within a certainty level.

7. The computing device (1200) according to any one of claims 3 to 6, characterized in that: The device (1200) is used to select the expert reviewer (821) based on the expert reviewer's (821) knowledge of the input command (831).

8. The computing device (1200) according to any one of claims 3 to 7, characterized in that: The apparatus (1200) is configured to determine the group-based knowledge tag (832) by performing the following operations: providing the input command (831) to a plurality of expert reviewers (821); Each expert reviewer (821) evaluates the input command (831); Aggregate the corresponding expert reviewers' (821) evaluations; The population-based knowledge tags (832) are determined based on the aggregated expert reviewer (821) evaluations.

9. The computing device (1200) according to claim 8, characterized in that The apparatus (1200) is configured to aggregate the respective expert reviewer (821) evaluations by weighting each expert reviewer (821), the weighting being dependent on the expert reviewer's (821) knowledge of the input command (831).

10. The computing device (1200) according to any one of the preceding claims, characterized in that The device (1200) is used to: Dividing the input command (831) into a plurality of atomic operations and corresponding parameters (834); evaluating each of the atomic operations and corresponding parameters (834) to determine whether the atomic operations and corresponding parameters (834) are safe using a population-based knowledge label (832); According to each of the atomic operations and corresponding parameters (834) being safe, the input command (831) is executed.

11. The computing device (1200) according to claim 10, characterized in that The device (1200) is used for dividing the input command (831) into a plurality of atomic operations and corresponding parameters (834) according to the mutual relationship between the pairs of atomic operations and corresponding parameters (834).

12. The computing device (1200) according to claim 10 or 11, characterized in that The apparatus (1200) is configured to evaluate each of the atomic operations and corresponding parameters (834) to determine whether the atomic operations and corresponding parameters (834) are safe based on the context of the atomic operations and corresponding parameters (834).

13. A cloud infrastructure, characterized in that Comprising a computing device (700) as claimed in any one of the preceding claims.

14. The cloud infrastructure according to claim 13, wherein: Including operator (701), agent (703) and target host (702): The agent (703) is used to: obtain an input command (831) from the operator (701) and evaluate the input command (831); The target host (702) is used to execute the input command (831).

15. A method (1100) for secure command execution, characterized in that: The method (1100) comprises the following steps: Get input command from operator (1101); evaluating the input command to determine whether the input command is safe through a population-based knowledge tag (1102); According to the input command being safe, the input command is executed (1103).

Images