State expression device, learning device, state expression method, and state expression program

Abstract

To provide a state representation apparatus, a learning apparatus, a state representation method, and a state representation program that can generate an efficient representation indicating a state of a host.SOLUTION: A state representation apparatus 1 includes a tree structure generation unit 11 that configures, for each node corresponding to a host, a tree structure with each type of attribute related to the host as a node lower than the node corresponding to the host, and each value of the attribute as a node lower than the node corresponding to the type of the attribute, and assigns, for each of the node, coding data indicating the type of the attribute and the value of the attribute, and an encoding unit 12 that generates a predetermined dimensional vector as a representation indicating a state of the host by an encoding function using the tree structure and the coding data as input. The encoding function is learnt so that a representation that reflects a similarity of the tree structure and the coding data is output by a representation learning algorithm.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an apparatus, a method, and a program for obtaining an expression indicating a state of a network system as input data for machine learning. [Background technology]

[0002] Conventionally, attempts have been made to improve the efficiency of network system management by using machine learning techniques such as reinforcement learning. For example, Non-Patent Document 1 proposes a technique for automating vulnerability testing in a network system by using reinforcement learning.

[0003] In order to handle a network system with machine learning techniques such as reinforcement learning, it is necessary to express the state of the network system in a format that can be input to a machine learning algorithm, such as a vector or a matrix. Specifically, it is necessary to express the network topology and the properties of the hosts connected to the network system, such as the network subnet, addresses, device types, and services or applications installed, in the form of a vector or a matrix.

[0004] In the conventional method, first, the properties of hosts connected to a network are organized as pairs (tuples) of an attribute and a value, and then a unique code is assigned to each value in each attribute. For example, if the attribute port number can take three values, 22, 80, and 3389, these three values ​​can be assigned as vector-format codes such as [1,0,0], [0,1,0], and [0,0,1]. Finally, flag-format data expressing the status of the host is constructed by concatenating the codes indicating the respective values ​​obtained from all the attributes (for example, see Non-Patent Documents 1 and 2). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Z. Hu et al., "Automated penetration testing using deep reinforcement learning," Proc. EuroS&P Workshops, 2020. [Non-Patent Document 2] R. Maeda et al., "Automating post-exploitation with deep reinforcement learning," Computers & Security, 2021. [Non-Patent Document 3] KS Tai et al., "Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks," Proc. ACL, 2015. [Non-Patent Document 4] Z. Zhang et al., "Deep Learning on Graphs: A Survey," IEEE Transactions on Knowledge and Data Engineering, 2020. [Non-Patent Document 5] T. Chen et al., "A simple framework for contrastive learning of visual representations," Proc. International Conference on Machine Learning, 2020. Summary of the Invention [Problem to be solved by the invention]

[0006] However, in conventional methods, in order to express the host state in detail, it is necessary to define a large number of attributes, and the vectors or matrices that are constructed are long. As a result, not only is more memory required for processing by machine learning, but the number of states increases in reinforcement learning, and the convergence of learning deteriorates. In addition, after constructing a machine learning model, in order to add a new attribute to the host state representation, it is necessary to reconstruct the machine learning model with an increased number of dimensions and relearn it. Due to these problems, conventional methods can only handle limited problem settings, making it difficult to handle real-world network systems.

[0007] An object of the present invention is to provide a state expression device, a learning device, a state expression method, and a state expression program that are capable of generating an efficient expression that shows the state of a host. [Means for solving the problem]

[0008] The state representation device of the present invention comprises a tree structure generation unit that constructs, for a node corresponding to a host, a tree structure in which types of attributes related to the host are placed as nodes lower than the node corresponding to the host, and values ​​of each attribute are placed as nodes lower than the node corresponding to the type of attribute, and that assigns coded data indicating the type of attribute or the value of the attribute to each node, and an encoding unit that generates a vector with a predetermined number of dimensions as a representation indicating the state of the host using an encoding function that has the tree structure and the coded data as input, and the encoding function is trained by a representation learning algorithm to output a representation that reflects the similarity between the tree structure and the coded data.

[0009] The tree structure generating unit may construct the tree structure by subdividing the attribute types or the attribute values ​​into parent-child relationships.

[0010] The representation learning algorithm may be unsupervised contrastive learning.

[0011] The encoded data may be real values ​​or a real vector and the representation may be a real vector.

[0012] The learning device of the present invention randomly constructs, for a node corresponding to a host, a number of tree structures in which types of attributes related to the host are placed as nodes lower than the node corresponding to the host, and values ​​of each attribute are placed as nodes lower than the node corresponding to the type of attribute, and each node is assigned coded data indicating the type of attribute or the value of the attribute; an encoding unit that generates a vector of a predetermined number of dimensions as an expression indicating the state of the host using an encoding function that takes the tree structure and the coded data as input; and a learning unit that learns the encoding function using an expression learning algorithm to output an expression that reflects the similarity between the tree structure and the coded data.

[0013] The state representation method of the present invention includes a tree structure generation unit step of constructing, for a node corresponding to a host, a tree structure in which types of attributes related to the host are placed as nodes lower than the node corresponding to the host, and values ​​of each attribute are placed as nodes lower than the node corresponding to the type of attribute, and assigning coded data indicating the type of attribute or the value of the attribute to each node, and an encoding step of generating a vector with a predetermined number of dimensions as a representation indicating the state of the host using an encoding function that has the tree structure and the coded data as input, and the encoding function is trained by a representation learning algorithm to output a representation that reflects the similarity between the tree structure and the coded data.

[0014] A state representation program according to the present invention causes a computer to function as the state representation device. Effect of the Invention

[0015] The present invention provides an efficient representation of the state of a host. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing a functional configuration of a state representation device according to an embodiment. [Diagram 2] 11 is a diagram illustrating, in a table format, the correspondence between attribute types and attribute values ​​in the embodiment. FIG. [Diagram 3] FIG. 2 is a diagram illustrating an example of a tree structure showing the status of a host in an embodiment. [Figure 4] 1 is a flowchart showing the procedure of a state representation method in an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] An example of an embodiment of the present invention will now be described. The state representation device of this embodiment is a device that generates a representation indicating the state (properties) of a network host in vector format, for use as input data for a machine learning model or the like.

[0018] FIG. 1 is a block diagram showing the functional configuration of a state representation device 1 according to this embodiment. The state representation device 1 is an information processing device (computer) that includes a control unit 10, a storage unit 20, various input / output devices, communication devices, and the like.

[0019] The control unit 10 is a part that controls the entire state representation device 1, and realizes each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU.

[0020] The storage unit 20 is a storage area for various programs for causing the hardware group to function as the state representation device 1, various data, and the like, and may be a ROM, a RAM, a flash memory, a hard disk drive (HDD), or the like.

[0021] The control unit 10 includes a tree structure generating unit 11, an encoding unit 12, and a learning unit 13. Using these functional units, the state representation device 1 also functions as a learning device that organizes the states of sample hosts into a tree structure and optimizes the encoding function for embedding this into a vector using a representation learning algorithm, and outputs a representation showing the state of any host using the optimized encoding function. The learning device may be provided independently of the state representation device 1, in which case the parameters of the encoding function optimized by the learning device are provided to the state representation device 1.

[0022] The tree structure generating unit 11 constructs a tree structure in which, for a node corresponding to a host, the types of attributes related to this host are placed as nodes lower than the node corresponding to the host, and the values ​​of each attribute are placed as nodes lower than the node corresponding to each type of attribute, and assigns coded data indicating the type or value of the attribute to each node. The coded data may be a real value or a real vector. At this time, tree structure generating unit 11 may construct a tree structure by subdividing attribute types or attribute values ​​into parent-child relationships.

[0023] The encoding unit 12 generates a vector with a predetermined number of dimensions as an expression indicating the state of the host, using an encoding function that receives the tree structure and the encoded data as input. Note that the generated representation may be a real vector.

[0024] The learning unit 13 uses multiple randomly constructed tree structures and the encoded data assigned to each node to learn the encoding function through a representation learning algorithm so as to output an expression that reflects the similarity between the tree structures and the encoded data. The representation learning algorithm may be, for example, unsupervised contrastive learning.

[0025] [Representing host states using a tree structure] The tree structure of this embodiment is composed of a set of nodes. A node is expressed as (value, set of child nodes). A single node is set as the root of the tree. All nodes other than the root have exactly one parent node. In other words, nodes designated as child nodes of each node do not have any other parent nodes.

[0026] The value of each node indicates the type of an attribute for indicating the state of the host, or the value of the attribute. The type of attribute refers to the type of information obtained from the host, such as the OS running on the host, the services (applications), the open ports, etc. The value of the attribute refers to the value that actually corresponds to the host for the type of attribute, such as the name of the OS running, the name of the application, the number of the open port, etc.

[0027] FIG. 2 is a diagram showing, in a table format, the correspondence between attribute types and attribute values ​​in this embodiment. Based on information acquired from the host, for example, the value "Windows (registered trademark)" is associated with the attribute type "OS", and multiple values ​​"SSH, RDP, Apache (registered trademark) 2.4" are associated with the attribute type "Service".

[0028] FIG. 3 is a diagram illustrating a tree structure showing the state of a host in this embodiment. The host "Machine A" is the root, and the attribute types "OS," "Service," and "Port" are each child nodes. Although the node corresponding to the host is illustrated as the root here, this is not limiting, and for example, a plurality of hosts in a network including this host may be represented in a hierarchical structure.

[0029] In addition, if both the type and value of an attribute can be subdivided to express it hierarchically, the corresponding node can be divided into multiple hierarchies. For example, if the version number of application A is 2.4, it can be divided into "application A", "2" (major version), and "4" (minor version), and expressed as a parent-child relationship starting from "application A".

[0030] [Tree Encoding] Let the tree structure be a graph G = (V, E), where V is a set of nodes and E is a set of edges. An edge e ∈ E is expressed as a tuple of nodes, and an edge (u, v) exists when a node v ∈ V exists as a child node of a node u ∈ V.

[0031] The matrix X denotes a set of values, and the value of node v is x v ∈X. In this case, if the value of the node cannot be directly expressed by a number or a vector, it is encoded and expressed in a number or vector format. The encoding method is not limited. v is a real value or a vector whose elements are real numbers.

[0032] Let us define an encoding function that encodes the entire tree structure based on a graph G and a set of values ​​X as f Enc Let the expression corresponding to the tree structure be h, then h=f Enc The result is (G,X). Here, the expression is a vector of real numbers having a specified number of dimensions. A specific example of the encoding function is a Tree LSTM (see, for example, Non-Patent Document 3) using a recurrent neural network, or a graph learning algorithm (see, for example, Non-Patent Document 4).

[0033] For example, first, x v Using the graph learning algorithm, we first calculate the representation h corresponding to node v. v Then, the read function f ReadoutBy reading and aggregating the representations of all nodes, we can obtain the representation h for the tree structure. Readout As a concrete example of this, we use f Readout ({h v |v∈V})=Σ v∈V h v Examples include:

[0034] [Representation learning algorithm] Representation learning aims to obtain a representation that reflects important features in the input data. For the original input data X, we use an encoding function f Enc To obtain a representation h, representation learning involves applying an encoding function f Enc is optimized.

[0035] One of the representation learning algorithms is unsupervised contrastive learning. Unsupervised contrastive learning is an algorithm that optimizes an encoding function to obtain a representation of a sample of interest through comparison with similar and dissimilar samples without using a label. Here, a sample refers to a high-dimensional vector or matrix, or information that cannot be expressed in matrix form.

[0036] For example, the method described in Non-Patent Document 5 can be applied to expression learning as follows. Here, we consider an input sample x k To obtain the representation h corresponding to (k∈[N]), we use the encoding function f Enc Think about optimizing it.

[0037] First, the learning unit 13 performs two data augmentation operations t and t' to extract the input sample x k Two similar samples from x~ 2k-1 ,x~ 2k Data augmentation is a process of generating new samples similar to existing samples in order to ensure a sufficient number of training data in machine learning. For example, in the case of an image, this refers to rotation or enlargement, and in the case of a graph, the replacement of node attributes.

[0038] Next, the encoding function f Enc Therefore, h=f Enc (x k ), the learning unit 13 obtains the representation h as two samples x~ obtained from the data augmentation operation. 2k-1 ,x~ 2k The corresponding expression for h 2k-1 ,h 2k Further, the learning unit 13 obtains the two representations h by using the projection function g used for representation learning. 2k-1 ,h 2k For z=g(h), we have two latent representations z 2k-1 ,z 2k get.

[0039] Here, the latent representation z i ,z j The similarity of i,j =z i T z j / (∥z i ∥∥z j ∥) and the loss for positive examples is

number

number

[0040] FIG. 4 is a flowchart showing the procedure of the state representation method in this embodiment. In step S1, the state representation device 1 determines the types and values ​​of attributes that the host can take as its state. This information may be defined by input from a user, or may be extracted from the results of collecting logs in a real environment.

[0041] In step S2, the tree structure generator 11 randomly selects attribute types and values ​​from the ranges determined in step S1, sets them as virtual hosts, and expresses this state as a tree structure and a set of attribute values ​​(G, X). The tree structure generator 11 generates a large number of these (N pieces) to create a data set D.

[0042] In step S3, the encoding unit 12 calculates an encoding function f Enc Configure and initialize. After that, in steps S4 to S5, the learning unit 13 uses expression learning to learn the encoding function f Enc Optimize the weights of Here, unsupervised contrastive learning is used.

[0043] In step S4, the learning unit 13 selects the samples (G k ,X k )(k∈[N]), use data augmentation operations t, t' to 2k-1 ,X' 2k-1 ),(G' 2k ,X' 2k ) to construct a data set D' containing 2N pieces of data.

[0044] In step S5, the learning unit 13 calculates the data set D′ and the encoding function f Enc Using the above loss function as unsupervised contrast learning, we set the encoding function f Enc and optimizing the weights of the projection function g.

[0045] In this way, in step S6, the encoding unit 12 encodes an arbitrary sample (G, X) using the optimized encoding function f Enc By giving h=fEnc We obtain the representation h as (G,X).

[0046] According to this embodiment, the state representation device 1 (learning device) represents the state of the host in advance using a tree structure, and optimizes an encoding function that converts the tree structure into a vector with a predetermined number of dimensions using representation learning such as unsupervised contrastive learning. As a result, the state representation device 1 inputs the state of any host represented by the tree structure into the optimized encoding function, thereby obtaining a vector-format representation that effectively represents the state of this host. That is, the state representation device 1 is capable of representing a large amount of information about the host state with a fixed number of dimensions that is smaller than conventionally possible, and can output an efficient representation showing the host state while suppressing an increase in memory.

[0047] In addition, in conventional flag-format representations, the values ​​themselves have no meaning, and for example, mutual similarity is not expressed by distance or the like and cannot be determined mathematically. On the other hand, the state representation device 1 of this embodiment represents hierarchical information in a tree structure, and further outputs a vector that reflects the similarity of the tree structure and the encoded data by a representation learning algorithm. Therefore, the state representation device 1 can output an expression showing the state of the host as a vector with a certain number of dimensions that is mathematically meaningful even if the amount of information increases. As a result, for example, a mathematically similar expression can be obtained for an unknown similar host, so that appropriate analysis results can be expected in various analyses such as security judgments that use such expressions as input, making it possible to deal with unknown information.

[0048] Furthermore, the state representation device 1 can appropriately represent more detailed host information as mathematically meaningful vectors by subdividing attribute types or attribute values ​​into parent-child relationships to form a tree structure and further by handling real values ​​or real vectors as encoded data.

[0049] Furthermore, with this embodiment, for example, the state of a network system can be appropriately vectorized to make management more efficient, making it possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0050] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the above-described embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0051] The state representation method and learning method by the state representation device 1 are realized by software. When realized by software, a program constituting this software is installed in an information processing device (computer). These programs may be recorded on a removable medium such as a CD-ROM and distributed to users, or may be distributed by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a Web service via a network without being downloaded. [Explanation of symbols]

[0052] 1 State representation device (learning device) 10 Control section 11 Tree structure generator 12 Encoding section 13 Learning Department 20 Memory section

Claims

1. a tree structure generating unit that configures a tree structure in which, for a node corresponding to a host, types of attributes related to the host are placed as nodes lower than the node corresponding to the host, and values ​​of each attribute are placed as nodes lower than the node corresponding to the type of attribute, and that assigns coded data indicating the type of attribute or the value of the attribute to each node; an encoding unit that generates a vector with a predetermined number of dimensions as an expression indicating a state of the host by an encoding function that receives the tree structure and the encoded data as input, A state representation device in which the encoding function is trained through unsupervised contrastive learning to output a representation that reflects the similarity between the tree structure and the encoded data.

2. 2. The state representation device according to claim 1, wherein the tree structure generating unit configures the tree structure by subdividing the attribute types or the attribute values ​​into parent-child relationships.

3. 3. The state representation device according to claim 1, wherein the encoded data is a real value or a real vector, and the representation is a real vector.

4. a tree structure generating unit that randomly generates a plurality of tree structures in which, for a node corresponding to a host, types of attributes related to the host are placed as nodes lower than the node corresponding to the host, and values ​​of each attribute are placed as nodes lower than the node corresponding to the type of attribute, and that assigns coded data indicating the type of attribute or the value of the attribute to each node; an encoding unit that generates a vector with a predetermined number of dimensions as an expression indicating a state of the host by an encoding function that receives the tree structure and the encoded data as input; a learning unit that learns the encoding function by unsupervised contrastive learning so as to output an expression that reflects the similarity between the tree structure and the encoded data.

5. a tree structure generating unit step of constructing a tree structure in which, for a node corresponding to a host, types of attributes related to the host are placed as nodes lower than the node corresponding to the host, and values ​​of each attribute are placed as nodes lower than the node corresponding to the type of attribute, and giving each node coded data indicating the type of attribute or the value of the attribute; an encoding step of generating a vector with a predetermined number of dimensions as an expression indicating a state of the host by an encoding function that receives as input the tree structure and the encoded data, A state representation method in which the encoding function is trained through unsupervised contrastive learning to output a representation that reflects the similarity between the tree structure and the encoded data.

6. A state representation program for causing a computer to function as the state representation device according to claim 1 or 2.

Images