Packet data classification

Abstract

This document provides methods for classifying packet data, etc. [Solution] The method implemented by a computer includes: generating a target flow summary containing information about packet flow between two addresses during a target period based on input network packet data; generating a target latent space representation based on the target flow summary; identifying k reference latent space representations that are most similar to the target latent space representation, such that multiple reference latent space representations correspond to multiple reference flow summaries; generating a target flow report in natural language format based on the target flow summary using a first LLM; generating k reference flow reports in natural language format based on the k reference flow summaries using the first LLM; and classifying packet flows as malicious or harmless based on the target flow report and the k reference flow reports with corresponding classifications using a second LLM.

Description

Technical Field

[0001] The present invention relates to packet data classification, and more specifically, to a method implemented by a computer, a computer program, and an information processing device.

Background Art

[0002] The world of cybersecurity is becoming larger and more complex year by year. Both state and non-state actors attack and endanger organizations, infrastructure, elections, etc. There are many tools for defending against external attacks, but currently, those tools are dealing with the problem either by creating a series of rules regarding network communications or by detecting and investigating abnormal network communications. The former is a more basic "if... then..." statement, which scrutinizes a huge amount of communication records and highlights abnormal behaviors. The latter uses some deep learning algorithms (mainly variations of autoencoders (AE) or recurrent neural networks (RNN)) to classify network communications into two categories: normal and abnormal.

[0003] The emergence of large language models (LLMs) has shown great potential in many fields. These models have demonstrated excellent capabilities in understanding and describing natural language texts. For example, they can perform functions such as question answering (queries regarding history and science, etc.), filling in blanks within text, generating new content based on user requests (proposing store names, creating short stories, creating posts for social media, etc.), text summarization, and code generation.

[0004] Such LLMs are first pre-trained using a very large corpus of data, including texts on the Internet (such as Wikipedia, Reddit, etc.) and books in the public domain. After the pre-training stage is completed, they can be fine-tuned to solve specific tasks or used as general chatbots.

[0005] One of the major drawbacks of LLM is that it is primarily limited to natural text tasks. While efforts have been made, and continue to be made, to incorporate image generation and analysis capabilities into LLM, little has been done to understand and analyze non-text data, especially network transmissions.

[0006] The limitations of these models have been overcome in three ways. Prompt engineering is the process of structuring instructions given to an LLM to obtain the desired results. This can be a costly and time-consuming process, as the length of the prompt correlates with the cost incurred by the LLM provider and the time it takes for the LLM to process and return the results. ● Fine-tuning is the process of providing the model with prompt-response pairs that are similar to the information the user wants to obtain from the model. While this is a promising technique, it has limitations by its nature, and the model needs to be re-fine-tuned each time more data becomes available. ●RAG (Retrieval Augmented Generation) is a method that creates and maintains a database of necessary data, and when a prompt is received from a user, retrieves the data most relevant to the prompt from the database and creates a new prompt for LLM using the retrieved data. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] All three existing solutions rely on the use of human-readable text data, and by their very nature exclude technically structured (or semi-structured) data such as network transmissions in packet capture (PCAP) format.

[0008] Therefore, a method for classifying packet data is desired. [Means for solving the problem]

[0009] A method implemented by a computer according to an embodiment of the first aspect, Based on the input network packet data, generate a target flow summary containing information (in a semi-structured data format) about the (target) packet flow between two / a pair of (network) addresses / entities during the target period, (Using an encoder) to generate a target latent space representation corresponding to the target flow summary based on the target flow summary, The objective is to identify k reference latent space representations that are most similar to the target latent space representation from among multiple reference latent space representations, where the multiple reference latent space representations correspond to multiple reference flow summaries (based on reference network packet data) and multiple classifications, and k is an integer greater than 1. Using the first Large-Scale Language Model (LLM), a target flow report / portrait is generated in natural language format based on the target flow summary, and using the first LLM, k reference flow reports / portraits (corresponding to the k reference flow summaries, respectively) are generated in natural language format based on the k reference flow summaries, each corresponding to the k identified reference latent space representations. Using the second LLM, the (target) packet flow is classified as malicious or harmless based on the target flow report / portrait and k reference flow reports / portraits with corresponding classifications (the second LLM is fine-tuned / trained based on the training flow report / portrait to classify the training flow description as malicious or harmless). A method having the above is disclosed herein.

[0010] For example, please refer to the attached drawing. [Brief explanation of the drawing]

[0011] [Figure 1] These are diagrams useful for understanding the present invention. [Figure 2] These are diagrams useful for understanding the present invention. [Figure 3] These are diagrams useful for understanding the present invention. [Figure 4] These are diagrams useful for understanding the present invention. [Figure 5] This is a diagram for explaining steps. [Figure 6] This is a diagram for explaining steps. [Figure 7] This is a flowchart for explaining steps. [Figure 8] This is a flowchart for explaining steps. [Figure 9] This is a flowchart for explaining steps. [Figure 10] This is a flowchart for explaining steps. [Figure 11] This is a flowchart for explaining steps. [Figure 12] This is a flowchart for explaining steps. [Figure 13] This is a flowchart for explaining steps. [Figure 14] This is a diagram representing a flow summary. [Figure 15] This is a diagram representing a numerical vector. [Figure 16] This is a diagram representing an embedding. [Figure 17] This is a diagram representing a device. [Figure 18] This is a table. [Figure 19] This shows a graph. [Figure 20] This shows a graph [Figure 21] This shows a graph. [Figure 22] This shows a matrix.

Mode for Carrying Out the Invention

[0012] Brief Explanation of Technical Terms Used ● Anomaly detection: The process of identifying patterns or behaviors in a system that deviate from the normal, standard, or expected state and are inconsistent with the rest of the dataset. ● Anomaly Classification: The process of classifying anomalies into known classes. This helps in understanding the type of threat and the appropriate response. ● Root Cause Analysis (RCA): A problem-solving method used to identify the root cause of a security event. Its purpose is to identify the cause of a security incident and take preventative measures for the future. ● Generative AI: An artificial intelligence system that can generate text, images, videos, and other data using generative models. These models learn patterns and structures from input training data and generate new data with similar characteristics. ● Large-scale language models (LLMs): AI models with superior capabilities to perform general-purpose language generation, translation, summarization, question answering, and other natural language processing tasks. ● Fine-tuning: The process of further training a pre-trained AI model on a smaller, more targeted dataset. This is done to adjust the model's performance to suit a specific task or domain, rather than restarting the training process from scratch. ●Network traffic: The flow of data packets moving across a network at a specific point in time. ● PCAP file: A common format for saving packet captures. A PCAP file contains an exact copy of every byte of every packet observed on the network. ● Search Augmentation Generation (RAG): A method that improves the accuracy and reliability of generative AI models by using facts obtained from external sources. ● Latent Space: Embedding of item sets within a manifold. Similar items are placed close to each other. ● Autoencoder: An artificial neural network used to learn to efficiently code unlabeled data. It consists of two parts: an encoder that converts the input data into a low-dimensional representation, and a decoder that recreates the input data from the encoded representation.

[0013] Anomaly detection and root cause analysis (RCA) in cybersecurity are typically performed manually by experts. Because experts in this field are limited and the analysis work is often highly complex, there is a need to automate these processes. Furthermore, there is a growing demand for detailed descriptions of security incidents and for identifying those incidents.

[0014] Traditional machine learning algorithms have been used for anomaly classification of network data for over a decade (e.g., Shon, Taeshik, and Jongsub Moon, “A hybrid machine learning approach to network anomaly detection”, Information Sciences 177.18 (2007): 3799-3821, and Sommer, Robin, and Vern Paxson, “Outside the closed world: On using machine learning for network intrusion detection”, 2010 IEEE symposium on security and privacy, IEEE, 2010). The adoption of generative AI and LLMs is relatively recent, and there are limited peer-reviewed studies addressing this topic. This is due to the fact that while LLMs excel at understanding and processing human language, they often face challenges when applied to technical texts (Yang, Jingfeng, et al, “Harnessing the power of LLMs in practice: A survey on chatgpt and beyond”, ACM Transactions on Knowledge Discovery from Data 18.6 (2024): 1-32). The challenges are particularly pronounced in the context of network data intended for machine understanding rather than human use.

[0015] In the field of cybersecurity, various techniques are employed, including prompt engineering. For example, the authors of "Lanobert" (Lee, Yukyung, Jina Kim, and Pilsung Kang, "Lanobert: System log anomaly detection based on bert masked language model", Applied Soft Computing 146 (2023): 110689) fine-tuned an LLM by training it on harmless network logs. This enabled the model to accurately predict the next token (i.e., log) in harmless scenarios, while intentionally failing to predict anomalous tokens. This approach leverages the model's different performance in predicting harmless and anomalous log sequences, allowing the system to effectively detect anomalies. In their paper, “Transformer-based LLMs in Cybersecurity: An in-depth Study on Log Anomaly Detection and Conversational Defense Mechanisms”, 2023 IEEE International Conference on Big Data (BigData), IEEE, et al., Balasubramanian, Prasasthy, Justin Seby, Panos Kostakos, et al., presented a refined LLM and rule-based detection logic for the purpose of detecting anomalies in network log files.

[0016] There are no studies that have directly used network traffic (PCAP files) with LLM.

[0017] As disclosed herein, a finely tuned LLM functions as a classifier, while the embedding mechanism, database, and retrieval mechanism function as supporting functions for the LLM. This disclosure demonstrates that they can not only detect anomalous patterns within a network but also classify attack types by training the LLM on various attack scenarios. Furthermore, the LLM is used to provide a meaningful explanation of network events.

[0018] The goal of this disclosure is to develop new, more robust cybersecurity tools that leverage the strengths of LLM, the ubiquity of network packets, and their informational value. In its specific implementation, this framework primarily consists of two stages.

[0019] The first is a database containing network communications along with human-readable text context. This stage also includes the ability to embed these network communications along with their context and retrieve the communications stored in the database by comparing them with other network communications.

[0020] The second approach involves using a finely tuned LLM alongside other readily available LLMs to analyze unknown network communications by leveraging the capabilities of the LLMs and the context of similar communications. This analysis determines whether the queried communications are part of a malicious attack and performs a root cause analysis (RCA) to recommend ways to defend against similar attacks in the future.

[0021] Both stages help overcome LLM's weaknesses with technical and non-textual data. The first stage converts non-textual data into a human-readable format and stores it to aid in the analysis of new and unknown data. The second stage prepares LLM to accept network communication data, providing more robust analysis, particularly in the field of cybersecurity.

[0022] The implementation disclosed herein provides a rich set of features that can function as part of a support system to enhance attack detection, analysis, explanation, mitigation, and prevention in a more efficient and less labor-intensive manner. These features include mechanisms for representing, storing, and retrieving network traffic in a manner useful to LLMs, and training and guidance for LLMs to more accurately and thoroughly support cybersecurity tasks involving network communications.

[0023] The implementation disclosed herein ultimately provides robust cybersecurity analysis of network communication data, such as in the form of Internet Packets (PCAPs), including classifications of multiple different attacks. This is achieved by leveraging the analytical capabilities of LLM and overcoming its shortcomings in the following ways: ● Build a system like RAG that provides human-readable context for unknown internet packets. ● Fine-tune the LLM to convert network communication data into a human-readable and contextualized format, and return a classification of the data (whether it represents part of an attack, and if so, which attack).

[0024] This resolves the issues described herein when using LLM for cybersecurity purposes.

[0025] The specific implementation details are described below, with reference to Figures 5-12.

[0026] This implementation uses an existing labeled network traffic dataset. This dataset is divided into small chunks defined as 5-second intervals of network packet flow. This results in the following two processes: ●The flow is converted into a more human-readable format by LLM based on the extracted tabular format. Then, labels (harmless or malicious due to a specific attack) are assigned and assembled into complete query response tuples used for fine-tuning LLM. Features are extracted from these chunks and converted into numerical vectors. These numerical vectors serve as training data for an autoencoder, where the numerical vector representations are converted into latent space representations by a pre-trained encoding function. These latent space representations are stored in a vector store along with data extrapolated from labels.

[0027] When a new fragment of network traffic needs to be classified and analyzed, the fragment is also converted into a latent space representation, and the nearest similar chunk is retrieved from the vector store along with its extrapolated data. The queried network traffic fragment and the retrieved fragment are converted into a human-readable format by the LLM. These are combined to create the informational and question portions of the prompt. This new prompt is then passed to a fine-tuned LLM for classification.

[0028] The implementation improves upon classical anomaly classification and RCA approaches by enabling the use of LLMs while addressing shortcomings such as token limitations, readability, and type strictness of the data used for their pre-training sets.

[0029] This implementation can be divided into 12 mechanisms and 3 stages. All stages share several mechanisms. To explain this completely and accurately, we will first outline the mechanisms and then explain how the different stages utilize them.

[0030] mechanism: 1. Flow Description 2. Flow vectorization 3. Latent space generation 4. Vector Store Construction 5. Flow embedding 6. Flow text conversion 7. Fine-tuning the LLM 8.K Nearest Neighbor Flow Search 9. Prompt generation 10. Single-stage classification 11. Multi-stage classification 12. Report Generation

[0031] step: A. Database initialization B. Fine Tuning C. Network Transmission Diagnosis

[0032] mechanism The "mechanisms" and "stages" described herein can be considered as modules that perform specific operations / steps. It is understood that a given mechanism may be used in different ways at different stages. For example, a flow description mechanism may process labels (classifications) for each packet flow in stage A. database initialization, but not in stage C. network transmission diagnostics.

[0033] Flow description The initial input for the implementation and this specific mechanism is a PCAP file and, optionally, a label file. The PCAP file contains packets that have passed through a specific network element, preferably a router, in an organized, readable format. The label file contains information about the packets, especially if they are related to a particular network attack. This mechanism examines the PCAP file and groups the packets into flows. A flow may be defined in two ways, for example. If the packets are sent over the TCP protocol, the flow may be defined as a TCP connection (as described in RFC 793). Otherwise, if the packets are sent over UDP (User Datagram Protocol), the flow is described as all packets sharing a 4-tuple (source ID, destination ID, source port, destination port). That is, all packets with the same source ID, destination ID, source port, and destination port are grouped together in a flow. These flows are defined as bidirectional flows, where the source is defined as the sender (host or sender) and the destination is defined as the server. Flows may be grouped in two different ways, for example. One option is to define the flow using all the packets in a single PCAP file. Another option is to describe the flow using the interval between packets in the file (e.g., 5 seconds). The latter allows for the definition of more flows. After each flow is defined, a set of features is extracted from it and stored in a JSON file. The features are as follows: ●Features extracted directly: transmission protocol, source IP, destination IP, source port, destination port, start time, end time, application, service name, source host name, destination host name, internal source (boolean), internal destination (boolean), initial sequence number (TCP only), final sequence number (TCP only), initial acknowledgment number (TCP only), final acknowledgment number (TCP only). ●Statistical information: ○For transmitted data: Number of packets, amount of data (in bits), average packet size (in bits), standard deviation of packet size (in bits) ○For received data: Number of packets, amount of data (in bits), average packet size (in bits), standard deviation of packet size (in bits) ○Total time during which the flow occurred.

[0034] If a label file is also included, the labels are also stored in the JSON file. Generally, each packet can have a label (classification) corresponding to a specific attack, and a vector inherits the attacks present in the packets from which the vector originates.

[0035] In summary, the final output of this section is a JSON file that lists all flows, with each flow being a dictionary containing the characteristics described above.

[0036] Subsequently, a second JSON file is created. This file is similar to the original, but some features have been changed or removed to generalize the flow. Specifically, no particular IP address or time is specified. The source IP, address IP, start time, and end time are removed. Also, the hostname is changed to a generic host_i.

[0037] The output of this mechanism is sometimes called a flow summary for a given packet flow.

[0038] Figure 1 shows an example of generating a flow summary 12 ("network flow features") based on PCAP data 11.

[0039] Flow vectorization This mechanism receives a JSON file compiled by the flow description unit and converts each flow into a numerical vector. This is done using a known conversion algorithm based on various characteristics of the flow. ●Category Features: These are classifications into seven categories based on transmission protocol, application layer, source IP, destination IP, completion of transmitted data, completion of received data, and destination port based on usage frequency. These features are converted using One-Hot encoding. ●Numerical Features: These include the number of packets received and transmitted, the amount of data received and transmitted, the flow length (in nanoseconds), the average size of received and transmitted packets, and the standard deviation of the sizes of received and transmitted packets. These are normalized and then min-max scaled, for example, using either a [0,1] scale or a [-1,1] scale. ● High-dispersion features: These include only the destination port. These are first logarithmically scaled, and then min-max scaled, similar to numerical features. ● Remaining unlabeled features may be omitted as they may be considered irrelevant to understanding the flow's functionality.

[0040] After the conversion is complete, two files are generated. The first is a CSV file containing the new vectors. The second is another CSV file containing further vectors, where, for each further vector, five vectors are accumulated using the average of their feature values. This is done, for example, in a sliding window with a size of 5 and a step of 2. Each further vector inherits the attack classification of its five original vectors. For example, if one of the five vectors corresponds to an active eavesdropping attack, that further vector will be associated with an active eavesdropping attack (if another of the five original vectors relates to a different attack, the order of the vectors (ID / serial number) of the first one determines which attack the further vector inherits).

[0041] latent space generation In this mechanism, a numerical vector is passed through the autoencoder to train it. The loss function is the reconstruction loss of the numerical vector.

[0042] After the autoencoder is trained on numerical vectors, the flow embedding mechanism passes all numerical vectors to the embedding portion of the autoencoder to generate flow embeddings. The autoencoder weights are preserved. Such embeddings for a vector store are sometimes called reference latent space representations.

[0043] Vector store construction The search and retrieval mechanisms used are based on a vector store. These structures enable faster retrieval of vectors based on their distance from the queried vector. Each data point stored in the vector store includes the following: ● Embedding ●Document ●ID ● Metadata

[0044] The vector store used in this implementation works as follows: For each flow defined and described in the flow description, data points are stored in the vector store. Embeddings are embeddings generated by the latent space generation mechanism. A "document" is a representation of the flow that can be found in the JSON file output by the flow description. The ID is the location of the flow (can be thought of as a serial number). Metadata is all data that can be extracted from the flow's labels, for example, data present within labels. In this implementation, metadata includes at least classification (e.g., attack).

[0045] In general, a vector store can be implemented using any suitable storage technique. For example, it is not necessary to store the relevant information as data points containing the fields mentioned above, as long as the relevant information can be retrieved for a given reference latent space representation.

[0046] Network packet data may be used from the same or similar system / network from which the "unknown flow" or "target packet flow" was acquired, or from a different / unrelated system / network.

[0047] Flow embedding This mechanism receives a numerically vectorized flow and uses the encoder portion of a (trained) autoencoder to generate an embedding of the flow. Such an embedding may be called a latent space representation. For the generation of a vector store, such an embedding may be called a reference embedding or reference latent space representation and is based on reference PCAP data. For "unknown flows," which may be called target packet flows based on target PCAP data, the embedding may be called a target embedding or target latent space representation.

[0048] Flow text conversion In this mechanism, the output of the flow description mechanism is passed through an LLM (e.g., the first LLM) to generate a more human-readable format. The output is a paragraph reflecting the data in the flow summary (e.g., tabular format), but it allows the model to effectively analyze and classify unknown network flows. This output may be called a (reference / target) flow report and is provided in natural language format.

[0049] Figure 2 shows an example of generating a flow report 13 based on a flow summary 12.

[0050] LLM fine tuning To fine-tune the relevant LLMs (e.g., the second LLM described below, and optionally, the third LLM and at least one further LLM), an example LLM is provided that includes three parts: a system message, a query, and an answer. The system message is similar to the following:

number

[0051] It is understood that a finely tuned multi-stage classification LLM (described later) can have several variations, such as requiring only the classification of malicious and harmless elements, determining which category of attack a query originates from, or which attack within that category best represents the flow.

[0052] The k mentioned is the number of nearest flows used in the network transmission diagnostics phase (described below). The query includes "unknown flows" for training, along with k "nearest flows" that simulate the prompts provided to the fine-tuned LLM in the actual network transmission diagnostics phase. Each "nearest flow" includes a human-readable text of the flow provided from flow textification and its classification determined in the flow description process (e.g., extracted from a label file). The "unknown flows" for training are also provided in the form of human-readable text generated by the relevant LLM. Each "unknown flow" for training is associated with a flow classification (as ground truth data). The weights of the LLM are adjusted based on the predictions made by the LLM in order to fine-tune the LLM.

[0053] K-Nearest Neighbor Flow Search The vector store generated by the vector store generation mechanism has the ability to retrieve and return any number of data points (or can be thought of as another mechanism that uses a vector store). The returned data points are those that have the closest embedding (e.g., in a geometric sense in latent space) to the queried embedding. This implementation uses two functions, L2 similarity and cosine similarity, to measure the distance between embeddings. The returned data points are returned along with all the features they contain (or at least their reference latent space representation is returned, with other relevant information such as classification retrieved from the vector store or elsewhere).

[0054] Prompt generation This mechanism receives human-readable text versions of the queried flow and the k nearest retrieved flows from the flow textening mechanism, and for the latter, also receives their classes / classifications. The prompt would look like this:

number

[0055] This is a sufficient prompt, as the intended LLM is fine-tuned to recognize such prompts and return classifications of the queried flow.

[0056] Single-stage classification The prompts generated by the prompt generation mechanism are used in a finely tuned LLM (second LLM), and the returned responses, along with the specific attacks detected, are either "harmless" or "malicious."

[0057] Figure 3 illustrates a single-stage classification implementation in which a packet flow is classified as either one of several attacks (Address Resolution Protocol Man in the Middle (ARP MitM), Active Wiretap, Simple Service Discovery Protocol (SSDP), and SYN Denial of Service (DoS)) or harmless (Benign).

[0058] Multi-stage classification This stage (as an alternative to single-stage classification) involves five different fine-tuned models (LLMs), each with different system messages to achieve different objectives. These models include: ● One model classifies queried flows into "harmless" and "malicious" categories. ● One model categorizes "malicious" queried flows into several attack categories, such as denial-of-service (DoS) attacks, man-in-the-middle (MitM) attacks, and reconnaissance attacks. ● Three models that receive "malicious" query flows categorized into specific categories and classify them into specific attacks.

[0059] The prompts generated by the prompt generation mechanism are input to these models in the following classification order: 1. A model for classifying "harmless" and "malicious" things. 2. Models for classifying attacks into categories 3. A specific model that classifies queries into attacks within defined categories.

[0060] Generally, having five LLMs is not mandatory, and other implementations may, more generally, have a second, third, and at least one additional LLM, as will be discussed later.

[0061] Figure 4 illustrates a multi-stage classification implementation in which a packet flow is classified as either harmless or malicious in the first stage (by the second LLM), and if it is malicious, it is classified as corresponding to one of several attack types (including man-in-the-middle (MitM) and denial-of-service (DoS)) in the third stage (by the third LLM described later), and in the third stage, depending on the determined attack type, it is classified as corresponding to one of several attacks (ARP MitM or active eavesdropping in the case of a MitM attack type, or SSDR Flood or SYN DoS in the case of a DoS attack type) in the third stage (by at least one further LLM described later).

[0062] Report generation The response from one of the classification mechanisms (single-stage or multi-stage) is fed into the LLM (the sixth LLM, sometimes called the "analysis report LLM") along with a human-readable text transcription of the flow, creating a complete report describing the attack, its origin and steps, and network-implementable mitigation and preventative measures. This complete report, sometimes called the analysis report, is distinct from the "flow report" generated by the flow transcription mechanism.

[0063] step Database initialization This stage involves receiving labeled network data and establishing a mechanism that will later be used to generate reports analyzing the network data. This stage is illustrated in Figure 5 and includes steps S51 to S55.

[0064] The pipeline is as follows: The S51.PCAP file and the label file representing the labeled network data are inserted into the flow description mechanism and processed according to the flow description mechanism. S52. The output general-purpose flow features are then inserted into the flow vectorization mechanism and processed according to the flow vectorization mechanism. S53. The output vector is then used to train the autoencoder in the latent space generation mechanism. S54. The same vector (or other vectors generated based on other PCAP files) is then passed through the encoder portion of the autoencoder in the flow embedding mechanism. S55. The generated embeddings are used in the vector store generation mechanism to construct the vector store (along with the flow summary).

[0065] Fine tuning This stage involves receiving labeled network data and fine-tuning all fine-tuned LLMs used in the network transmission diagnostics stage.

[0066] The pipeline is as follows: 1. The PCAP file and a label file representing the labeled network data are inserted into the flow description mechanism. 2. The generic labeled flow is then converted into human-readable paragraph-like text using a flow text conversion mechanism. 3. Each of the six fine-tuned LLMs (one for single-stage classification and five for multi-stage classification) is fine-tuned using a human-readable flow specified in the LLM fine-tuning mechanism. Alternatively, only the single-stage classification LLM or the five multi-stage classification LLMs may be fine-tuned.

[0067] The two stages described above can be considered as single stages, for example, "preparatory" stages.

[0068] Network Transmission Diagnostics This stage involves receiving unlabeled network transmission data, analyzing it to determine whether it is harmless or malicious, and, in the latter case, taking measures to mitigate it. This stage is illustrated in Figure 6 and includes steps S61 to S65.

[0069] The pipeline is as follows: The S61.PCAP file is input into the flow description mechanism and processed according to the flow description mechanism. S62. The general-purpose flow characteristic output is then inserted into the flow vectorization mechanism and processed according to the flow vectorization mechanism. S63. The output vector is then passed through the encoding portion of the autoencoder in the flow embedding mechanism. S64. Each embedding is passed to the vector store to retrieve its nearest data point using the K nearest neighbor acquisition mechanism. S65. Non-generic flow summaries are paired with acquired data points to form prompts in the prompt generation mechanism. The generated prompts are then inserted into a single-stage or multi-stage classification mechanism to generate a classification of the input flow. The report generation mechanism is used to generate a final, robust analysis report on the attack.

[0070] The network transmission diagnostics stage may further include taking some action based on the classification of a malicious (or specific attack), for example, instructing at least one entity involved in the packet flow to restrict communication. Additionally or alternatively, the analysis report may be sent to at least one entity and / or the network administrator.

[0071] In the implementation described above, the target PCAP file will result in multiple target packet flows, and therefore multiple target latent space embeddings. Here, "multiple" is not mandatory; only one target packet flow is conceivable and may ultimately be classified.

[0072] Figure 7 is a flowchart representing the database initialization phase, including steps S71 to S75. Figure 7 illustrates this phase using so-called mechanisms. Step S71 includes 1. using the flow description mechanism, step S72 includes 2. using the flow vectorization mechanism, step S73 includes 3. using the latent space generation mechanism, step S74 includes 5. using the flow embedding mechanism, and step S75 includes 4. using the vector store generation mechanism. The above description of this phase and related mechanisms is applicable here.

[0073] Figure 10 is a further flowchart representing the database initialization stage, including steps S101 to S105. In step S101, the PCAP and labels are received, feature extraction is performed, and a flow summary is generated according to the 1. flow description mechanism. In step S102, a vectorized flow is generated according to the 2. flow vectorization mechanism. In step S103, the AE is trained according to the 3. latent space generation mechanism. In step S104, encoding is performed to generate an embedded flow according to the 5. flow embedding mechanism. In step S105, the flow summary and embedded flow are saved as vector store data points according to the 4. vector store generation mechanism.

[0074] Figure 8 is a flowchart representing the LLM fine-tuning stage, including steps S81 to S83. Figure 8 illustrates this stage using a so-called mechanism. Step S81 includes using 1. the flow description mechanism, step S82 includes using 6. the flow text conversion mechanism, and step S83 includes using 7. the LLM fine-tuning mechanism.

[0075] Figure 11 shows the LLM fine-tuning stages, which are further flowcharts including steps S111-S113. In step S111, PCAPs and labels are received, and a flow summary is generated for them according to 1. Flow Description Mechanism. In step S112, a flow report (human-readable flow) is generated according to 6. Flow Textification. In step S113, fine-tuning of single-stage and / or multi-stage classification is performed according to 7. LLM fine-tuning (as well as according to 11. Multi-stage classification and 10. Single-stage classification).

[0076] Figure 9 is a flowchart representing the network transmission diagnostic stage, including steps S91 to S98. Figure 9 illustrates this stage using so-called mechanisms. Step S91 includes 1. using the flow description mechanism, S92 includes 2. using the flow vectorization mechanism, S93 includes 5. using the flow embedding mechanism, S94 includes 8. using the K nearest neighbor acquisition mechanism, step S95 includes 6. using the flow text mechanism, step S96 includes 9. using the prompt generation mechanism, step S97 includes 10 / 11. using the single / multi-stage classification mechanism, and step S98 includes 12. using the report generation mechanism.

[0077] Figure 12 is a further flowchart representing the network transmission diagnostic stage, including steps S121 to S127, where a PCAP is received (corresponding to an unknown flow; in this case, the data does not include the attack type, i.e., classification), in step S121, 1. a flow summary is generated based on the PCAP according to the flow description mechanism (general flow summary and specific flow summary; the specific flow summary includes address names, etc.), in step S122, 2. a vector (vectorized flow) is generated based on the (general) flow summary according to the flow vectorization mechanism, and in step S123, 5. flow embedding The vector is encoded to generate embedded flows according to the mechanism, in step S124 the k nearest embedded flows (data points) are retrieved from the vector store according to the 8. k nearest neighbor acquisition mechanism, in step S125 a prompt is generated based on the k nearest summaries and the (specific) summary of the unknown flow according to the 9. prompt generation mechanism, in step S126 the unknown flow is classified according to the 10. single-stage classification mechanism or the 11. multi-stage classification mechanism, and in step S127 a report (analysis report) is generated based on the classification according to the 12. report generation mechanism.

[0078] Figure 13 is a flowchart showing the method including steps S10 to S50.

[0079] Step S10 includes generating a target flow summary. That is, step S10 includes generating a target flow summary, based on the input network packet data, which includes information (in a semi-structured data format) about the (target) packet flow between two / a pair of (network) addresses / entities during the target period.

[0080] Step S20 includes generating a target latent space representation. That is, step S20 includes generating a target latent space representation corresponding to the target flow summary based on the target flow summary (using an encoder).

[0081] Step S30 includes identifying the k most similar reference latent space representations. That is, step S30 includes identifying the k reference latent space representations from among a plurality of reference latent space representations that are most similar to the target latent space representation, where the plurality of reference latent space representations correspond to a plurality of reference flow summaries (based on reference network packet data) and a plurality of classifications, and k is an integer greater than 1.

[0082] Step S40 includes generating a target flow report and k reference flow reports. Specifically, step S40 includes generating a target flow report / portrait in natural language format based on the target flow summary using a first large language model (LLM), and generating k reference flow reports / portraits (corresponding to each of the k reference flow summaries) in natural language format based on the k reference flow summaries (each corresponding to each of the k identified reference latent space representations) using the first LLM.

[0083] Step S50 includes generating a classification. Specifically, step S50 includes using the second LLM to classify the (target) packet flow as malicious or harmless based on the target flow report / portrait and k reference flow reports / portraits with corresponding classifications (the second LLM is fine-tuned / trained based on the training flow report / portrait to classify the training flow description as malicious or harmless).

[0084] Step S10 is considered in some respects to 1. the flow description mechanism, and the corresponding explanation may apply to step S10, and vice versa. Step S20 is considered in some respects to 2. the flow vectorization mechanism and 5. the flow embedding mechanism, and the corresponding explanation may apply to step S20, and vice versa. Step S30 is considered in some respects to 8. the k nearest neighbor acquisition mechanism, and the corresponding explanation may apply to step S30, and vice versa. Step S40 is considered in some respects to 6. the flow textification mechanism, and the corresponding explanation may apply to step S40, and vice versa. Step S50 is considered in some respects to 10. the single-stage classification mechanism and / or 11. the multi-stage classification mechanism, and the corresponding explanation may apply to step S50, and vice versa.

[0085] According to Figure 13, a method implemented by a computer involves generating a target flow summary (in a semi-structured data format) containing information about the (target) packet flow between two / a pair of (network) addresses / entities during a target period, based on input network packet data; generating a target latent space representation corresponding to the target flow summary (using an encoder) based on the target flow summary; and identifying k reference latent space representations from among multiple reference latent space representations that are most similar to the target latent space representation, where the multiple reference latent space representations correspond to multiple reference flow summaries (based on reference network packet data) and multiple classifications, and k is an integer greater than 1; and a first large-scale language model (LLM). A method is disclosed that includes generating a target flow report / portrait in natural language format based on a target flow summary using a first LLM, generating k reference flow reports / portraits in natural language format based on k reference flow summaries corresponding to each of the k identified reference latent space representations using a first LLM, and classifying the (target) packet flow as malicious or harmless using a second LLM based on the target flow report / portrait and the k reference flow reports / portraits with corresponding classifications (the second LLM is fine-tuned / trained based on the training flow report / portrait to classify the training flow description as malicious or harmless).

[0086] The input network packet data may include packet capture (PCAP) data.

[0087] The target flow summary may contain information from JavaScript® Object Notation (JSON) files.

[0088] The target flow summary may include statistical information (and information indicating two addresses) about the packets received and transmitted in the packet flow.

[0089] Each reference flow summary may contain statistical information (and information indicating two addresses) about the packets received and transmitted in the corresponding reference packet flow.

[0090] The target flow summary (and each reference flow summary) may include, as information, the number of packets received, the number of packets transmitted, the size of the data received, the size of the data transmitted, the average received packet size, the average transmitted packet size, the standard deviation of the received packet size, the standard deviation of the transmitted packet size, the protocol, the source IP address, the destination IP address, the source port, the destination port, the start time (of the packet flow), the end time (of the packet flow), the total time the packet flow occurred, the application layer, the service name, the source host name, the destination host name, the internal source, the internal destination, the initial sequence number, the final sequence number, the initial acknowledgment number, and the final acknowledgment number.

[0091] Generating a target flow summary may involve defining one of two addresses as the source and the other as the destination.

[0092] Generating a target latent space representation may involve generating a target numerical vector based on a target flow summary, and then generating a target latent space representation based on the target numerical vector.

[0093] Generating a target numerical vector may involve converting information within the target flow summary into numerical format by using one-hot encoding of categorical information and normalizing the numerical information.

[0094] Generating a target latent space representation may involve using a trained encoder in an autoencoder training process that includes generating a trained latent space representation based on a training numerical vector.

[0095] Generating a target latent space representation may involve using a trained encoder in an autoencoder training process that trains an autoencoder including an encoder and a decoder, the autoencoder training process including using the encoder to generate a trained latent space representation based on a training numerical vector, to generate a reconstructed numerical vector based on the trained latent space representation, and to adjust the autoencoder weights based on the reconstruction loss.

[0096] A computer-based method may include generating a training latent space representation by generating multiple training flow summaries based on training network packet data in the training representation generation process, each training flow summary containing information (in a semi-structured data format) about training packets between two addresses during a training period (the training flow summary corresponds to multiple address pairs and multiple training periods), and generating multiple training latent space representations (corresponding to the training flow summaries) based on the training flow summaries (using an encoder).

[0097] Generating a training latent space representation may involve generating training numerical vectors based on a training flow summary and generating a training latent space representation based on those training numerical vectors.

[0098] Generating training numerical vectors may involve converting information within the training flow summary into numerical format by using one-hot encoding of categorical information and normalizing the numerical information.

[0099] Generating a target latent space representation may involve using a trained encoder in an autoencoder training process that includes generating a trained latent space representation based on a training numerical vector.

[0100] The method performed by the computer may include generating further training numerical vectors by averaging each of the multiple training numerical vectors (in a sliding window manner), and generating the training latent space representation may further include generating the training latent space representation based on the further training numerical vectors.

[0101] The computer-based method may include performing an autoencoder training process before generating the target latent space representation.

[0102] The computer-based method may include repeating the autoencoder training process until the reconstruction loss converges.

[0103] Each latent space representation may be a representation of the encoder in its latent space.

[0104] Identifying the k reference latent space representations that are most similar to the target latent space representation may involve determining the L2 distance and / or cosine similarity between the target latent space representation and each reference latent space representation.

[0105] Classifying a packet flow as malicious may, by the Second LLM, involve classifying the packet flow as one of several attacks.

[0106] The methods performed by the computer may include, if a classification of malicious is generated for a packet flow, using the third LLM to classify the packet flow as one of several attacks.

[0107] Multiple attacks may include Address Resolution Protocol Man-in-the-Middle (ARP MitM), active eavesdropping, Simple Service Discovery Protocol (SSDP), SYN denial of service (DoS), fuzzing, and operating system (OS) scanning.

[0108] The method performed by the computer may include, if a classification of malicious is generated for a packet flow, using a third LLM to classify the packet flow as one of several attack types.

[0109] Multiple attack types may include man-in-the-middle (MitM) attacks, denial-of-service (DoS) attacks, and reconnaissance attacks.

[0110] The method performed by the computer may include classifying the packet flow as one of several attacks corresponding to that attack type, using at least one additional LLM depending on the classification of the attack type.

[0111] At least one additional LLM may include an LLM corresponding to each attack type.

[0112] The method performed by the computer may include, if a classification of malicious is generated for a packet flow, using a third LLM to classify the packet flow as one of several attack types, and then using at least one further LLM, depending on the classification of the attack type, to classify the packet flow as one of several attacks corresponding to that attack type.

[0113] The second LLM (and optionally, the third LLM) (and even more optionally, at least one further LLM) may be trained according to a fine-tuning process that includes, for example, the LLM in question or each LLM, using the LLM to generate classifications in a training flow report and adjusting the weights of the LLM based on the difference between the generated classifications and the ground truth classifications corresponding to the training flow report.

[0114] The computer-based method may include a fine-tuning process before generating the classification.

[0115] A computer-generated method may include, in the representation generation process, generating multiple reference latent space representations by generating multiple reference flow summaries based on reference network packet data, each reference flow summary containing information (in a semi-structured data format) about reference flow packets between two addresses during a reference period (the reference flow summary corresponds to multiple address pairs and multiple reference periods), and generating multiple reference latent space representations (corresponding to the reference flow summaries) based on the reference flow summaries.

[0116] Each reference flow summary may contain information about the reference packet flow in question, according to the target flow summary.

[0117] Generating a reference latent space representation may involve generating a reference numerical vector based on a reference flow summary, and then generating a reference latent space representation based on the reference numerical vector.

[0118] Generating a reference numerical vector may involve converting information within the training flow summary into numerical format by using one-hot encoding of categorical information and normalizing the numerical information.

[0119] The method performed by the computer may include generating further reference numerical vectors by averaging each of the aforementioned multiple reference numerical vectors (in a sliding window manner), and generating a reference latent space representation may further include generating a reference latent space representation based on the further reference numerical vectors.

[0120] Each packet in the reference network packet data may be associated with a classification, and the classification corresponding to a given reference latent space representation may depend on the classification of the packets on which that reference latent space representation is based.

[0121] The reference period can be defined using a sliding window method.

[0122] The method implemented by the computer may include, if a packet flow is identified as malicious, instructing at least one entity associated with two addresses to take action in response to that malicious packet flow.

[0123] The instruction may be to restrict communications.

[0124] The method implemented by the computer may include notifying at least one entity associated with two network addresses that a packet flow is malicious, if the packet flow is identified as malicious.

[0125] The computer-based method may include generating an analysis report containing information about the classification of packet flows.

[0126] The report may include recommendations to mitigate malicious packets and / or attacks if the flow is identified as malicious.

[0127] In this application, when executed on a computer, the computer generates a target flow summary (in a semi-structured data format) containing information about the (target) packet flow between two / a pair of (network) addresses / entities during the target period, based on input network packet data; generates a target latent space representation corresponding to the target flow summary (using an encoder); and identifies k reference latent space representations from among multiple reference latent space representations that are most similar to the target latent space representation, where the multiple reference latent space representations correspond to multiple reference flow summaries (based on reference network packet data) and multiple classifications, and k is an integer greater than 1; and uses a first large-scale language model (LLM) to... Also disclosed is a computer program that performs a method including generating a target flow report / portrait in natural language format based on a get flow summary, generating k reference flow reports / portraits in natural language format based on k reference flow summaries corresponding to each of the k identified reference latent space representations using a first LLM, and classifying the (target) packet flow as malicious or harmless based on the target flow report / portrait and the k reference flow reports / portraits with corresponding classifications using a second LLM (the second LLM is fine-tuned / trained based on the training flow report / portrait to classify the training flow description as malicious or harmless).

[0128] The present invention provides a memory and a processor connected to the memory, wherein the processor generates a target flow summary (in a semi-structured data format) containing information about the (target) packet flow between two / a pair of (network) addresses / entities during a target period, based on input network packet data; generates a target latent space representation corresponding to the target flow summary (using an encoder) based on the target flow summary; and identifies k reference latent space representations from among multiple reference latent space representations that are most similar to the target latent space representation, where the multiple reference latent space representations correspond to multiple reference flow summaries (based on reference network packet data) and multiple classifications, and k is an integer greater than 1; and a first large-scale language model (LLM) is provided. Information processing device is also disclosed, which is configured to: generate a target flow report / portrait in natural language format based on a target flow summary using a first LLM, generate k reference flow reports / portraits in natural language format based on k reference flow summaries corresponding to each of the k identified reference latent space representations, and classify the (target) packet flow as malicious or harmless based on the target flow report / portrait and the k reference flow reports / portraits with corresponding classifications using a second LLM (the second LLM is fine-tuned / trained based on the training flow report / portrait to classify the training flow description as malicious or harmless).

[0129] The characteristics of the method may also be applied to the aspects of computer programs and information processing devices, and vice versa.

[0130] The implementations described above with respect to Figures 5-12 can be understood as specific implementations of the method shown in Figure 13.

[0131] The example / test is briefly described according to the implementation described above, with reference to Figures 5-12. In this example, only the network transmission diagnostic stage is shown. This test was performed on the Kitsune dataset (Shon, Taeshik, and Jongsub Moon, “A hybrid machine learning approach to network anomaly detection”, Information Sciences 177.18 (2007): 3799-3821), specifically on its SSDP Flood PCAP and Label files. The flow description mechanism generated five flow summaries, shown in Figure 14. The vectorization mechanism, based on the flow mechanism, generated numerical vectors, shown in Figure 15. Furthermore, the latent space representation based on these numerical vectors is shown in Figure 16.

[0132] This embedding (latent space representation) is then used to query the vector store to find the five closest referential flows. For brevity, the referential flows are shown, but each is representative of an SSDP Flood attack. A prompt was generated and forwarded (via the Azure API) to a single-stage fine-tuned LLM model in ChatGPT 3.5. For the classification of the five unknown flows, the answer "malicious - SSDP Flood" was returned. This is correct.

[0133] The following describes some decisions related to existing methods for anomaly detection and root cause analysis (RCA) in cybersecurity.

[0134] Current cybersecurity methods for anomaly detection and root cause analysis (RCA) heavily rely on manual work by experts, which is time-consuming and labor-intensive. This reliance stems from problems such as the limited number of experts and the complexity of the work, highlighting the need for automation. Furthermore, existing methods often fail to provide the detailed, human-readable explanations of security incidents that are essential for effective incident response.

[0135] While the use of LLM can help address these issues, LLM in cybersecurity has the following limitations: Current solutions operate only on natural language text. This applies to both LLM and RAG solutions. LLM is pre-trained on a large webpage corpus. This corpus also includes network traffic data, but the amount is limited and sporadic. Therefore, "out-of-the-box" LLMs like ChatGPT have significant limitations in analyzing and understanding network traffic. RAG, which can be used to compensate for these knowledge limitations of LLMs, also focuses entirely on natural language text data, but it faces the same limitations. Less-than-ideal improvements include: 1. Pre-training LLM on a large network traffic data corpus: While this yields desirable results, it is too costly for many companies and requires a large corpus that is not freely available. 2. Converting network traffic to text: Doing so directly results in a loss of nuance in the network data. Converting network traffic to stories significantly increases the number of words required, making it difficult for LLMs to understand large amounts of network traffic data. 3. Using RAG for text-based network traffic: Comparing directly text-based network traffic data points using existing embedding methods fails to compare the nuances of network data, and this search targets irrelevant network traffic data points.

[0136] The method logic disclosed herein solves these problems by first transforming a network device into a latent space that takes into account the nuances of network traffic data, and then using that, along with a textualized abbreviated representation of the network traffic, as a “retrieved document” for the LLM. This reduces the amount of tokens used in prompts, along with a finely tuned LLM that understands prompt formatting and expected thought patterns, enabling the LLM to understand natural language and provide complex and robust answers, including classification, mitigation, and future prevention.

[0137] The method logic disclosed herein, along with LLM, unlocks the use of network transmission data and can be used for general network maintenance and network design that leverages the learning capabilities of LLM.

[0138] This invention discloses a mechanism for converting network transmission data into text that can be compared with other network transmission data, thereby generating a classification system that not only provides classification but also leverages the capabilities of LLM to provide security support.

[0139] The disclosed advantages of methodology are as follows: ● Automated Anomaly Detection and Classification: This innovative approach brings accuracy and clarity to automated procedures for detecting and classifying network anomalies, reducing the need for expert intervention. Fine-tuning LLM for network security allows for efficient processing of large amounts of data, helping to reduce human factors in managing these tasks. ● Detailed description of security incidents: The proposed methodology provides detailed descriptions of security incidents that are easy for humans to understand, helping network administrators analyze causes more efficiently, thereby reducing working time and improving network performance. ● Improved efficiency and reduced costs: This methodology reduces not only the time spent on manual analysis and intervention, but also the costs associated with the intervention. ● Improved accuracy and reliability: Advanced LLMs trained on specific network data improve the accuracy and reliability of network anomaly detection and attack type classification. ● Scalability and adaptability: System automation allows the system to expand or contract as needed based on available data on the network, making it adaptable to various network structures and sizes. ● Context-aware analysis: Using method logic such as k-closest search, LLM first obtains the necessary context from known network flows. This allows LLM to identify patterns and similarities, leading to more accurate results. ● PCAP file analysis: By grouping network traffic packets into flows, anomalies can be directly detected and classified from PCAP files, which are commonly used for capturing network traffic.

[0140] Figure 17 is a block diagram of an information processing device 10 or computing device 10, such as a data storage server, which may be used to embody the present invention, implement some or all of the operations of a method for embodying the present invention, and perform some or all of the tasks of the apparatus of the embodiment. The computing device 10 may be used to implement any of the above method steps, such as steps S10 to S50 and / or any of the operations of the above mechanisms / modules, for example, any of mechanisms 1 to 12 and / or any of steps A to C.

[0141] The computing device 10 includes a processor 993 and memory 994. Optionally, the computing device also includes a network interface 997 for communication with other such computing devices, for example, with other computing devices in embodiments of the invention. Optionally, the computing device further includes one or more input mechanisms 996, such as a keyboard and a mouse, and one or more display units 995, such as a monitor. These elements may assist user interaction. The components are connectable to each other via a bus 992.

[0142] Memory 994 may include computer-readable media, and this term can refer to a single or multiple media configured to carry computer-executable instructions (e.g., a centralized or distributed database and / or associated caches and servers). Computer-executable instructions may include, for example, instructions and data that are accessible to a computer (e.g., one or more processors) and cause the computer to perform one or more functions or operations. For example, computer-executable instructions may include instructions for implementing the methods disclosed herein or any of the method steps disclosed herein, e.g., steps S10 to S50 and / or any of the mechanisms / modules described above, e.g., any of mechanisms 1 to 12 and / or any of the operations of steps A to C. Thus, the term “computer-executable storage medium” may also include any medium that can store, encode or carry a set of instructions for machine execution and cause a machine to perform any one of the method steps of this disclosure. Accordingly, the term “computer-readable storage medium” may be understood to include, but is not limited to, solid-state memory, optical memory, and magnetic media. Such computer-readable media may include, but are not limited to, non-transient computer-readable storage media such as random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk type read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, and flash memory devices (e.g., solid-state memory devices).

[0143] The processor 993 is configured to control the computing device and perform processing operations, for example, by executing computer program code stored in memory 994 to implement any of the method steps or mechanism / module operations described herein. Memory 994 stores data read and written by the processor 993, and may store the above-described packet data and / or flow summaries and / or numerical vectors and / or latent space representations and / or flow reports and / or vector stores and / or LLM and / or LLM / encoder weights and / or information on attacks and attack types and / or reference data and / or training data and / or analysis reports and / or other data, and / or programs for performing any of the above-described method steps / operations, for example, steps S10 to S50 and / or any of the above-described mechanisms / modules, for example, any of mechanisms 1 to 12 and / or any of the operations of stages A to C. As referred to herein, the processor may include one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. The processor may include a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing another instruction set or a combination of instruction sets. The processor may also include one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and network processors. In one or more embodiments, the processor is configured to execute instructions for performing the operations discussed herein. Processor 993 can be considered to include any of the modules described above. Any operations described as being implemented by a module may be performed by a computer, for example, by a processor.

[0144] The display unit 995 may display representations of data stored by the computing device, such as packet data and / or flow summaries and / or numerical vectors and / or latent space representations and / or flow reports and / or vector stores and / or information about attacks and attack types and / or reference data and / or training data and / or analysis reports and / or GUI windows and / or interactive representations that enable the user to interact with the device 10, for example by drag-and-drop or selection interactions, and / or any other outputs as described above. It may also display cursors and dialog boxes and screens that enable interaction between the user and programs and data stored in the computing device. The input mechanism 996 may enable the user to input data and instructions into the computing device, such as enabling the user to input any of the user inputs described above.

[0145] The network interface (network I / F) 997 may be connected to a network such as the Internet, and can connect to other such computing devices via the network. The network I / F 997 can control data input from or output to other devices via the network.

[0146] Other peripherals such as microphones, speakers, printers, power supply units, fans, cases, scanners, and trackballs may also be included in the computing device.

[0147] The operation of the method embodying the present invention may be performed by a computing device / apparatus 10 as shown in Figure 17. Such a computing device does not need to have all the components shown in Figure 17, but may consist of a subset of these components. For example, apparatus 10 may have a processor 993 and memory 994 connected to the processor 993. Alternatively, apparatus 10 may have a processor 993, memory 994 connected to the processor 993, and a display 995. The method embodying the present invention may be performed by a single computing device that communicates with one or more data storage servers over a network. The computing device may be the data storage itself that stores at least a portion of the data.

[0148] The method for realizing the present invention may be carried out by a plurality of computing devices operating in coordination with each other. One or more of the plurality of computing devices may be data storage servers that store at least a portion of the data.

[0149] The present invention may be implemented in digital electronic circuits, or in computer hardware, firmware, software, or a combination thereof. The present invention may also be implemented as a computer program or computer program product, that is, a computer program tangibly embodied in a non-temporary information carrier, for example in a machine-readable storage medium, or in a propagating signal for execution by one or more hardware modules or for controlling the operation of one or more hardware modules.

[0150] Computer programs may take the form of standalone programs, computer program portions, or more than one computer program, and may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a data processing environment. Computer programs may be deployed to run in one module, or in multiple modules located in one or more locations and interconnected by a communication network.

[0151] The method steps of the present invention, for example, steps S10 to S50 and / or any of the mechanisms / modules described above, for example, any of mechanisms 1 to 12 and / or any of the operations of steps A to C, may be performed by one or more programmable processors executing a computer program to perform the functions of the present invention by acting on input data to generate an output. The apparatus of the present invention may be implemented as programmed hardware or as a special-purpose logic circuit including an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0152] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, as well as one or more processors of any type of digital computer. Generally, a processor receives instructions and data from read-only memory, random-access memory, or both. An essential element of a computer is a processor for executing instructions, coupled with one or more memory devices for storing instructions and data.

[0153] The above embodiments of the present invention may, advantageously, be used independently of other embodiments or in any feasible combination with one or more other embodiments.

[0154] Implementation and experimental results This section describes the implementation and experimental results.

[0155] 1. General Architecture The main objective of the implementation / design is to create a RAG-like mechanism that receives network communications as packets, groups them as flows between two devices, and returns information about similar flows that are useful for classification by LLM. This is achieved in two stages. The first stage is the preparation stage, in which a vector store database is generated. To generate this database, a set of packets is processed to create a latent space representation. This representation is then paired with metadata and stored in the database. The second stage is the inference phase, in which an unknown set of packets is classified. In this stage, the retrieval mechanism in the database is used to retrieve the K flows that are closest (in the latent space) to the target flow. The design includes a flow description unit, a flow vectorization unit, a latent space generation unit, a flow embedding unit, and a flow retrieval unit.

[0156] 1.2 Flow Description Section The system's inputs are a packet capture (PCAP) file and a label file, which is used only during the initial stage. The PCAP file contains packets processed by network appliances such as routers. The label file records whether packets are harmless or part of an attack and is used to create the context for the vector store in subsection 2.6. This mechanism examines the PCAP file and groups packets into flows. Flows are defined in two ways. If the packets are TCP, the flow is defined as a TCP connection. Otherwise, the flow is described as all packets sharing a 4-tuple: source IP, destination IP, source port, destination port. These flows are defined as bidirectional flows, with the source defined as the sender (host or sender) and the destination defined as the receiver (or server). Non-TCP flows are grouped in two different ways. The first option is to define the flow using all packets in a single PCAP file. The second is to describe the flow using 5-second intervals between packets in the file. The latter defines more flows. Once each flow is defined, a set of features is extracted from it and saved in a JSON file. The features are as shown in Figure 18, which is a table summarizing the features. Directional features are acquired twice: the first time from packets sent from the sender to the receiver, and the second time from the reverse direction.

[0157] If a label file is included, the labels are also stored in the JSON file, and each flow is defined as malicious if even one of its packets is malicious (flows containing packets from different attacks are not included). Finally, to avoid overfitting to specific characteristics, several features are generalized. Source and destination IPs, along with start and end times, are removed. Hostnames are also changed to generic host_i. In summary, the final output of this section is a JSON file listing all flows, with each flow being a dictionary containing the features described above.

[0158] 1.3 Flow vectorization unit This module receives a flow JSON compiled by the flow description and converts each flow into a numerical vector. These numerical vectors are the data points that make up the database to be used in future sections of the design.

[0159] The first decision is how to aggregate the different flows defined by the flow description section (Section 1.2). One approach is to treat each individual flow as a separate data point. This approach may not be optimal, as none of the top silhouette scores are derived from individual flows, as shown in the silhouette scores described in subsection 3.2 (Figures 19 and 20).

[0160] This is likely because analyzing a single flow fails to detect many anomalies. For example, a DDoS attack consists of numerous seemingly harmless transmissions from many different senders targeting a single recipient. Therefore, a single flow may not have any outstanding features, but when viewed together, many flows differ. To address this, four aggregation schemes are considered. The first is the single-flow scheme outlined above. The next takes five consecutively initiated flows and groups them together; this is sometimes called the adjacency scheme. The next is five consecutively initiated flows from a specific sender (sender scheme). Finally, there are five consecutively initiated flows to a specific recipient (receiver scheme). Thus, three new JSON files are created with grouped flows. The flows are grouped into sliding windows that jump two flows per grouping. These groups are not a comprehensive list of all possible groupings, and others are outside the scope of this white paper. Next, vectorization is performed on each of the aggregated datasets using known transformation algorithms for the various features of the flows. For grouped flows, the transformed feature is the average of each transformed value in the flows. If at least one flow in a group of flows is malicious, the group of flows is labeled as malicious.

[0161] ●Category Features: These are classifications into seven categories based on transmission protocol, application layer, source IP, destination IP, completion of transmitted data, completion of received data, and destination port based on usage frequency. These features are converted using One-Hot encoding. ●Numerical Features: These include the number of packets received and transmitted, the amount of data received and transmitted, the flow length (in nanoseconds), the average size of received and transmitted packets, and the standard deviation of the sizes of received and transmitted packets. These are min-max scaled using either a [0,1] scale or a [-1,1] scale. ● High-dispersion features: These include only the destination port. These features are first logarithmically scaled, and then min-max scaled, similar to numerical features. ● Remaining unlabeled features may be omitted as they may be considered irrelevant to understanding the flow's functionality.

[0162] 1.4 Latent space generator To enable a mechanism like RAG, it is first necessary to create a search capability that takes a data point and returns the most similar data point that is pre-stored. This is done through a latent space representation that finds latent variables within the data points and uses them to place each data point in its latent space. The tool for achieving this is the autoencoder (AE). An AE is a type of machine learning model that learns latent variables of various data using unsupervised learning. An AE consists of two parts. The first is the encoder, which takes data x and generates latent variables in the vector z. The second is the decoder, which attempts to reconstruct x from z. The model is learned by backpropagation using the reconstruction loss between the original data point and the output of the decoder. After learning, each part has a different use. The decoder can generate new data by taking a noise vector of the same size as z and using it to generate a new x'. Other research is also being done on creating NIDS using autoencoders. In this method, the AE model is trained only on harmless data and the reconstruction error is reduced only with that data. If any abnormal data is input into the trained AE, the error will increase significantly. However, this does not enable the search function and is therefore unsuitable for our purposes here. In the first stage, the autoencoder is trained on the training set, but after training (and after the optimal AE hyperparameters have been estimated from K-fold cross-validation), the decoder is discarded and only the encoder is retained. As mentioned earlier, this encoder can take any data point and return its latent space representation. Then, in the first stage, the entire training set is passed through the encoder and their latent space representations are stored in a vector store along with their labels. These become the new data points, i.e., the labeled latent space vectors of the previous data points. In the second stage, the encoder is used to create a latent space representation of the unlabeled test data.

[0163] 1.5 Flow embedding section This is simply the encoder portion of the trained AE shown in the previous subsection. It is distinguished by its dual use. In the first stage, it computes the latent space representation of the training data points and stores them in a vector store along with their labels to create data points that a RAG-like mechanism will retrieve and use. In the second stage, it computes the latent space representation of unlabeled test data and uses it in the retrieval mechanism. In this case, the data points collected from the latent space generation unit are stored in a vector store. This completes the first stage of the mechanism, and a RAG-like mechanism becomes available.

[0164] 1.8 Flow Acquisition Section In the second stage, after the latent space representation of some data point from the flow vectorization unit is computed, the acquisition mechanism compares the distance of that data point to the data points stored in the vector store. In experiments, the distance function used to compare distances is either L2 or cosine similarity. Only the data point with the minimum distance from all stored data points is returned. These are acquired along with their labels, which will be used to provide context for LLM to analyze the packet data in the future.

[0165] 2. Implementation 2.1 Dataset The dataset used in this experiment consists of packets captured during the aforementioned Kitsune1 experiment. This includes nine different PCAP files, one for each attack, and their corresponding label files.

[0166] 2.2 Environment The experiments were performed on an Asure StandardNC24ads A100 v4 server running Ubuntu 22.04 OS. All experiments were executed using Python scripts with version 3.10.12 or Jupyter Notebooks. Key Python libraries used included Pandas for data processing, PyTorch for AE, Sklearn for results collection, and ChromaDB for vector storage.

[0167] 3. Experiment and Results This section describes the steps taken to test the mechanism. The objective was to create a latent space representation of network communications collected from network packets. Next, we attempted to create an acquisition mechanism that queries network communications representing attacks within that latent space and retrieves other communications from the same type of attack.

[0168] 3.1 Procedures Each attack's PCAP is divided into 5-second fragments, and each fragment is saved again in a new PCAP file. Simultaneously, a new label file is created containing only the packets saved in the relevant PCAP file. Each PCAP file, along with its corresponding label file, is converted into a flow. Each flow is described by the flow descriptor described in subsection 1.2, along with its label features. The label features are as follows: ● The average number of packets sent from a flow sender to a receiver that were flagged as malicious. ● The average number of packets received by a flow from the sender to the receiver and flagged as malicious. ●It is a class of flow, and if any of the averages are greater than 0, it is determined to be an attack; otherwise, it is determined to be harmless. ●It is determined to be a subclass of the class and is one of the following: ○If the flow is classified as harmless, it is labeled as harmless (benign). ○If the class is Fuzzing or OS Scan, then it is Recon. ○If the class is ARP MitM, Active Wiretap, or Video Injection, then it is MitM. ○If the class is SSDP Flood, SSL Renegotiation, or SYN DoS, it is a DoS attack.

[0169] All flows are then merged into two files: the first file contains all flows extracted from the original PCAP, and the second file contains flows extracted from the segmented files. The files are split into training and test files using a 20% test size. For each completed file, a transformer is created by the process described in the Flow Vectorization section (subsection 1.3). All aggregated flows are extracted from each attack file and, sorted by their start time, the average is taken over a sliding window of length 5 and step size 2. At this point, 16 separate training sets have been separated by PCAP size (segmented fully or intervally), vector normalization ([-1,1] or [0,1]), and aggregation (none, start time only, start time and sender, start time and receiver). For each of the 16 training sets, the AE was tuned with various AE-specific hyperparameters to find the best model for that particular set. This was performed by applying a 5-fold hyperparameter adjustment to each combination, based on the average reconstruction loss.

[0170] Next, the encoder portion of the trained AE becomes the embedding function for that dataset.

[0171] Finally, the vector store is initialized using the ChromaDB library 2. For each point in the training set, an embedding is created using the aforementioned embedding function, the document represented by the corresponding data point in the florist, and the data point's label as metadata. Two collections are created for each training set: the first uses L2 distance as the distance function, and the second uses cosine similarity.

[0172] The flow on the test set for each dataset is also passed to the dataset's embedding function, which then creates the embedded test set.

[0173] 3.2 Experiment In the first experiment, we tested the architecture's embedding capabilities. If the embedding is successful, it creates a latent spatial representation that can easily separate different classes (all eight attack data and harmless data), allowing new data to be easily placed near data points of the same class as it arrives. The second goal is to also be able to separate subclasses. This would demonstrate that there are underlying characteristics common to similar attacks.

[0174] Silhouette scores were used to test the separability of the embedding functions. This metric provides a score from -1 to 1 indicating how similar each object is to its own cluster and how different it is from other clusters. This cluster scoring method is particularly good because it does not require training data and can score clustering based on which cluster each point belongs to (Meshal Shutaywi and Nezamoddin N Kachouie. 2021. Silhouette Analysis for Performance Evaluation in Machine Learning with Applications to Clustering. Entropy 23, 6 (2021), 759). Doing this for all points gives a clue as to how well different classes can be separated, but this should not be given much weight due to the large differences in the number of data points in each class.

[0175] These results indicate some feasibility for this method. The best results were obtained when using complete flows in the range [-1,1] and placing them adjacent to each other. However, since it is unlikely that such complete captures will be available and the analysis will require significant resources, we have also included results using only data available at 5-second intervals. Flows sent from specific senders also yielded promising results of up to 0.4 in the same range. All of these are shown in Figures 19 and 20. It should also be noted that the silhouette scores were obtained based on subclasses. Here, the highest overall score was 0.62 for the same dataset as the class-based silhouette scores. Also, in the interval dataset, the highest subclass-based silhouette score was 0.52 for adjacent intervals in the range [0,1]. In all cases, the better distance metric was cosine similarity.

[0176] In the next experiment, we tested the acquisition mechanism. We queried the matching vector store using the latent space representation of each flow in the test set, and five results were returned. By checking the top 1, 3, and 5 retrieved results, three different predictions were extracted. In case of a tie, the result that was closer to the query was selected. Precision, precision, recall, and F1 score were calculated, but for brevity, only the F1 score will be discussed. Figure 21 shows the F1 score.

[0177] These datasets demonstrate high accuracy in classification. This is also true for the top 1 or top 5 datasets, with the difference between the best datasets in both sets being less than 0.001. What is observed here is that the best datasets vary between search scores and silhouette scores.

[0178] To analyze the success rate in more detail, a confusion matrix was created. This matrix shows the actual class on the y-axis and the predicted class on the x-axis. This allows us to see how accurately each class was classified and which classes it was most confused with.

[0179] Figure 22 shows the confusion matrices of the top F1-scoring datasets across all datasets and across interval datasets, respectively. Another way to calculate the success rate is to look at the distance between these confusion matrices and the identity matrix. The Frobenius norm was used. The distance is 1.6203 for all datasets, but 1.9633 for interval datasets only.

[0180] This method holds great potential for the future in various fields, including cybersecurity. The results demonstrate high effectiveness in obtaining correctly labeled data points, which were not originally available with packet-based LLM. This not only improves the capabilities of LLM in the cybersecurity field but also demonstrates access to all non-text data points. The mechanisms presented in this section, when used in combination with finely tuned LLMs, can improve the performance of classification tasks and provide cybersecurity recommendations.

[0181] Finally, this section introduced a mechanism for creating latent space representations of network packets using an autoencoder. This mechanism can be used independently or in combination with LLM for network anomaly classification. The functionality of this mechanism has been validated with experimental data using the Kitsune dataset. By extending the preprocessing of network packets, additional features can be introduced that may be useful for distinguishing between classes of network flows. Combining some of the aggregation techniques described in this section can improve classification performance against specific attacks such as DDoS. One limitation of this mechanism is the limited representation of temporal information between network packets and flows, but network characteristics can be represented more robustly by using transformers. This mechanism can be implemented using a multi-stage approach, which can improve the ability to classify anomalies that do not exist in the vector store database.

[0182] This disclosure includes the following additional information. (Note 1) A method performed by a computer, Based on the input network packet data, generate a target flow summary containing information (in a semi-structured data format) about the (target) packet flow between two / a pair of (network) addresses / entities during the target period, (Using an encoder) to generate a target latent space representation corresponding to the target flow summary based on the target flow summary, The process involves identifying k reference latent space representations that are most similar to the target latent space representation from among multiple reference latent space representations, where the multiple reference latent space representations correspond to multiple reference flow summaries (based on reference network packet data) and multiple classifications, and k is an integer greater than 1. Using the first Large-Scale Language Model (LLM), a target flow report / portrait is generated in natural language format based on the target flow summary, and using the first LLM, k reference flow reports / portraits are generated in natural language format based on the k reference flow summaries corresponding to each of the k identified reference latent space representations. A method comprising: using a second LLM to classify a (target) packet flow as malicious or harmless based on a target flow report / portrait and k reference flow reports / portraits with corresponding classifications (the second LLM is fine-tuned / trained based on training flow reports / portraits to classify training flow descriptions as malicious or harmless). (Note 2) The method performed by computer as described in Appendix 1, The input network packet data includes packet capture (PCAP) data. (Note 3) A method implemented by computer as described in Appendix 1 or Appendix 2, The target flow summary is a method that includes information within a JavaScript® Object Notation (JSON) file. (Note 4) A method performed by a computer as described in any of the appendices 1 to 3, A target flow summary is a method that includes statistical information (and information indicating two addresses) about packets received and transmitted in a packet flow. (Note 5) A method performed by a computer as described in any of the appendices 1 to 4, A reference flow summary is a method in which each includes statistical information (and information indicating two addresses) about the packets received and transmitted in the corresponding reference packet flow. (Note 6) A method performed by a computer as described in Appendix 4, A target flow summary (and each reference flow summary) is a method that includes, as information, the number of packets received, the number of packets transmitted, the size of the data received, the size of the data transmitted, the average received packet size, the average transmitted packet size, the standard deviation of the received packet size, the standard deviation of the transmitted packet size, the protocol, the source IP address, the destination IP address, the source port, the destination port, the start time (of the packet flow), the end time (of the packet flow), the total time the packet flow occurred, the application layer, the service name, the source host name, the destination host name, the internal source, the internal destination, the initial sequence number, the final sequence number, the initial acknowledgment number, and the final acknowledgment number. (Note 7) A method performed by a computer as described in any of the appendices 1 to 6, A method for generating a target flow summary includes defining one of two addresses as the source and the other as the destination. (Note 8) A method performed by a computer as described in any of the appendices 1 to 7, A method for generating a target latent space representation, comprising generating a target numerical vector based on a target flow summary and generating a target latent space representation based on the target numerical vector. (Note 9) The method performed by computer as described in Appendix 8, A method for generating a target numerical vector, comprising converting information within a target flow summary into numerical format by using one-hot encoding of categorical information and normalizing numerical information. (Note 10) A method performed by a computer as described in any of the appendices 1 to 9, A method for generating a target latent space representation, comprising using a trained encoder in an autoencoder training process that includes generating a trained latent space representation based on a training numerical vector. (Note 11) A method performed by a computer as described in any of the appendices 1 to 10, Generating a target latent space representation may include using a trained encoder in an autoencoder training process that trains an autoencoder including an encoder and a decoder, wherein the autoencoder training process includes using the encoder to generate a trained latent space representation based on a training numerical vector, generating a reconstructed numerical vector based on the trained latent space representation, and adjusting the weights of the autoencoder based on the reconstruction loss. (Note 12) A method performed by computer as described in Appendix 10, A computer-based method comprises generating a training latent space representation in a training representation generation process, wherein the training flow summary generates multiple training flow summaries based on training network packet data, each training flow summary containing information (in a semi-structured data format) about training packets between two addresses during a training period (the training flow summary corresponds to multiple address pairs and multiple training periods), and generating multiple training latent space representations (corresponding to the training flow summaries) based on the training flow summaries (using an encoder). (Note 13) A method performed by computer as described in Appendix 10, A method for generating a training latent space representation, comprising generating training numerical vectors based on a training flow summary and generating a training latent space representation based on the training numerical vectors. (Note 14) A method performed by computer as described in Appendix 13, A method for generating training numerical vectors, which involves converting information within a training flow summary into numerical format by using one-hot encoding of categorical information and normalizing numerical information. (Note 15) A method performed by computer as described in Appendix 13 or Appendix 14, A method performed by a computer, comprising generating further training numerical vectors by averaging each of several training numerical vectors (in a sliding window manner), and generating a training latent space representation, comprising generating a training latent space representation based on the further training numerical vectors. (Note 16) A method performed by a computer as described in any of the appendices 10 to 15, A method performed by a computer, which includes performing an autoencoder training process before generating a target latent space representation. (Note 17) A method performed by computer as described in Appendix 16, A method performed by a computer includes repeating the autoencoder training process until the reconstruction loss converges. (Note 18) A method implemented by a computer as described in any of the appendices 1 to 17, Each latent space representation is a representation of the encoder in its latent space, a method. (Note 19) A method implemented by a computer as described in any of the appendices 1 to 17, A method for identifying k reference latent space representations that are most similar to a target latent space representation, comprising determining the L2 distance and / or cosine similarity between the target latent space representation and each reference latent space representation. (Note 20) A method performed by a computer as described in any of the appendices 1 to 19, Classifying a packet flow as malicious is a method, according to the Second LLM, that involves classifying the packet flow as one of several attacks. (Note 21) A method performed by a computer as described in any of the appendices 1 to 20, A method performed by a computer, which includes classifying a packet flow as one of several attacks (corresponding to) a packet flow using a third LLM if a classification of malicious is generated for that packet flow. (Note 22) A method performed by computer as described in Appendix 20 or Appendix 21, Multiple attack methods include any of the following: Address Resolution Protocol Man in the Middle (ARP MitM), active eavesdropping, Simple Service Discovery Protocol (SSDP), SYN denial of service (DoS), fuzzing, and operating system (OS) scanning. (Note 23) A method performed by a computer as described in any of the appendices 1 to 19, A method performed by a computer, which includes classifying a packet flow as one of several attack types using a third LLM if a classification of malicious is generated for that packet flow. (Note 24) The method performed by computer as described in Appendix 23, Multiple attack types include methods that include any of the following: man-in-the-middle (MitM) attacks, denial-of-service (DoS) attacks, and reconnaissance attacks. (Note 25) A method performed by computer as described in Appendix 23 or Appendix 24, A method performed by a computer, comprising classifying a packet flow as one of several attacks corresponding to its attack type, using at least one further LLM depending on the classification of the attack type. (Note 26) A method performed by computer as described in Appendix 25, A method that includes at least one additional LLM, each corresponding to an LLM of each attack type. (Note 27) A method performed by a computer as described in any of the appendices 1 to 19, A method performed by a computer, comprising: if a classification of malicious is generated for a packet flow, using a third LLM to classify the packet flow as one of several attack types; and, depending on the classification of the attack type, using at least one further LLM to classify the packet flow as one of several attacks corresponding to that attack type. (Note 28) A method implemented by a computer as described in any of the appendices 1 to 27, A method in which a second LLM (and optionally a third LLM) (and further optionally at least one additional LLM) is trained according to a fine-tuning process that includes, for each LLM involved, generating classifications in a training flow report using the LLM and adjusting the weights of the LLM based on the difference between the generated classifications and the ground truth classifications corresponding to the training flow report. (Note 29) The method performed by computer as described in Appendix 28, A method performed by a computer, which includes a fine-tuning process before generating a classification. (Note 30) A method performed by a computer as described in any of the appendices 1 to 29, A computer-generated method is a method for generating multiple reference latent space representations, comprising: generating multiple reference flow summaries based on reference network packet data in a representation generation process, each reference flow summary containing information (in a semi-structured data format) about reference flow packets between two addresses during a reference period (the reference flow summary corresponds to multiple address pairs and multiple reference periods); and generating multiple reference latent space representations (corresponding to the reference flow summaries) based on the reference flow summaries. (Note 31) A method performed by computer as described in Appendix 30, Each reference flow summary contains information about the reference packet flow in question, in accordance with the target flow summary. (Note 32) A method performed by computer as described in Appendix 30 or Appendix 31, A method for generating a reference latent space representation, comprising generating a reference numerical vector based on a reference flow summary and generating a reference latent space representation based on the reference numerical vector. (Note 33) A method performed by a computer as described in Appendix 32, A method for generating a reference numerical vector, which involves converting information within a training flow summary into numerical format by using one-hot encoding of categorical information and normalizing the numerical information. (Note 34) A method performed by computer as described in Appendix 32 or Appendix 33, A method performed by a computer, comprising generating a reference latent space representation based on the further reference numerical vectors, wherein the method comprises generating a further reference numerical vector by averaging each of the aforementioned multiple reference numerical vectors (in a sliding window manner). (Note 35) A method implemented by a computer as described in any of the appendices 1 to 34, A method in which each packet in reference network packet data is associated with a classification, and the classification corresponding to a given reference latent space representation depends on the classification of the packets on which that reference latent space representation is based. (Note 36) A method performed by a computer as described in Appendix 35, The reference period is defined using a sliding window method. (Note 37) A method performed by a computer as described in any of the appendices 1 to 36, A method implemented by a computer, which includes, if a packet flow is identified as malicious, instructing at least one entity associated with two addresses to take action in response to that malicious packet flow. (Note 38) A method performed by computer as described in Appendix 37, The instruction is to restrict communication, or the method. (Note 39) A method implemented by a computer as described in any of the appendices 1 to 38, A method performed by a computer, which includes notifying at least one entity associated with two network addresses that a packet flow is malicious if the packet flow is identified as malicious. (Note 40) A method performed by a computer as described in any of the appendices 1 to 39, A method performed by a computer, which includes generating an analysis report containing information about the classification of packet flows. (Note 41) A method performed by a computer as described in Appendix 40, The report includes, if a flow is identified as malicious, recommendations for mitigating malicious packets and / or attacks. (Note 42) When run on a computer, the computer will Based on the input network packet data, generate a target flow summary containing information (in a semi-structured data format) about the (target) packet flow between two / a pair of (network) addresses / entities during the target period, The process involves generating a target latent space representation based on the target flow summary (using an encoder), and identifying k reference latent space representations that are most similar to the target latent space representation from among multiple reference latent space representations, where the multiple reference latent space representations correspond to multiple reference flow summaries (based on reference network packet data) and multiple classifications, and k is an integer greater than 1. Using the first Large-Scale Language Model (LLM), a target flow report / portrait is generated in natural language format based on the target flow summary, and using the first LLM, k reference flow reports / portraits are generated in natural language format based on the k reference flow summaries corresponding to each of the k identified reference latent space representations. A computer program that uses a second LLM to perform a method including classifying a (target) packet flow as malicious or harmless based on a target flow report / portrait and k reference flow reports / portraits with corresponding classifications (the second LLM is fine-tuned / trained based on training flow reports / portraits to classify training flow descriptions as malicious or harmless). (Note) 43 It has memory and a processor connected to the memory, and the processor is Based on the input network packet data, generate a target flow summary containing information (in a semi-structured data format) about the (target) packet flow between two / a pair of (network) addresses / entities during the target period, Generating a target latent space representation corresponding to the target flow summary based on the target flow summary using an encoder, and identifying k reference latent space representations that are most similar to the target latent space representation from among a plurality of reference latent space representations, wherein the plurality of reference latent space representations correspond to a plurality of reference flow summaries (based on reference network packet data) and a plurality of classifications, and k is an integer greater than 1, Using a first large language model (LLM), generating a target flow report / portrait in natural language format based on the target flow summary, and using the first LLM, generating k reference flow reports / portraits in natural language format based on the k reference flow summaries respectively corresponding to the identified k reference latent space representations, Using a second LLM, classifying (target) packet flows as malicious or harmless based on the target flow report / portrait and the k reference flow reports / portraits with corresponding classifications (the second LLM is fine-tuned / trained based on training flow reports / portraits to classify training flow descriptions as malicious or harmless). An information processing device configured to perform the above.

Explanation of Signs

[0183] 10 Information processing device (computing device) 11 PCAP data 12 Flow summary (network flow feature) 13 Flow report (paragraph-like description) 992 Bus 993 Processor 994 Memory 995 Display unit 996 Input mechanism 997 Network interface

Claims

1. A method performed by a computer, Based on the input network packet data, generate a target flow summary containing information about the packet flow between two addresses during the target period, Based on the aforementioned target flow summary, a target latent space representation corresponding to the aforementioned target flow summary is generated, The objective is to identify k reference latent space representations that are most similar to the target latent space representation from among multiple reference latent space representations, wherein the multiple reference latent space representations correspond to multiple reference flow summaries and multiple classifications, and k is an integer greater than 1. Using a first large-scale language model (LLM), a target flow report is generated in natural language format based on the target flow summary, and using the first LLM, k reference flow reports are generated in natural language format based on the k reference flow summaries corresponding to each of the k identified reference latent space representations. Using the second LLM, classify the packet flow as malicious or harmless based on the target flow report and the k reference flow reports having corresponding classifications. A method of having.

2. The target flow summary includes statistical information about packets received and transmitted in the packet flow. The method carried out by a computer as described in claim 1.

3. Generating the target latent space representation includes generating a target numerical vector based on the target flow summary and generating the target latent space representation based on the target numerical vector. The computer-based method described in claim 1 or 2.

4. The generation of the target numerical vector includes converting the information in the target flow summary into numerical values ​​by using one-hot encoding of the categorical information and by normalizing the numerical information. The method carried out by a computer as described in claim 3.

5. Generating the target latent space representation involves using a trained encoder in an autoencoder training process that includes generating a trained latent space representation based on a training numerical vector. The method carried out by a computer as described in claim 3.

6. The process includes performing the autoencoder training process before generating the target latent space representation. The computer-based method described in claim 5.

7. Identifying the k reference latent space representations that are most similar to the target latent space representation includes determining the L2 distance and / or cosine similarity between the target latent space representation and each reference latent space representation. The computer-based method described in claim 1 or 2.

8. Classifying the packet flow as malicious includes classifying the packet flow as one of several attacks by the second LLM. The computer-based method described in claim 1 or 2.

9. If a classification of "malicious" is generated for the packet flow, a third LLM is used to classify the packet flow as one of several attack types, and according to the classification of the attack type, at least one further LLM is used to classify the packet flow as one of several attacks corresponding to that attack type. The computer-based method described in claim 1 or 2.

10. In the representation generation process, Based on the reference network packet data, multiple reference flow summaries are generated, and each reference flow summary contains information about the reference flow packets between two addresses during the reference period. To generate the multiple reference latent space representations based on the aforementioned reference flow summary. The method involves generating the aforementioned multiple reference latent space representations. The computer-based method described in claim 1 or 2.

11. Generating the aforementioned reference latent space representation includes generating a reference numerical vector based on the aforementioned reference flow summary, and generating the aforementioned reference latent space representation based on the aforementioned reference numerical vector. The computer-based method described in claim 10.

12. This includes generating a further reference numerical vector by averaging each of the aforementioned multiple reference numerical vectors, Generating the aforementioned reference latent space representation further includes generating the reference latent space representation based on the aforementioned further reference numerical vectors. The method carried out by a computer as described in claim 11.

13. If a packet flow is identified as malicious, the system includes instructing at least one entity associated with the two addresses to take action in response to the malicious packet flow. The computer-based method described in claim 1 or 2.

14. When executed on a computer, the computer will Based on the input network packet data, generate a target flow summary containing information about the packet flow between two addresses during the target period, Based on the aforementioned target flow summary, a target latent space representation corresponding to the aforementioned target flow summary is generated, The objective is to identify k reference latent space representations that are most similar to the target latent space representation from among multiple reference latent space representations, wherein the multiple reference latent space representations correspond to multiple reference flow summaries and multiple classifications, and k is an integer greater than 1. Using a first large-scale language model (LLM), a target flow report is generated in natural language format based on the target flow summary, and using the first LLM, k reference flow reports are generated in natural language format based on the k reference flow summaries corresponding to each of the k identified reference latent space representations. Using the second LLM, classify the packet flow as malicious or harmless based on the target flow report and the k reference flow reports having corresponding classifications. A computer program that causes a method to be performed.

15. It has memory and a processor connected to the memory, The aforementioned processor, Based on the input network packet data, a target flow summary is generated that includes information about the packet flow between two addresses during the target period. Based on the target flow summary, a target latent space representation corresponding to the target flow summary is generated. From among multiple reference latent space representations, k reference latent space representations that are most similar to the target latent space representation are identified, and the multiple reference latent space representations correspond to multiple reference flow summaries and multiple classifications, and k is an integer greater than 1. Using the first Large-Scale Language Model (LLM), a target flow report is generated in natural language format based on the target flow summary, and using the first LLM, k reference flow reports are generated in natural language format based on the k reference flow summaries corresponding to each of the k identified reference latent space representations. Using the second LLM, the packet flow is classified as malicious or harmless based on the target flow report and the k reference flow reports having corresponding classifications. An information processing device configured in such a way.

Images