Industrial network anomaly behavior identification method, device, medium and product

Abstract

The application provides an industrial network abnormal behavior identification method, device, medium and product, relates to the technical field of industrial data processing, and is used for improving the efficiency and accuracy of abnormal behavior identification of an industrial control host. The method comprises the following steps: training a preset large language model based on first text information comprising structured text for indicating network traffic data and adding a behavior label, and second text information for indicating structured text about industrial control host log data, to obtain a target identification model for identifying whether the communication behavior of the industrial control host is abnormal. In this way, the target identification model trained based on the first text information and the second text information can analyze and identify the characteristics of new abnormal behaviors in real time, thereby improving the identification efficiency and accuracy of new abnormal behaviors.

Description

Technical Field

[0001] This application relates to the field of industrial data processing technology, and in particular to a method, device, medium and product for identifying abnormal behavior in industrial networks. Background Technology

[0002] Industrial control networks are communication networks specifically designed for industrial automation systems. They connect sensors, controllers, actuators, and other devices at the production site to enable industrial production. The industrial control host (ICN) plays a crucial role as the control core of the ICN, therefore, it is necessary to identify abnormal behavior in the ICN to ensure the stable operation of the industrial control network.

[0003] Solutions for identifying abnormal behavior in industrial control systems include rule-based identification and machine learning-based detection. However, rule-based identification suffers from low efficiency, while machine learning-based detection offers poor accuracy. Therefore, these solutions suffer from both low efficiency and low accuracy. Summary of the Invention

[0004] This application provides a method, device, medium, and product for identifying abnormal behavior in industrial networks, which improves the efficiency and accuracy of identifying abnormal behavior in industrial control hosts.

[0005] Firstly, this application provides a method for identifying abnormal behavior in industrial networks, including: Obtain a training sample set, which includes text information sample pairs with added behavior labels; the text information sample pairs include first text information and second text information; the first text information is used to indicate structured text about network traffic data generated by the communication behavior of the industrial control host, the second text information is used to indicate structured text about the log data of the industrial control host, and the behavior label is used to indicate whether the communication behavior is normal business behavior or abnormal business behavior. The preset large language model is trained based on the training sample set to obtain a target recognition model for identifying whether the communication behavior of the industrial control host is abnormal. The target recognition model is a large language model that has been fine-tuned for the business behavior of the industrial control host. The fine-tuned large language model refers to the model after adjusting the low-rank weight matrix of the large language model.

[0006] The technical solution provided in this application offers at least the following benefits: Based on first text information, including structured text indicating network traffic data with added behavioral tags, and second text information, including structured text indicating industrial control host log data, a pre-defined large language model is trained to obtain a target recognition model for identifying whether abnormal communication behavior of industrial control hosts exists. Thus, the target recognition model trained based on the first and second text information can instantly analyze and identify the characteristics of novel abnormal behaviors, thereby improving the efficiency of identifying novel abnormal behaviors. By combining multi-source information from network traffic data and host log data, the communication operation behavior in the industrial control network is analyzed, thereby improving the accuracy of identifying abnormal behaviors in industrial networks, and thus providing effective protection for the safe and stable operation of the industrial production environment.

[0007] One possible implementation involves acquiring a training sample set, including: acquiring first communication data and second communication data; the first communication data is used to indicate network traffic characteristics generated by the communication behavior of the industrial control host, and the second communication data is used to indicate host log data characteristics generated by the communication behavior. By using preset mapping rules, the first communication data and the second communication data are transformed into corresponding first text information and second text information; wherein, the preset mapping rules are used to transform the field information in the first communication data and the second communication data into corresponding structured text description information; Based on the first and second text information, construct text information sample pairs in the training sample set.

[0008] Another possible implementation is to train a pre-defined large language model based on a training sample set to obtain a target recognition model, including: generating embedding vector sample pairs corresponding to text information samples based on first text information and second text information; The embedded vector sample pair includes a first embedded vector and a second embedded vector; the first embedded vector is used to indicate the semantic encoding of network traffic data, and the second embedded vector is used to indicate the semantic encoding of log data. Based on the embedded vector sample pairs corresponding to the text information samples, a pre-set large language model is trained to obtain the target recognition model.

[0009] Another possible implementation involves generating embedding vector sample pairs corresponding to the text information samples based on the first and second text information, including: The first text information and the second text information are semantically encoded to obtain corresponding first semantic vectors and second semantic vectors; wherein, the first semantic vector is used to indicate the vector representation of network traffic data, and the second semantic vector is used to indicate the vector representation of host log data; the first semantic vector and the second semantic vector are in the same high-dimensional semantic space; A projection transformation is performed on the first semantic vector and the second semantic vector to obtain the first embedded vector and the second embedded vector after projection transformation, so that the first embedded vector and the second embedded vector after projection transformation are semantically aligned in the same high-dimensional semantic space. Based on the first embedding vector and the second embedding vector, construct embedding vector sample pairs.

[0010] Another possible implementation involves training a pre-defined large language model based on the embedded vector sample pairs corresponding to the text information samples to obtain the target recognition model, including: Based on the first and second embedding vectors in the semantically aligned embedding vector sample pairs, corresponding sample fusion feature vectors are generated; the sample fusion feature vectors are used to indicate the different dimensional features of the communication behavior corresponding to the embedding vector sample pairs. Based on the sample fusion feature vector with added behavioral labels, the initial classifier in the preset large language model is trained to obtain the target classifier in the target recognition model.

[0011] Another possible implementation is that the sample fusion feature vector includes a first feature vector, a second feature vector, a third feature vector, and a fourth feature vector; The first feature vector is used to indicate the first embedding vector, the second feature vector is used to indicate the second embedding vector, the third feature vector is used to indicate the semantic difference information between the first embedding vector and the second embedding vector, and the fourth feature vector is used to indicate the language association information between the first embedding vector and the second embedding vector.

[0012] Another possible implementation is that the first communication data includes first traffic data, second traffic data, third traffic data, and fourth traffic data; The first traffic data is used to indicate the sending time and rate characteristics of network traffic data packets generated by communication behavior; the second traffic data is used to indicate the size distribution characteristics of the transmitted content of network traffic data packets; the third traffic data is used to indicate the communication protocol characteristics; and the fourth traffic data is used to indicate the application content characteristics of network traffic data packets. The second communication data includes the first log data, the second log data, the third log data, and the fourth log data; The first log data is used to indicate the host status and status recording time characteristics generated by communication behavior; the second log data is used to indicate the network activity characteristics of the industrial control host; the third log data is used to indicate the storage activity characteristics of the industrial control host; and the fourth traffic data is used to indicate whether the configuration file of the industrial control host has changed.

[0013] Secondly, this application provides a method for identifying abnormal behavior in an industrial network, comprising: acquiring target text information pairs of an industrial control host, wherein the target text information pairs include network text information and log text information; the network traffic text information is used to indicate structured text about network traffic data generated by the current communication behavior of the industrial control host, and the log text information is used to indicate structured text about host log data generated by the current communication behavior; The target text information is input into the target recognition model to obtain the recognition result output by the target recognition model. The recognition result is used to indicate whether the current communication behavior is an abnormal business behavior. The target recognition model is trained based on the industrial network abnormal behavior recognition method described in the first aspect above.

[0014] One possible implementation involves inputting the target text information into a target recognition model to obtain the recognition result output by the target recognition model, including: The target text information pairs are converted into corresponding target embedding vector pairs; wherein, the target embedding vector pairs include network traffic embedding vectors and log embedding vectors, the network traffic embedding vectors are used to indicate the semantic encoding of network traffic data generated by the current communication behavior, and the log embedding vectors are used to indicate the semantic encoding of host log data generated by the current communication behavior; In response to the similarity between the network traffic embedding vector and the log embedding vector being less than or equal to a preset matching threshold, an identification result is generated to indicate that the network traffic embedding vector and the log embedding vector do not match; wherein, the similarity is used to indicate the degree of similarity between the semantically aligned network traffic embedding vector and the log embedding vector. In response to the similarity between the network traffic embedding vector and the log embedding vector being greater than a matching threshold, the network traffic embedding vector and the log embedding vector are input into the target classifier in the target recognition model to obtain the recognition result output by the target classifier.

[0015] Another possible implementation involves inputting the network traffic embedding vector and the log embedding vector into the target classifier in the target recognition model to obtain the recognition result output by the target classifier, including: Based on network traffic embedding vectors and log embedding vectors, a target fusion feature vector is generated; the target fusion feature vector is used to indicate different dimensional features of the current communication behavior. The target fusion feature vector is input into the target classifier to obtain the identification result used to indicate whether the current communication behavior is an abnormal business behavior.

[0016] Thirdly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the methods of the first and second aspects described above.

[0017] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the methods of the first and second aspects described above.

[0018] Fifthly, this application provides a computer program product comprising a computer program; when the computer program is run in an electronic device, the electronic device performs the methods described in the first and second aspects.

[0019] The beneficial effects of the second to fifth aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0020] Figure 1 A flowchart illustrating an industrial network abnormal behavior identification method provided in an embodiment of this application; Figure 2 A flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application embodiment; Figure 3 A flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application embodiment; Figure 4 A flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application embodiment; Figure 5 A flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application embodiment; Figure 6 A flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application embodiment; Figure 7 A system architecture diagram provided for an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0022] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0023] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0024] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0025] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0026] With the widespread adoption of IoT devices and the popularization of 5G technology, industrial equipment and production systems are becoming increasingly networked, leading to the construction of highly digitized industrial control networks. As the core of production operations, the security, stability, and reliability of industrial control networks directly determine the production efficiency and product quality of industrial enterprises.

[0027] However, the digitalization of industrial control networks is accompanied by an increasing number of cybersecurity threats. As the control core of industrial control networks, industrial control hosts face complex external network attacks and internal security threats, including but not limited to unauthorized business operations, industrial control equipment malfunctions, and malicious attacks, seriously threatening the stable operation of industrial production. Therefore, it is necessary to identify abnormal behavior of industrial control hosts to ensure the stable operation of industrial control networks.

[0028] Solutions for identifying abnormal behavior of industrial control hosts include abnormal behavior identification solutions based on manual rules, and abnormal behavior detection solutions based on machine learning or deep learning.

[0029] Manually-based anomaly detection schemes typically begin by having industry experts analyze the internal and external threat characteristics present in industrial control network security incidents. Then, detection rules are manually written based on these characteristics, or feature conversion algorithms are used to transform the features into detection and identification rules. However, this method requires industry experts to analyze and extract detection and identification rules for different industrial control protocols, resulting in a large maintenance workload, slow adaptation speed, and an inability to identify new types of anomaly attacks. This necessitates further maintenance and updates of the identification rules by industry experts, leading to low efficiency in identifying new anomalies.

[0030] Abnormal behavior detection solutions based on machine learning or deep learning, such as using dedicated DL (Deep Learning) / ML (Machine Learning) small models, typically collect network data or log data from industrial control hosts in industrial control networks, extract normal behavior pattern features from the network data or log data, construct a normal behavior baseline, and monitor the behavior of industrial control hosts in real time through the normal behavior baseline, thereby realizing the identification of abnormal behavior.

[0031] However, machine learning-based anomaly detection solutions rely on a single data source, analyzing behavior in industrial control networks from only one data source. This approach lacks comprehensiveness, and attacks in industrial control networks are often phased and progressive. For example, attackers may first probe for information, then escalate privileges, and finally damage equipment. Machine learning-based anomaly detection solutions focus on identifying anomalies in single data points at a specific moment, failing to establish correlations between multi-source data or to reason about potential anomalies that only manifest under multiple steps and conditions. They lack comprehensive analysis of novel and complex attack anomalies occurring over long time periods and in multiple stages, making it difficult to identify attack intent and resulting in high false positive rates and poor accuracy.

[0032] To address the aforementioned technical problems, this application provides a method, device, medium, and product for identifying abnormal behavior in industrial networks. Based on first text information including structured text with added behavioral tags to indicate network traffic data, and second text information including structured text indicating industrial control host log data, a preset large language model is trained to obtain a target recognition model for identifying whether abnormal communication behavior of industrial control hosts exists. Thus, the target recognition model trained based on the first and second text information can instantly analyze and identify the characteristics of new abnormal behaviors, thereby improving the efficiency of identifying new abnormal behaviors. By combining multi-source information from network traffic data and host log data, the communication operation behavior in industrial control networks is analyzed, thereby improving the accuracy of identifying abnormal behavior in industrial networks and providing effective protection for the safe and stable operation of industrial production environments.

[0033] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart illustrating an industrial network abnormal behavior identification method provided in an embodiment of this application. (Combined with...) Figure 1 As shown, the industrial network abnormal behavior identification method provided in this application embodiment includes the following steps: S101. Obtain the training sample set.

[0035] In some embodiments, the training sample set includes text information sample pairs with added behavioral labels; the text information sample pairs include first text information and second text information. The first text information is used to indicate structured text about network traffic data generated by the industrial control host's communication behavior, the second text information is used to indicate structured text about the industrial control host's log data, and the behavioral labels are used to indicate whether the communication behavior is normal or abnormal business behavior.

[0036] S102. Train the preset large language model based on the training sample set to obtain the target recognition model.

[0037] In some embodiments, the target recognition model is a large language model fine-tuned for the business behavior of the industrial control host. The fine-tuned large language model refers to the model after adjusting the low-rank weight matrix of the large language model.

[0038] Compared to the small, industrial control-specific models in related solutions, this application adopts a large language model for heterogeneous data primarily based on two considerations: First, the need for processing "unstructured" data. Traffic payload content and various types of untemplateable log data in industrial control hosts are essentially highly unstructured text or byte stream data, requiring the large language model's ability to understand, analyze, and reason about unstructured data. Second, the need for processing "massive heterogeneous" data. Under the condition of continuous operation of multiple protocols and multiple devices for a long time, industrial control hosts generate extremely large amounts of data, thus requiring the large language model's large-scale context processing capabilities and global semantic modeling capabilities.

[0039] The fine-tuned large language model is based on the LoRA (Low-Rank Adaptation of Large Language Models) low-rank incremental parameterization mechanism. The LoRA mechanism involves freezing the original weight matrix of the large language model and training only a set of newly added, structurally simple low-rank weight matrices. These low-rank weight matrices capture the knowledge changes required for downstream tasks, thereby achieving fine-tuning of the large language model, resulting in the target recognition model.

[0040] In this embodiment, the target recognition model trained based on the first and second text information in the training sample set can instantly analyze and identify the characteristics of new abnormal behaviors, thereby improving the efficiency of identifying new abnormal behaviors. By combining multi-source information such as network traffic data and host log data, the communication operation behavior in the industrial control network is analyzed, thereby improving the accuracy of identifying abnormal behaviors in the industrial network and providing effective protection for the safe and stable operation of the industrial production environment.

[0041] Figure 2 This is a flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in an embodiment of this application. (Combined with...) Figure 2 As shown, obtaining the training sample set in step S101 above can be achieved through the following steps: S1011, Obtain the first communication data and the second communication data.

[0042] In some embodiments, first communication data and second communication data are collected from an industrial control network. The first communication data indicates network traffic characteristics generated by the communication behavior of the industrial control host, and the second communication data indicates host log data characteristics generated by the communication behavior.

[0043] In some embodiments, for the first communication data, a large number of network data packets will be generated during an industrial control host communication process. This application uses all the network data packets generated during this complete communication process as a network communication stream.

[0044] This application uses Zeek to capture network communication flows in an industrial control environment and uses Python scripts to process and obtain basic information for each data packet: source IP address, destination IP address, source port number, destination port number, transport layer protocol, and transmitted data content. The basic information in the network communication flow is further processed to obtain the first communication data of the corresponding network communication flow. The first communication data includes first flow data, second flow data, third flow data, and fourth flow data.

[0045] For example, the first traffic data is used to indicate the sending time and rate characteristics of network traffic data packets generated by communication behavior, such as the timestamps of the first and last data packets in the network communication flow, the data packet sending rate, and the average, maximum, and minimum values ​​of the data packet intervals.

[0046] For example, the second traffic data is used to indicate the size distribution characteristics of network traffic data packet transmission content, such as the mean, skewness, and kurtosis of the data packet transmission content length, the ratio of sent to received data packet sizes, and the lifetime of the data packets.

[0047] For example, third-party traffic data is used to indicate communication protocol characteristics such as the latency of TCP (Transmission Control Protocol) handshake and teardown, the number of TCP retransmissions, the proportion of out-of-order packets, the trend of TCP window size changes, and the UDP (User Datagram Protocol) packet loss rate.

[0048] For example, the fourth traffic data is used to indicate the application content characteristics of network traffic data packets, such as the frequency of function code occurrence, the matching rate and latency distribution of requests and responses, and the proportion of abnormal instructions such as illegal function codes.

[0049] In some embodiments, log data collected by the Logstash open-source tool during a network communication flow, such as communication timestamps, CPU usage, running memory usage, the size of received and sent data packets, storage read and write speed, and configuration files of industrial control equipment, are processed to obtain second communication data. The second communication data includes first log data, second log data, third log data, and fourth log data.

[0050] For example, the first log data is used to indicate the host status generated by the communication behavior and the time characteristics of the status record, such as CPU utilization, running memory utilization, and communication timestamp.

[0051] For example, the second log data is used to indicate network activity characteristics of the industrial control host, such as the size of the data packets received and sent.

[0052] For example, third log data is used to indicate storage activity characteristics of the industrial control host, such as the storage read / write speed of the industrial control host hard drive or memory.

[0053] For example, the fourth log data is used to indicate whether the configuration file of the industrial control host has been changed.

[0054] S1012. By using a preset mapping rule, the first communication data and the second communication data are converted into corresponding first text information and second text information.

[0055] In some embodiments, a preset mapping rule is used to convert field information in the first communication data and the second communication data into first text information and second text information as structured text description information, that is, to fill in the blanks with the field information in the collected first communication data and the second communication data into the corresponding text statements.

[0056] The preset mapping rules are not simply a matter of "format conversion." Instead, they employ a stable and unified textual representation to completely map all necessary field information (such as timestamps, addresses, function codes, field values, and status changes) from the first and second communication data into a structured text description according to predetermined rules. In practice, this textual mapping is an "equivalent restatement," without compressing fields, incurring abstraction losses, or concealing any element data used for detection or analysis.

[0057] In some embodiments, the first text information includes a first traffic text, a second traffic text, a third traffic text, a fourth traffic text, and a fifth traffic text.

[0058] For example, the first traffic text could be something like "A complete network communication flow was recorded from source IP [source IP address]:[source port number] to destination IP [destination IP address]:[destination port number], based on the [transport layer protocol] protocol, during the time range [start timestamp] to [end timestamp]".

[0059] The second traffic text can be like this: "The duration of this communication flow is [duration] seconds. The average data packet sending rate is [data packet sending rate] packets / second. The average time interval between data packets is [average interval] milliseconds, the maximum interval is [maximum interval] milliseconds, and the minimum interval is [minimum interval] milliseconds."

[0060] The third traffic text can be as follows: "The average length of the data packet content is [mean length] bytes, and the length distribution shows a [skewness-based] distribution, such as: right-skewed / left-skewed / symmetrical (skewness = [skewness value]), and the distribution shape is [kurtosis-based], such as: steep / flat (kurtosis = [kurtosis value]). The ratio of the size of the sent and received data packets is [sent and received size ratio]."

[0061] The fourth traffic text could be as follows: "[If it is the TCP protocol]: The TCP three-way handshake connection establishment delay is [TCP handshake delay] milliseconds, and the four-way handshake connection termination delay is [TCP handshake delay] milliseconds. During the communication process, a total of [TCP retransmission count] retransmission events and [out-of-order packet ratio]% of out-of-order packets were detected. The TCP window size shows a [window size change trend, such as: stable / continuously increasing / sharply decreasing] trend. [If it is the UDP protocol]: Based on sequence number analysis, the estimated UDP packet loss rate is approximately [UDP packet loss rate]%."

[0062] The fifth traffic text could be something like, "At the application layer, multiple function codes were detected, with the most frequent being [Function Code A], [Function Code B], etc. The matching rate between requests and responses is [Match Rate]%, and the average response latency is [Mean Latency Distribution] milliseconds. [Percentage of Abnormal Instructions]% of abnormal instructions were detected, such as illegal function codes or format errors."

[0063] In some embodiments, the second text information includes a first log text, a second log text, a third log text, a fourth log text, and a fifth log text.

[0064] For example, the first log text could be something like "Status log of associated host [IP address or hostname] at [timestamp] (or the time period from [start timestamp] to [end timestamp]).

[0065] The second log text can be something like "System Performance: Host CPU utilization is [CPU utilization]%, RAM utilization is [RAM utilization]%".

[0066] The third log text could be something like, "During this period, the host received [received data packet size] KB of data and sent [sent data packet size] KB of data."

[0067] The fourth log text could be something like, “The read speed of the host storage (hard drive / flash memory) is [storage read speed] MB / s, and the write speed is [storage write speed] MB / s.”

[0068] The fifth log text can be like this: "[If the industrial control host's configuration file has not been changed]: Verification shows that the core configuration file of the industrial control device has not been changed. [If the industrial control host's configuration file has been changed]: [High risk alert] Verification shows that the core configuration file of the industrial control host has been modified."

[0069] The content within brackets “[]” represents the extracted first and second communication data.

[0070] S1013. Based on the first text information and the second text information, construct text information sample pairs in the training sample set.

[0071] In some embodiments, based on the obtained first and second text information, a matching text information sample pair is constructed. , , ), y∈{normal,attack}, where y is the behavior label, "normal" indicates that the collected communication behavior is normal business behavior, and "attack" indicates that the collected behavior is abnormal business behavior. This is the first text information. This is the second text information.

[0072] In this embodiment of the application, by converting the network traffic data and log data generated by the communication behavior of the industrial control host into corresponding text information, the target recognition model trained based on the large language model can understand and analyze the characteristics of the communication behavior of the industrial control host, thereby improving the accuracy of identifying abnormal behavior of the industrial control host.

[0073] Figure 3 This is a flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application. (Combined with...) Figure 3 As shown, in step S102 above, training the preset large language model based on the training sample set to obtain the target recognition model can be implemented as follows: S1021. Based on the first text information and the second text information, generate embedded vector sample pairs corresponding to the text information samples.

[0074] In some embodiments, within an industrial control network environment, the first text information used to indicate network traffic data and the second text information used to indicate host log data differ in structure and semantics, making it difficult to obtain an effective unified representation through direct feature extraction. To address this heterogeneous data encoding problem, embodiments of this application employ a preset large language model with frozen weight matrix parameters to perform deep semantic encoding on the two modalities, obtaining corresponding embedding vector sample pairs.

[0075] The embedded vector sample pair includes a first embedded vector and a second embedded vector; the first embedded vector is used to indicate the semantic encoding of network traffic data, and the second embedded vector is used to indicate the semantic encoding of log data.

[0076] For example, by using the weight parameters frozen in the pre-set large language model, semantic encoding is performed on the first text information and the second text information respectively to obtain the corresponding text information that is in the same... The first and second semantic vectors of the dimensional semantic space.

[0077] For example, for the first semantic vector, it is represented by the following formula (1): (1); in, This is the first semantic vector. This is the first text information. The first semantic vector is used to indicate the vector representation of network traffic data, which is a preset weight parameter frozen in the large language model.

[0078] For example, the second semantic vector is represented by the following formula (2): (2); in, For the second semantic vector, This is the second text information. The second semantic vector is used to indicate the vector representation of host log data, which is a set of weight parameters frozen in the pre-defined large language model.

[0079] After obtaining the first and second semantic vectors in the same high-dimensional semantic space, that is, after achieving unified encoding of network traffic data and host log data, the data vectors of different modalities still have the problem of semantic space inconsistency. That is, related network traffic data and host log data may be far apart in the original embedding space, making it impossible to directly perform similarity calculation and matching judgment.

[0080] Therefore, to solve the cross-modal semantic alignment problem, the embodiments of this application utilize a learnable normalization function in a preset projection transformation network and a two-layer multilayer perceptron structure. and Nonlinear projection transformations are performed on the first and second semantic vectors of the two modalities respectively to obtain the first and second embedding vectors after projection transformation, so that the first and second embedding vectors after projection transformation are semantically aligned in the same high-dimensional semantic space.

[0081] For example, for the first embedding vector, it is shown by the following formula (3): (3); in, This is the first embedding vector. The first semantic vector is used to indicate the semantic encoding of network traffic data.

[0082] For example, for the second embedding vector, it is shown by the following formula (4): (4); in, For the second embedding vector, The second semantic vector, the second embedding vector, is used to indicate the semantic encoding of the log data. The division operation implements the L2 normalization function, which eliminates the influence of vector magnitude differences, making similarity calculation more stable and reliable. Simultaneously, the learnable nature of the projection network allows it to adaptively learn the optimal cross-modal mapping relationship, ultimately achieving accurate anomaly detection and judgment within a unified semantic space.

[0083] Based on the first and second embedding vectors, construct embedding vector sample pairs with added action labels.

[0084] In some embodiments, the preset projection transformation network is trained using a contrastive learning mechanism, which minimizes the distance between the first semantic vector and the second semantic vector from the same secure communication event in the unified semantic space after projection, while maximizing the distance between the first semantic vector and the second semantic vector from different events or unrelated data, thereby achieving semantic alignment of cross-modal data.

[0085] In solutions for detecting abnormal behavior in industrial control networks, it is necessary to accurately distinguish between normal behavior, abnormal attack behavior, and random combinations of network log mismatches.

[0086] Existing solutions typically employ single-modal feature extraction or simple feature concatenation, relying on traditional distance metrics such as Euclidean distance or Manhattan distance for similarity judgment. This one-way similarity calculation is easily affected by data distribution biases and cannot fully explore the deep semantic relationships between network traffic and host logs. Consequently, it performs poorly in complex attack scenarios, especially when dealing with heterogeneous data with misaligned time sequences or noise interference, resulting in a high false positive rate. To address this sample differentiation challenge, this application's embodiments construct a bidirectional contrastive learning mechanism. By introducing a softmax normalization function controlled by a temperature parameter, precise semantic alignment between the first semantic vectors is achieved.

[0087] For example, in the contrastive learning mechanism of this application embodiment, the similarity between the first embedded vector and the second embedded vector after projection is first calculated by the cosine similarity formula. The similarity between the first embedded vector and the second embedded vector is expressed by the following formula (5): (5); in, Let be the similarity between an embedded vector and a second embedded vector. This is the first embedding vector. This is the second embedding vector.

[0088] Based on the similarity between the first and second embedding vectors, a bidirectional loss function is constructed, which is expressed by the following formula (6): (6); in, Let be the similarity between an embedded vector and a second embedded vector. The temperature parameter is used. The bidirectional loss function controls the steepness of the similarity distribution through the temperature parameter τ, and considers the bidirectional matching relationship between network traffic data and host log data and between host log data and network traffic data. Through symmetry constraints, it ensures that network traffic data and host log data with the same communication behavior (including normal and abnormal communication behavior) are clustered in the high-dimensional feature space, while mismatched network traffic data and host log data are effectively separated outside the decision boundary, thereby improving the accuracy and robustness of sample differentiation.

[0089] S1022. Based on the embedded vector sample pairs corresponding to the text information samples, train the preset large language model to obtain the target recognition model.

[0090] In some embodiments, relying solely on single similarity information is insufficient to accurately distinguish between three complex states: normal communication behavior, abnormal communication behavior, and mismatch between network traffic data and host log data. Therefore, more refined feature representations are needed to capture their inherent differences.

[0091] In some embodiments, after obtaining the semantically aligned first and second embedding vectors, a corresponding sample fusion feature vector is constructed based on the first and second embedding vectors. The sample fusion feature vector comprehensively characterizes the correlation features between network traffic data and host log data from different semantic perspectives, thereby enabling the initial classifier training of a preset large language model based on the constructed sample fusion feature vector.

[0092] In this embodiment, the weight matrix parameters frozen in the pre-defined large language model are used. This approach fully leverages the rich semantic representation capabilities and cross-modal understanding capabilities of large language models pre-learned on massive multi-domain data, mapping heterogeneous industrial control data into a unified high-dimensional semantic space. Simultaneously, it avoids the catastrophic forgetting and overfitting problems that may result from full-parameter fine-tuning on limited industrial anomaly detection samples, significantly reducing training computational overhead and ensuring the stability, consistency, and generalization ability of the encoding process. This lays a solid feature representation foundation for subsequent industrial network anomaly behavior recognition based on multimodal fusion features.

[0093] Figure 4 This is a flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application. (Combined with...) Figure 4 As shown, in step S1022 above, training the preset large language model based on the embedded vector sample pairs corresponding to the text information samples to obtain the target recognition model can be implemented as follows: S1221. Based on the first and second embedding vectors in the semantically aligned embedded vector sample pairs, generate the corresponding sample fusion feature vectors.

[0094] In some embodiments, the sample fusion feature vector is used to indicate different dimensional features of the communication behavior corresponding to the embedded vector sample pairs. The sample fusion feature vector includes a first feature vector, a second feature vector, a third feature vector, and a fourth feature vector.

[0095] For example, the sample fusion feature vector is represented by the following formula (7): (7); in, For sample fusion feature vectors, The first eigenvector, This is the second feature vector. The third eigenvector, This is the fourth feature vector. The first feature vector is used to indicate the first embedding vector, the second feature vector is used to indicate the second embedding vector, the third feature vector is used to indicate the semantic difference information between the first and second embedding vectors, and the fourth feature vector is used to indicate the linguistic association information between the first and second embedding vectors.

[0096] Specifically, the first and second feature vectors preserve the original embedded representations of network traffic data and host log data, ensuring that domain-specific information is not lost. The third feature vector captures the degree of divergence between the two types of data across various dimensions through element-wise absolute difference calculation, highlighting the semantic differences between heterogeneous data sources. The fourth feature vector mines the collaborative interaction patterns between the two vectors in corresponding dimensions through element-wise multiplication operations, revealing deep correlations between the data. By concatenating and combining these four feature vectors at different semantic levels, the classifier in the pre-defined large language model can simultaneously utilize comprehensive information from the three dimensions of similarity, difference, and interaction to establish a more robust decision boundary in the high-dimensional feature space, enabling it to distinguish between normal behavior and abnormal attack behavior in the vector space matching network traffic data and host log data.

[0097] S1222. Based on the sample fusion feature vector with added behavioral labels, train the initial classifier in the preset large language model to obtain the target classifier in the target recognition model.

[0098] In some embodiments, after obtaining the sample fusion feature vector with corresponding behavioral labels, it is passed through a multilayer perceptron initial classifier. The fused feature vectors of the samples with added behavioral labels are processed to output raw scores containing three categories: normal business behavior, abnormal business behavior, and data mismatch. Among them, data mismatch is used to indicate that the network traffic embedding vector and the log embedding vector do not match. The raw scores are expressed by the following formula (8): (8); in, For sample fusion feature vectors, As the initial classifier, The original score corresponds to the label type. Based on the original score of the corresponding label type, the corresponding standard cross-entropy loss function is constructed. The standard cross-entropy loss function is shown in the following formula (9): (9); in, To encode the real tags corresponding to the behavioral tags using one-hot encoding, This is the original score for the corresponding tag type.

[0099] In some embodiments, the initial classifier is trained for classification using the standard cross-entropy loss function.

[0100] In multi-task learning scenarios, the two objectives of contrastive learning and classification learning may conflict. Therefore, this application employs an adaptive joint optimization strategy to coordinate these two training objectives. The core of the adaptive joint optimization strategy lies in dynamically balancing the potential conflict between feature representation learning and anomaly classification tasks, avoiding gradient conflicts and training oscillations caused by traditional fixed weight combinations. The adaptive joint optimization strategy constructs a total loss function based on a bidirectional loss function and a standard cross-entropy loss function, and uses this total loss function to achieve multi-task learning.

[0101] For example, the total loss function As shown in the following formula (10): (10); in, These are the weighting coefficients. It is a bidirectional loss function. The standard cross-entropy loss function is used. The weight coefficient λ adopts a dynamic adjustment strategy: an adaptive adjustment mechanism is used during the training phase. In the early stage of training, a larger weight coefficient, such as 0.8, is set to prioritize ensuring that bidirectional reconstruction contrastive learning can build a stable and semantically discriminative feature space, so that the network traffic feature vectors and host log feature vectors of the same source event are closely clustered in the embedding space, while the feature vectors of different source events are far apart. As training progresses, the contrastive learning weight coefficient is gradually reduced to a smaller value, such as 0.5, to strengthen the classifier's ability to distinguish abnormal patterns. This achieves a progressive optimization process from coarse-grained feature alignment to fine-grained anomaly recognition, ultimately ensuring that the trained target recognition model has both good multimodal feature fusion capabilities and maintains high-precision anomaly detection performance.

[0102] In this embodiment, the initial classifier in the preset large language model is trained based on the first and second embedding vectors in each embedding vector sample pair to obtain a target recognition model including a target classifier. This enables the target classifier in the target recognition model to instantly identify whether the current communication behavior is normal business behavior, abnormal business behavior, or whether the current communication data matches when identifying the current communication behavior of the industrial control host.

[0103] In some embodiments, a preset large language model is trained to obtain a target recognition model that is a large language model fine-tuned for the business behavior of the industrial control host. The fine-tuned large language model refers to the model after adjusting the low-rank weight matrix of the large language model.

[0104] For example, the above training process, namely the adaptive joint optimization strategy, serves as the driving mechanism for generating the fine-tuned large language model. During the training loop, the algorithm uses the total loss function to calculate the gradient and specifically updates the weight matrix parameters and projection network parameters in the LoRA module through backpropagation. In short, the processing logic of the training process, such as loss calculation and backpropagation, is the means, while the fine-tuned large model is the result of this process converging after multiple iterations, i.e., obtaining the target recognition model, which enables the target recognition model to understand and detect anomalies in heterogeneous data in the industrial control field.

[0105] In some embodiments, traditional large language model fine-tuning methods typically employ full-parameter fine-tuning or random layer selection strategies. This coarse-grained fine-tuning approach not only incurs huge computational overhead and storage burden, but also easily undermines the generalization ability of pre-trained models, leading to overfitting and feature representation degradation in industrial control network anomaly detection tasks.

[0106] Therefore, in the process of fine-tuning the preset large language model, the embodiments of this application utilize a specific top-level set of the large model encoder. LoRA adapters are precisely inserted to fine-tune large language models. Because top-level features have higher abstraction and task relevance, they significantly improve the ability to capture complex semantic relationships between industrial control network traffic and host logs; therefore, the parameters... Values ​​should be selected from the range of layers 2, 4, or 6.

[0107] For example, in each selected level, embodiments of this application implement the core linear mapping module of the self-attention mechanism. and the gated feedforward network module of the multilayer sensor LoRA modification is implemented. In the core linear mapping module, optimizations are made from the perspective of information flow processing, and in the self-attention module... , , The three weight matrices are responsible for mapping the input features into three different representation spaces: query, key, and value. These three mappings directly determine the model's ability to model the relationships between different semantic components in the industrial control network data. The output projection matrix controls the final integration of attention information. LoRA adaptation of these four key components can precisely adjust the model's sensitivity to abnormal patterns in the industrial control network. In gated feedforward networks, The matrix controls the selective transmission of information through a gating mechanism. The matrix is ​​responsible for the nonlinear transformation and dimensional expansion of features. The matrix enables feature compression and output mapping. This triplet constitutes the core pathway for nonlinear feature learning in the Transformer architecture. Modifying it with LoRA enables the model to learn more accurate nonlinear transformation functions for specific abnormal patterns of industrial control networks while maintaining pre-trained knowledge.

[0108] By selectively modifying the seven key weight matrices mentioned above, this embodiment of the application can achieve comprehensive optimization of the two main information processing channels in the Transformer architecture—attention calculation and feedforward processing—with minimal parameter overhead. This avoids parameter redundancy and computational burden caused by modifying all linear layers, while ensuring precise control of the core functional modules of the model. As a result, it achieves the optimal balance between parameter efficiency and performance improvement in the anomaly detection task of industrial control networks, while also ensuring the stability of the fine-tuning process and the reliability of the model output.

[0109] In some embodiments, this application demonstrates freezing the original weight matrix and only adjusting the incremental matrix. Training is performed to achieve the technical goal of efficient fine-tuning. Since the effective changes of the weight matrix of most deep learning models during the fine-tuning process are often concentrated in a relatively low-dimensional subspace, the embodiments of this application use a low-rank decomposition form to parameterize the weight increment. The weight increment is represented by the following formula (11): (11); Among them, matrix As a dimensionality reduction mapper, it compresses the input features into a latent space of rank r; matrix For dimensionality upscaling mappers, the latent features are restored to the original output dimension, with the rank parameter... Scaling factor used to ensure parameter efficiency This is used to precisely control the intensity of the LoRA effect.

[0110] During the forward propagation process, the embodiments of this application are based on incremental learning, while maintaining the original pre-trained model. Based on the output, only add task-specific incremental correction items. This avoids the catastrophic forgetting problem that may result from direct modification of the original weights, and achieves accurate adaptation to new tasks. The feature vector output by the corresponding linear layer is expressed by the following formula (12): (12); Where y is the feature vector output by the linear layer, and x is the input feature vector (such as semantic embeddings of network traffic or logs); W is the original weight matrix frozen in the pre-trained large language model, used to preserve the model's general language understanding ability; ΔW is the weight increment matrix learned for the industrial control network anomaly detection task; A∈ r×din is a trainable dimensionality reduction mapping matrix used to compress high-dimensional input features into a low-dimensional latent space; B∈ dout×r is a trainable up-dimensional mapping matrix used to restore low-dimensional features to the output dimension. Together, they constitute a low-rank decomposition representation of ΔW. s is a scaling factor used to adjust the influence of the low-rank adapter on the final output of the model, ensuring the numerical stability of the fine-tuning process.

[0111] In other embodiments, under the condition of limited abnormal samples in the industrial control network, in order to prevent the large language model from overfitting to specific abnormal patterns, this application uses LoRA dropout to increase the randomness and generalization ability of training. This method obtains the optimized calculation formula (13) by applying a random mask following a Bernoulli distribution to the intermediate calculation result Ax: (13); Where m is a random mask vector following a Bernoulli distribution, its elements consisting of 0s and 1s; ⊙ is the Hadamard product, i.e., element-wise multiplication. (m⊙A) represents the random discarding strategy applied to the output features of the reduced-dimensional projection matrix A. y is the feature vector output by the linear layer, W is the frozen original weights, x is the input feature vector, s is the scaling factor, and B is the increased-dimensional matrix. By introducing the random mask vector m, this embodiment of the application randomly blocks the transmission of some low-rank features during training, effectively preventing the large language model from overfitting to specific industrial control network traffic patterns or log noise, thereby improving robustness against unknown attack behaviors.

[0112] In some embodiments, by embedding targeted low-rank adapters into the top-level attention module and the gated multilayer perceptron (MLP) module of the large language model, the model's ability to detect abnormal behavior in industrial control networks is improved in a parametrically efficient manner. Regarding the modification of the attention mechanism, this application introduces a low-rank matrix... , and To augment the pre-trained query, key, and value projection matrices , and This enables the large language model to learn the unique correlation patterns of industrial control network data, and the augmented output is shown in the following formula (14): (14); in, For the augmented output, Used to generate precise query representations for specific fields of industrial control protocols, capturing the inherent association patterns of industrial control network data. and Optimize the mapping relationship between the key space and the value space separately. and These are preset augmentation parameters, which enable the attention mechanism to more effectively identify and respond to abnormal communication patterns in industrial control networks, thereby improving the ability to detect abnormal communication patterns.

[0113] In some embodiments, in MLP structure optimization, the output projection is achieved through... The aggregated attention information is processed in a task-specific manner to ensure the effective transmission of attack abnormal behavior signals. The final output tensor after processing by the attention module is expressed by the following formula (15): (15); in, This is the final output tensor after processing by the attention module. This is a weighted attention aggregation result calculated based on query, key, and value, which includes captured contextual information related to industrial control data. This refers to the frozen original output projection weight matrix in the pre-defined large language model, which is responsible for integrating the results of multi-head attention. This is a low-rank increment matrix specifically trained for the output projection layer. This is achieved by introducing [a specific method / mechanism] into this layer. The pre-defined large language model can recalibrate the information flow output by the attention mechanism to ensure that key semantic features about "abnormal business behavior of abnormal attacks" or "violations" are effectively enhanced and preserved when passed to subsequent layers.

[0114] In other embodiments, for the gated MLP structure with a preset large language model, the present invention further modifies the up projection and gated projection in the gated MLP structure simultaneously, introducing low-rank matrices respectively. and This achieves the transformation of the top projection and gated projection. The transformed top projection is expressed by the following formula (16): (16); Where U is the output vector of the upper projection layer. The input feature vector (usually derived from the output of the attention layer) is the input feature vector that enters the feedforward network layer. The original weight matrix of the upper projection in the pre-defined large language model is preserved by parameter freezing to maintain its basic feature selection capability. It is a low-rank incremental matrix corresponding to the projection, learned to adapt to the data distribution unique to the industrial control field.

[0115] The modified gated projection is expressed by the following formula (17): (17); Where G is the output vector of the gated projection, which determines the pass-through rate of information in subsequent nonlinear activations. The input feature vector is used to enter the feedforward network layer. To preserve the basic feature selection capability of the original gating weight matrix of the pre-defined large language model, parameter freezing is used. It is a low-rank incremental matrix corresponding to the gated projection, learned to adapt to the data distribution unique to the industrial control field. Through optimization... The embodiments of this application can finely adjust the sensitivity of the large language model to specific industrial control protocol features or abnormal behavior patterns, and selectively amplify feature signals related to security threats by using a gating mechanism, while suppressing irrelevant background noise.

[0116] This enables large language models to learn domain-specific transformation patterns of industrial control network data. Selective enhancement of anomalous features is achieved through a gating mechanism. Then, the output vector G of the gating projection is subjected to nonlinear activation processing to obtain σ(G), which is then subjected to Hadamard product operation with the output vector U of the upper projection layer to obtain the enhanced feature M=σ(G)⊙U. Based on the enhanced feature and the improved lower projection, the final mapping is performed to obtain the final output feature vector. The final output feature vector is obtained by the following formula (18): (18); in, The projection matrix under the pre-trained baseline. The downprojection matrix is ​​dynamically adjusted to ensure that features are correctly projected into the output space of the anomaly detection task. To enhance features.

[0117] In some embodiments, regarding the dual encoder configuration used during training and fine-tuning, the network traffic encoder takes into account the significant differences between network traffic data and host log data in data distribution, semantic representation, and anomaly behavior. and host log encoder By configuring independent low-rank weight matrix parameters A and B, intermodal interference that may be caused by weight sharing is avoided, ensuring optimal extraction and fusion of features from the two modalities.

[0118] In some embodiments, in terms of parameter efficiency design, the number of new parameters added to a single linear layer is expressed by the following formula (19): (19); in, This represents the total number of additional training parameters introduced by the LoRA adapter for the corresponding linear layers. This total number of additional training parameters is the only parameter that needs to be updated during the fine-tuning training phase of the pre-defined large language model. The rank of the low-rank weight matrix is ​​a hyperparameter set in the embodiments of this application (e.g., ΔW=s). The common dimension of weight matrices A and B in BA), its value is much smaller than the minimum value of the original dimension (r). min( , (), used to control the compression ratio parameter, and Let represent the input and output feature dimensions of the linear layer weight matrix in the original pre-trained model, respectively. Equation (19) reveals the mathematical principle of the LoRA mechanism: by decomposing the huge weight increment matrix into a matrix of shape r× The dimension-reduced matrix A and the shape are The increased-dimensional matrix B of ×r ensures that the number of new parameters is only linearly related to the rank r, rather than proportional to the size of the original matrix. This strongly proves that this application achieves accurate adaptation to the anomaly detection task of industrial control network with minimal parameter overhead.

[0119] In the embodiments of this application, compared with the complexity of full parameter fine-tuning, this application significantly reduces the computational overhead, enabling industrial control systems to deploy high-performance anomaly detection models with limited edge computing resources.

[0120] In some embodiments, during the training and fine-tuning process of a preset large language model, for the loss function of the fine-tuning, this application embodiment continues to use the total loss based on the above-described adaptive joint optimization strategy. (10) The contribution of contrastive learning and classification learning is balanced by dynamically adjusting the weight coefficient λ.

[0121] In the specific implementation of gradient updates, for any linear layer in the LoRA module...

[0122] When the upstream gradient is The input is The gradient of the LoRA parameter is calculated using the chain rule. The gradient of the LoRA parameter is given by the following formula (20): (20); Where s is the scaling factor and r is the low-rank dimension. The gradient formula (20) shows that the LoRA module achieves efficient updating of the low-rank weight matrix parameters A and B through low-rank decomposition, enabling the pre-set large language model to perform accurate feature learning and classification boundary optimization for abnormal patterns of industrial control networks while maintaining the original semantic understanding ability.

[0123] In some embodiments, the parameters A and B of the trained LoRA low-rank weight matrix are fused into the original weights W of the preset large language model using the following formula (21): (twenty one); Where s is a learnable scaling factor used to control the adaptation strength, and low-rank weight matrices are discarded after merging. This achieves zero latency for single-path forward inference.

[0124] Figure 5 This is a flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application. (Combined with...) Figure 5 As shown in the figure, another method for identifying abnormal behavior in industrial networks provided in this application includes the following steps: S201. Obtain the target text information pair from the industrial control host.

[0125] In some embodiments, the target text information pair includes network text information and log text information; the network traffic text information is used to indicate structured text about network traffic data generated by the current communication behavior of the industrial control host, and the log text information is used to indicate structured text about host log data generated by the current communication behavior.

[0126] S202. Input the target text information into the target recognition model to obtain the recognition result output by the target recognition model.

[0127] In some embodiments, the network text information and log text information are converted into corresponding high-dimensional and comparable vector representations, namely network traffic embedding vectors and log embedding vectors, through the embedding layer of the target recognition model.

[0128] In some embodiments, the target classifier in the target recognition model generates a recognition result based on the transformed network traffic embedding vector and log embedding vector, and the recognition result indicates whether the current communication behavior of the industrial control host is normal business behavior, abnormal business behavior, or data mismatch.

[0129] In this embodiment of the application, the target recognition model trained based on the first and second text information in the training sample set can analyze and identify the features of new abnormal behaviors in real time, thereby improving the efficiency of identifying new abnormal behaviors.

[0130] Figure 6 This is a flowchart illustrating another method for identifying abnormal behavior in industrial networks provided in this application. (Combined with...) Figure 6 As shown, in step S202 above, inputting the target text information into the target recognition model and obtaining the recognition result output by the target recognition model can be implemented through the following steps: S2021. Convert the target text information pairs into corresponding target embedding vector pairs.

[0131] In some embodiments, the network text information and log text information in the target text information pair are converted into corresponding target embedding vector pairs through the embedding layer of the target recognition model.

[0132] The target embedding vector pair includes a network traffic embedding vector and a log embedding vector. The network traffic embedding vector is used to indicate the semantic encoding of the network traffic data generated by the current communication behavior, and the log embedding vector is used to indicate the semantic encoding of the host log data generated by the current communication behavior.

[0133] In some embodiments, the target recognition model calculates the similarity between the network traffic embedding vector and the log embedding vector, which is used to indicate the geometric proximity of the semantically aligned network traffic embedding vector and the log embedding vector in the high-bit semantic space.

[0134] Here, similarity is a quantitative value calculated using the cosine similarity formula, typically ranging from -1 to 1. It measures the degree of overlap between the projected network traffic embedding vector and the log embedding vector in the vector space direction. Physically, it represents the semantic consistency between two heterogeneous data sets; that is, the target recognition model determines the extent to which these two sets of data describe the same communication event. Technically, "from the same communication action event" is a prerequisite for constructing sample pairs, while "similarity" is an optimization metric for training the target recognition model. The bidirectional contrastive learning mechanism in this application drives the model to learn deep semantic relationships across modalities by maximizing the computational similarity of data pairs from "the same communication action event" and minimizing the computational similarity of data pairs from "different communication events," thereby achieving the alignment of heterogeneous data.

[0135] S2022. In response to the similarity between the network traffic embedding vector and the log embedding vector being less than or equal to a preset matching threshold, generate an identification result indicating that the network traffic embedding vector and the log embedding vector do not match.

[0136] In some embodiments, after calculating the similarity between the network traffic embedding vector and the log embedding vector, the similarity between the network traffic embedding vector and the log embedding vector is compared with a preset matching threshold.

[0137] For example, if the similarity between the network traffic embedding vector and the log embedding vector is less than or equal to a preset matching threshold, it is determined that there is a contextual inconsistency or semantic mismatch between the current network traffic data and the host log data. The data batch is then rejected and marked as a "mismatch" event. Based on the marked "mismatch" event, an identification result indicating that the network traffic embedding vector and the log embedding vector do not match is generated.

[0138] By rejecting and filtering mismatched network traffic embedding vectors and log embedding vectors, irrelevant inputs caused by timestamp offsets, misaligned data sources, or low event correlation can be effectively filtered out. This significantly avoids subsequent target classifiers making invalid or misleading judgments based on erroneous or irrelevant multimodal information, thereby greatly improving the reliability and robustness of the overall detection and recognition process.

[0139] S2023. In response to the similarity between the network traffic embedding vector and the log embedding vector being greater than the matching threshold, the network traffic embedding vector and the log embedding vector are input into the target classifier in the target recognition model to obtain the recognition result output by the target classifier.

[0140] In some embodiments, if the similarity between the network traffic embedding vector and the log embedding vector is greater than a preset matching threshold, the multimodal anomaly discrimination stage can only proceed after confirming that the data batch has a high degree of contextual consistency and semantic relevance.

[0141] For example, in the multimodal anomaly detection stage, the target recognition model concatenates the validated, semantically relevant network traffic embedding vector with the log embedding vector to generate a target fusion feature vector. This target fusion feature vector indicates different dimensions of the current communication behavior.

[0142] For example, after obtaining the target fusion feature vector, the target fusion feature vector is then input into the target classifier. The target classifier utilizes the sample fusion feature vector, which is highly discriminative and semantically related and trained under the contrastive learning paradigm, to accurately identify and distinguish between normal business behavior and abnormal business behavior, and finally outputs an identification result indicating whether the current communication behavior is normal business behavior or attack abnormal behavior.

[0143] In this embodiment, by filtering the embedding vectors based on the similarity of each embedding vector, irrelevant inputs caused by timestamp offsets, misaligned data sources, or low event correlation can be effectively avoided. This significantly prevents the subsequent target classifier from making invalid or misleading judgments based on erroneous or irrelevant multimodal information, thereby improving the reliability and robustness of the overall detection process. Moreover, by training and fine-tuning the target recognition model, fast and accurate abnormal behavior classification can be achieved, ultimately achieving accurate, stable, and efficient identification of abnormal behaviors in industrial control networks.

[0144] Figure 7 A system architecture diagram provided for an embodiment of this application includes: a data preprocessing module 701, a multimodal encoder module 702, a large model fine-tuning module 703, and a recognition module 704.

[0145] The data preprocessing module 701 is used to collect network traffic and host log data from the industrial control network, extract features from the network traffic and host log data, and convert them into corresponding traffic description information and log description information as text information. This enables the construction of text information sample pairs in the training sample set based on the traffic description information and log description information as text information, which are then applied to the above steps S1011 to S1013.

[0146] The multimodal encoder module 702 is used to encode the traffic description information and log description information, which are text information, respectively, using corresponding encoders to generate embedded vector samples with added action tags corresponding to the text information samples. For example, if the action tag is "normal behavior", the text information sample is "normal behavior". and actions labeled as abnormal behavior ,in, This is the first embedding vector used to indicate the semantic encoding of network traffic data. The second embedding vector, used to indicate the semantic encoding of log data, is an embedding vector sample pair applied to step S1021 above.

[0147] The large model fine-tuning module 703 is used to train the preset large language model based on the embedded vector sample pairs corresponding to the text information samples, i.e. LoRA fine-tuning. During LoRA fine-tuning, the parameter freezing technique is used to fix the main parameters of the preset large language model, and only the top projection layer and classification layer are trained. Low-rank adaptive LoRA technique can be selectively introduced for fine-tuning to obtain the target recognition model, which is applied to the above step S1022.

[0148] The recognition module 704 is used to input the target text information of the current industrial control host into the target recognition model, and convert it into the corresponding embedding vector through the industrial control network behavior decoder trained in the target recognition model. The target classifier in the target recognition model outputs the recognition result based on the converted embedding vector. If the network data and log data do not match, it is applied to the above steps S201 to S202.

[0149] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0150] This application embodiment can divide the industrial network abnormal behavior identification device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0151] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device includes: a processor 802, a communication interface 803, and a bus 804. Optionally, the electronic device may also include a memory 801.

[0152] Processor 802 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 802 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 802 may also be a combination of functions implementing computing capabilities, such as a combination including CPU0 and CPU1, a DSP, and a microprocessor.

[0153] The communication interface 803 includes a receiving unit and a transmitting unit, and is used to connect with other devices via a communication network. This communication network can be Ethernet, a wireless access network, a wireless local area network (WLAN), etc.

[0154] The memory 801 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0155] As one possible implementation, the memory 801 can exist independently of the processor 802. The memory 801 can be connected to the processor 802 via a bus 804 and is used to store instructions or program code. When the processor 802 calls and executes the instructions or program code stored in the memory 801, it can implement the industrial network abnormal behavior identification method provided in this embodiment of the invention.

[0156] In another possible implementation, the memory 801 can also be integrated with the processor 802.

[0157] The 804 bus can be an extended industry standard architecture (EISA) bus, etc. The 804 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0158] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0159] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the aforementioned computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The aforementioned computer-readable storage medium can also be an external storage device of the aforementioned electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the aforementioned electronic device. Further, the aforementioned computer-readable storage medium can include both internal storage units and external storage devices of the aforementioned electronic device. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0160] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to execute any of the industrial network abnormal behavior identification methods provided in the above embodiments.

[0161] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for identifying abnormal behavior in industrial networks, characterized in that, include: Obtain a training sample set, which includes text information sample pairs with added behavioral labels; The text information sample pair includes first text information and second text information; the first text information is used to indicate structured text about network traffic data generated by the communication behavior of the industrial control host, the second text information is used to indicate structured text about the log data of the industrial control host, and the behavior tag is used to indicate whether the communication behavior is normal business behavior or abnormal business behavior. The preset large language model is trained based on the training sample set to obtain a target recognition model for identifying whether the communication behavior of the industrial control host is abnormal; wherein, the target recognition model is a large language model fine-tuned for the business behavior of the industrial control host, and the fine-tuned large language model refers to the model after adjusting the low-rank weight matrix of the large language model.

2. The method according to claim 1, characterized in that, The acquisition of the training sample set includes: Acquire first communication data and second communication data; the first communication data is used to indicate the network traffic characteristics generated by the communication behavior of the industrial control host, and the second communication data is used to indicate the host log data characteristics generated by the communication behavior; By using a preset mapping rule, the first communication data and the second communication data are converted into corresponding first text information and second text information; wherein, the preset mapping rule is used to convert the field information in the first communication data and the second communication data into corresponding structured text description information; Based on the first text information and the second text information, construct the text information sample pairs in the training sample set.

3. The method according to claim 1, characterized in that, The step of training a preset large language model based on the training sample set to obtain a target recognition model includes: Based on the first text information and the second text information, generate an embedding vector sample pair corresponding to the text information sample; The embedded vector sample pair includes a first embedded vector and a second embedded vector; the first embedded vector is used to indicate the semantic encoding of the network traffic data, and the second embedded vector is used to indicate the semantic encoding of the log data. Based on the embedded vector sample pairs corresponding to the text information samples, the preset large language model is trained to obtain the target recognition model.

4. The method according to claim 3, characterized in that, The step of generating embedding vector sample pairs corresponding to the text information samples based on the first text information and the second text information includes: The first text information and the second text information are semantically encoded to obtain corresponding first semantic vectors and second semantic vectors; wherein, the first semantic vector is used to indicate the vector representation of network traffic data, and the second semantic vector is used to indicate the vector representation of host log data; the first semantic vector and the second semantic vector are in the same high-dimensional semantic space. The first semantic vector and the second semantic vector are subjected to projection transformation to obtain the first embedding vector and the second embedding vector after projection transformation, so that the first embedding vector and the second embedding vector after projection transformation are semantically aligned in the same high-dimensional semantic space. Based on the first embedding vector and the second embedding vector, the embedding vector sample pair is constructed.

5. The method according to claim 4, characterized in that, The step of training the preset large language model based on the embedded vector sample pairs corresponding to the text information samples to obtain the target recognition model includes: Based on the first and second embedding vectors in the semantically aligned embedding vector sample pair, a corresponding sample fusion feature vector is generated; the sample fusion feature vector is used to indicate different dimensional features of the communication behavior corresponding to the embedding vector sample pair. Based on the sample fusion feature vector with the added behavior labels, the initial classifier in the preset large language model is trained to obtain the target classifier in the target recognition model.

6. The method according to claim 5, characterized in that, The sample fusion feature vector includes a first feature vector, a second feature vector, a third feature vector, and a fourth feature vector; Wherein, the first feature vector is used to indicate the first embedding vector, the second feature vector is used to indicate the second embedding vector, the third feature vector is used to indicate the semantic difference information between the first embedding vector and the second embedding vector, and the fourth feature vector is used to indicate the language association information between the first embedding vector and the second embedding vector.

7. The method according to claim 2, characterized in that, The first communication data includes first traffic data, second traffic data, third traffic data, and fourth traffic data; Wherein, the first traffic data is used to indicate the sending time and rate characteristics of the network traffic data packets generated by the communication behavior; the second traffic data is used to indicate the size distribution characteristics of the transmitted content of the network traffic data packets; the third traffic data is used to indicate the communication protocol characteristics; and the fourth traffic data is used to indicate the application content characteristics of the network traffic data packets. The second communication data includes first log data, second log data, third log data, and fourth log data; Wherein, the first log data is used to indicate the host status and status recording time characteristics generated by the communication behavior; the second log data is used to indicate the network activity characteristics of the industrial control host; the third log data is used to indicate the storage activity characteristics of the industrial control host; and the fourth traffic data is used to indicate whether the configuration file of the industrial control host has been changed.

8. A method for identifying abnormal behavior in industrial networks, characterized in that, include: Obtain target text information pairs from the industrial control host, the target text information pairs including network text information and log text information; the network traffic text information is used to indicate the structured text about network traffic data generated by the current communication behavior of the industrial control host, and the log text information is used to indicate the structured text about host log data generated by the current communication behavior; The target text information is input into the target recognition model to obtain the recognition result output by the target recognition model. The recognition result is used to indicate whether the current communication behavior is an abnormal business behavior. The target recognition model is trained based on the method described in any one of claims 1 to 7.

9. The method according to claim 8, characterized in that, The step of inputting the target text information into the target recognition model to obtain the recognition result output by the target recognition model includes: The target text information pair is converted into a corresponding target embedding vector pair; wherein, the target embedding vector pair includes a network traffic embedding vector and a log embedding vector, the network traffic embedding vector is used to indicate the semantic encoding of the network traffic data generated by the current communication behavior, and the log embedding vector is used to indicate the semantic encoding of the host log data generated by the current communication behavior; In response to the similarity between the network traffic embedding vector and the log embedding vector being less than or equal to a preset matching threshold, an identification result is generated to indicate that the network traffic embedding vector and the log embedding vector do not match; wherein, the similarity is used to indicate the degree of similarity between the network traffic embedding vector and the log embedding vector after semantic alignment; In response to the similarity between the network traffic embedding vector and the log embedding vector being greater than the matching threshold, the network traffic embedding vector and the log embedding vector are input into the target classifier in the target recognition model to obtain the recognition result output by the target classifier.

10. The method according to claim 9, characterized in that, The step of inputting the network traffic embedding vector and the log embedding vector into the target classifier in the target recognition model to obtain the recognition result output by the target classifier includes: Based on the network traffic embedding vector and the log embedding vector, a target fusion feature vector is generated; wherein, the target fusion feature vector is used to indicate different dimensional features of the current communication behavior; The target fusion feature vector is input into the target classifier to obtain the identification result used to indicate whether the current communication behavior is an abnormal business behavior.

11. An electronic device, characterized in that, The device includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the industrial network abnormal behavior identification method as described in any one of claims 1 to 7, or the industrial network abnormal behavior identification method as described in any one of claims 8 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the industrial network abnormal behavior identification method as claimed in any one of claims 1 to 7, or the industrial network abnormal behavior identification method as claimed in any one of claims 8 to 10.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the industrial network abnormal behavior identification method as described in any one of claims 1 to 7, or the industrial network abnormal behavior identification method as described in any one of claims 8 to 10.

Images