An industrial control malicious code detection method based on content block
By optimizing the detection of malicious code in industrial control systems using a content-block-based hash algorithm and Hamming distance, the problem of data silos was solved, the model performance and detection efficiency were improved, data privacy and security were ensured, and the quality and types of the dataset were expanded.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUNAN KUANGAN NETWORK TECH CO LTD
- Filing Date
- 2023-03-24
- Publication Date
- 2026-07-03
Smart Images
Figure CN116361796B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of industrial control technology, and in particular relates to a method for detecting malicious code in industrial control systems based on content segmentation. Background Technology
[0002] Malicious code detection has always been a crucial component of industrial control system (ICS) security. In recent years, with the increasing variety of malicious code and the growing volume of business, current mainstream manual and software detection methods have shown limitations in efficiency and detection rates. Therefore, training malicious detection code using machine learning has become a new and popular technological direction. However, due to the stronger data privacy requirements and fewer publicly available datasets in the ICS field, the data quality and quantity in machine learning cannot be guaranteed, resulting in poor overall model performance.
[0003] Malicious code refers to computer code that is intentionally created or configured to pose a danger or potential threat to a network or system. Therefore, malicious code detection has always been an important research area in industrial control system (ICS) security. However, traditional malicious code detection methods mostly rely on manual analysis and software identification. Therefore, once new types of malicious code emerge that are unknown to detection personnel, they can easily bypass detection, causing significant impacts on ICS security. Consequently, in recent years, there has been an increasing number of methods for using machine learning to train models to detect malicious code.
[0004] As we all know, the quality of a model is determined not only by its structural rationality but, more importantly, by the quality of its training data. Federated learning is no exception. If each client's dataset is well-partitioned, the overall performance of the federated learning model will be excellent. However, due to commercial value, legal and regulatory requirements, and other reasons, clients with better datasets are unwilling to share their data with other clients. Other clients, in turn, may not achieve satisfactory training results due to flaws in their datasets. This results in significant performance differences between models trained by different clients, with models trained by clients using inferior datasets failing to comprehensively detect malicious code.
[0005] Current issues related to isolated systems and malware detection include:
[0006] The main methods of federated learning for training malicious code have not proposed an effective solution to data silos, resulting in different detection capabilities and performance of the models trained by different clients.
[0007] Current machine learning methods for detecting malware mainly involve converting malware into grayscale images and using deep learning to generate detection models. However, if new types of malware emerge, regenerating the dataset becomes extremely cumbersome.
[0008] While existing technologies guarantee data privacy, the overall communication and computing processes are relatively complex.
[0009] Wang Y, Tong Y, Shi D, et al., in the non-patent literature "An efficient approach for cross-silo federated learning to rank[C] / / 2021 IEEE 37th International Conference on Data Engineering (ICDE).IEEE,2021:1128-1139", proposed a cross-silo federated ranking learning algorithm, which optimizes text queries through ranking learning, that is, it uses machine learning to rank the importance of keywords of a word. The steps of this method are as follows:
[0010] For both the querying party and the text party, a query is performed once per target document. First, the keywords to be queried are hashed using a function to obtain a one-dimensional vector of the form (n,1), with n rows and one column. Each value in the vector represents the column number in the matrix to be queried; for example, a value of 2 in the third row means the value in the second column of the third row of the target matrix is to be queried. To ensure data privacy and prevent attackers from obtaining the original keywords by analyzing the hash function, the querying party perturbs and adjusts the vector values of random row numbers before sending the adjusted vector to the text party. The text party also hashes its own text. Using two hash functions H(a) and G(b), a two-dimensional matrix is generated to represent the correlation between any two words, with each cell representing the correlation between one word and another keyword. The text party receives the query vector from the querying party, retrieves the values in the matrix based on the vector values, and obtains another one-dimensional vector of the form (n,1). To prevent attackers from obtaining the original matrix content by analyzing the obtained vector and thus acquiring the text data... The text side also modifies the results: it adds a specific value d to the value of each row in the vector to change the numerical value. This way, if d is unknown to the attacker, the matrix content cannot be deduced. After receiving the query results, the query side extracts the relevance data based on the row numbers of the vector that have not been disturbed, and then obtains the median of all the data as the relevance of this query keyword in the specific text.
[0011] This approach is commonly used in search engines, thus possessing significant economic and technological value for internet companies. However, for this very reason, ranking learning within the federated learning framework also faces the problem of data silos. Summary of the Invention
[0012] In view of this, this invention proposes a reliable privacy protection method that, even with a small dataset, allows interaction with other users through content-block-based hash calculations to expand the dataset and improve the model's recognition rate for various types of malicious code. Therefore, the purpose of this invention is to: utilize a hash algorithm that can express data characteristics to quickly process datasets while fully guaranteeing data privacy; employ clustering algorithms to accelerate multi-party data query speeds, significantly improve the speed of malicious code dataset generation and preprocessing, and enhance dataset quality. Ultimately, this aims to improve the performance of the malicious code model and expand the types of malicious code the model can detect.
[0013] This invention discloses a method for detecting malicious code in industrial control systems based on content segmentation, comprising the following steps:
[0014] S1: Each user handles their own malicious code sample;
[0015] S2: Read the file by file data stream, using two parallel sliding data windows, one to read odd-numbered bytes and the other to read even-numbered bytes;
[0016] S3: For each bit read, calculate the hash within the window. If it equals the special hash value used for boundary determination, then end the current slide.
[0017] S4: After the sliding ends, sum the hash values of every 16 slidings and use the result as the total hash.
[0018] S5: Repeat steps S2-S4 until the entire file stream has been read, completing the current file's block division process;
[0019] S6: Extract the hash value of the distinctive malicious code segment of each malicious code file, use Hamming distance to determine the similarity of each malicious code hash value, and generate several clusters;
[0020] S7: Each user sends a query request, sending the hash value of their malicious code to other users;
[0021] S8: Other users compare the hash sent with their own generated clusters to obtain the specific type of malicious code being queried, and then provide feedback to the querying user.
[0022] S9: Based on the returned query results, the user optimizes the classification of the current code, expands their dataset, and finally begins training and testing the model according to the existing malicious code model design.
[0023] Furthermore, according to the formula hash[i]=hash[i-2]<<1+Gear[byte[i]], the hash within the window is calculated once, where i represents the current position of the sliding window, hash represents the hash value at the current position, Gear represents a hash table containing 256 special hash values, and byte represents the decimal number of the byte at the current position.
[0024] Furthermore, the Hamming distance formula is as follows:
[0025] d(x,y)=∑x[i]⊕y[i)
[0026] Where x[i] and y[i] represent the hash values of two distinctive malicious code segments.
[0027] The beneficial effects of this invention are as follows:
[0028] Using this invention, the queryer can obtain the correlation between a certain keyword and a certain text, which fully protects the data privacy of both parties, and at the same time provides the necessary training data for ranking learning.
[0029] This invention is the first to apply content-based chunking technology to the field of malware detection. By using chunking, attack fragments within malware are generated into unique hash values, which are then used as the basis for judging code similarity. Leveraging the irreversible nature of hashing, the data privacy and security of all data owners are guaranteed. This is the first time that a content-based chunking hash algorithm has been used to process malware. Compared to current deep learning methods for generating grayscale images of malware, the hash algorithm significantly accelerates data preprocessing while also ensuring the quality of the dataset.
[0030] This invention is the first to propose using hashing to determine code similarity, thereby optimizing the training set. No such method has been used in existing malware model training. By employing a reliable hashing algorithm, the privacy of malware source file data is guaranteed, solving the critical data silo problem in federated learning and effectively improving the performance of all client-trained models.
[0031] Since it is necessary to compare differences between various files in both datasets, this invention combines hash results with Hamming distance, transforming a large number of malicious code libraries into several clusters with distinct characteristics. By comparing the similarity with the central feature hash of the clusters, and then specifically comparing it with the code within the cluster, this comparison approach greatly improves query speed. Combining Hamming distance to generate malicious code clusters reduces the n-to-n complexity of cross-island queries to 1-to-n queries, significantly reducing computational overhead and further optimizing the communication process, thereby accelerating dataset generation. Attached Figure Description
[0032] Figure 1 The present invention provides a flowchart of hash calculation based on content segmentation;
[0033] Figure 2 The process of expanding the malicious code dataset in this invention. Detailed Implementation
[0034] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.
[0035] This invention improves upon the algorithm provided in the non-patent literature "An efficient approach for cross-silo federated learning to rank," and references... Figure 1 The content-block-based hash calculation process is as follows: The file stream is read in parallel using two windows, one for odd-even hashing. Each window slides 16 bits before calculating a hash. Finally, all hashes are combined to obtain the final fingerprint hash. The steps of the method are as follows:
[0036] Each user begins processing their own malicious code sample;
[0037] The file is read as a data stream, using two parallel sliding data windows, one for reading odd-numbered bytes and the other for reading even-numbered bytes.
[0038] For each bit read, the formula hash[i] = hash[i-2] << 1 + Gear[byte[i]] is used, where i represents the current position of the sliding window, hash represents the hash value at that position, Gear represents a hash table containing 256 special hash values, and byte represents the decimal number of the byte at the current position. The hash within the window is calculated once; if it equals the special hash value used for boundary checks, the current sliding ends. After the sliding ends, the hash values from every 16 slides are summed, and the result is the total hash. A hash algorithm maps a binary value of arbitrary length to a shorter, fixed-length binary value; this smaller binary value is called the hash value. A hash value is a unique and extremely compact numerical representation of a piece of data. If a piece of plaintext is hashed and even a single letter is changed, subsequent hashes will produce different values. Finding two different inputs that hash to the same value is computationally impossible, so the hash value of data can verify the integrity of the data.
[0039] Repeat steps 2-4 until the entire file stream has been read, completing the file segmentation process. This segmentation technique generates a unique hash value from the malicious code's attack fragments, which is then used to determine code similarity. Leveraging the irreversible nature of hashing, it ensures the data privacy and security of all data owners. This is the first time a content-segmented hash algorithm has been used to process malicious code. Compared to current deep learning methods for generating grayscale images of malicious code, this hash algorithm significantly accelerates data preprocessing while maintaining good dataset quality.
[0040] The hash value of the distinctive malicious code segment of each malicious code file is extracted, and the similarity of each malicious code hash value is determined using Hamming distance to generate several clusters. The Hamming distance is calculated by comparing the number of different bits in two binary strings. In some embodiments, cosine similarity is also used for similarity measurement. Since it is necessary to compare differences between files in both datasets, this invention combines the hash results with Hamming distance, transforming a large malicious code library into several clusters with distinct characteristics. By comparing the similarity with the central feature hash of the cluster, and then specifically comparing it with the code within the cluster, this comparison approach greatly improves query speed. Combining Hamming distance to generate malicious code clusters transforms the n-to-n complexity of cross-island queries into 1-to-n queries, significantly reducing the computational overhead of queries, further optimizing the communication process, and thus accelerating the generation of datasets.
[0041] In some embodiments, Hamming distance is used to determine the similarity of each malware hash value. The Hamming distance formula is as follows:
[0042] d(x,y)=∑x[i]⊕y[i)
[0043] Where x[i] and y[i] represent the hash values of two distinctive malicious code segments.
[0044] In some embodiments, cosine similarity is used to determine the similarity of each malware hash value. The cosine similarity formula is as follows:
[0045]
[0046] Where A and B represent the malicious code hash vectors, respectively.
[0047] Each user sends a query request, sending the hash value of their malicious code to other users;
[0048] Other users compare the hash sent to them with their own generated clusters to obtain the specific type of malicious code they are looking for, and then provide feedback to the user who made the query.
[0049] Based on the returned query results, users can optimize the classification of the current code, expand their dataset, and finally begin training and testing the model according to the existing malware model design. In some embodiments, a GAN (Generative Adversarial Network) is also used to generate the dataset. The model is a deep learning convolutional neural network.
[0050] Deep learning convolutional neural networks include, but are not limited to, MLP (Multilayer Perceptron), LSTM (Long Short-Term Memory Neural Network), BERT (Autoencoder Model), Transformer, GPT-3 (Autoregressive Model), and Attention (Self-Attention Mechanism).
[0051] The beneficial effects of this invention are as follows:
[0052] Using this invention, the queryer can obtain the correlation between a certain keyword and a certain text, which fully protects the data privacy of both parties, and at the same time provides the necessary training data for ranking learning.
[0053] This invention is the first to apply content-based chunking technology to the field of malware detection. Chunking technology generates a unique hash value from attack fragments within malware, which is then used as the basis for judging code similarity. Leveraging the irreversible nature of hashing, it ensures the data privacy and security of all data owners. This is the first time a content-based chunking hash algorithm has been used to process malware. Compared to current deep learning methods for generating grayscale images of malware, hashing algorithms significantly accelerate data preprocessing while also ensuring the quality of the dataset.
[0054] This invention is the first to propose using hashing to determine code similarity, thereby optimizing the training set. No such method has been used in existing malware model training. By employing a reliable hashing algorithm, the privacy of malware source file data is guaranteed, solving the critical data silo problem in federated learning and effectively improving the performance of all client-trained models.
[0055] Since it is necessary to compare differences between various files in both datasets, this invention combines hash results with Hamming distance to transform a large number of malicious code libraries into several clusters with distinct characteristics. By comparing the similarity with the central feature hash of the clusters, and then specifically comparing it with the code within the cluster, this comparison approach greatly improves query speed. Combining Hamming distance to generate malicious code clusters reduces the n-to-n complexity of cross-island queries to a 1-to-n query, significantly reducing the computational overhead of queries, further optimizing the communication process, and thus accelerating the generation of datasets.
[0056] As used herein, the term "preferred" is meant as an example, illustration, or illustration. Any aspect or design described herein as "preferred" need not be construed as being more advantageous than other aspects or designs. Rather, the use of the term "preferred" is intended to present the concept in a specific manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusionary "or." That is, unless otherwise specified or clear from the context, "X uses A or B" naturally includes either of the permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.
[0057] Furthermore, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the aforementioned components (e.g., elements, etc.), the terminology used to describe such components is intended to correspond to any component (unless otherwise indicated) that performs the specified function of said component (e.g., is functionally equivalent to it), even if structurally not equivalent to the disclosed structure performing the functions in the exemplary implementations of this disclosure shown herein. Moreover, although specific features of this disclosure have been disclosed with respect to only one of several implementations, such features may be combined with one or more features of other implementations that may be desirable and advantageous for a given or particular application. Furthermore, with regard to the use of the terms “comprising,” “having,” “containing,” or variations thereof in the Detailed Description or claims, such terms are intended to be included in a manner similar to the term “including.”
[0058] The functional units in this invention embodiment can be integrated into a processing module, or each unit can exist physically separately, or multiple units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. The aforementioned devices or systems can execute the storage methods in the corresponding method embodiments.
[0059] In summary, the above embodiments are one implementation of the present invention, but the implementation of the present invention is not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made that deviate from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.
Claims
1. A method for detecting malicious code in industrial control systems based on content segmentation, characterized in that, Includes the following steps: S1: Each user handles their own malicious code sample; S2: Read the file by file data stream, using two parallel sliding data windows, one to read odd-numbered bytes and the other to read even-numbered bytes; S3: For each bit read, calculate the hash within the window. If it equals the special hash value used for boundary determination, then end the current slide. S4: After the sliding ends, sum the hash values of every 16 slidings and use the result as the total hash. S5: Repeat steps S2-S4 until the entire file stream has been read, completing the current file's block division process; S6: Extract the hash value of the distinctive malicious code segment of each malicious code file, use Hamming distance to determine the similarity of each malicious code hash value, and generate several clusters; S7: Each user sends a query request, sending the hash value of their malicious code to other users; S8: Other users compare the hash sent with their own generated clusters to obtain the specific type of malicious code being queried, and then provide feedback to the querying user. S9: Based on the returned query results, the user optimizes the classification of the current code, expands their dataset, and finally begins training and testing the model using a deep learning convolutional neural network.
2. The method for detecting malicious code in industrial control systems based on content segmentation according to claim 1, characterized in that, According to the formula hash[i]=hash[i-2]<<1+Gear[byte[i]], calculate the hash within the window once, where i represents the current position of the sliding window, hash represents the hash value at the current position, Gear represents a hash table containing 256 special hash values, and byte represents the decimal number of the byte at the current position.
3. The method for detecting malicious code in industrial control systems based on content segmentation according to claim 1, characterized in that, The Hamming distance formula is as follows: d(x,y)=∑x[i]⊕y[i) Where x[i] and y[i] represent the hash values of two distinctive malicious code segments.
4. The industrial control system malicious code detection method based on content segmentation according to claim 1, characterized in that, In step S6, cosine similarity is used to measure similarity.
5. The method for detecting malicious code in industrial control systems based on content segmentation according to claim 1, characterized in that, In step S9, a GAN (Generative Adversarial Network) is used to generate a dataset.
6. The method for detecting malicious code in industrial control systems based on content segmentation according to claim 1, characterized in that, The deep learning convolutional neural network includes at least one of MLP, LSTM, BERT, Transformer, GPT-3, and Attention.