Cloud scenario malicious traffic analysis detection method and system among microservices and storage medium

Abstract

The application provides a cloud scene malicious traffic analysis and detection method and system among microservices, a storage medium, microservice information is collected, a baseline arranged in time sequence is established according to a unique identifier; one-dimensional time sequence data is acquired, and malicious request traffic of a normal sample is constructed; and a network model is trained. The application integrates discrete data by extracting one-dimensional time sequence data, so that the result is more accurate, and false negatives and false positives are reduced. The application extracts protocol features, action features, request features, response features, malicious traffic feature characteristics and time characteristics, uses the characteristic of data in a specific scene to train the network model, the network model of the application detects one frame of data in all traffic instead of classifying traffic data; the application uses a corrector in combination with a service process environment to accurately identify normal traffic, and can find malicious behavior disguised as normal traffic by using the principle of negative feedback, so the application has good practicability.

Description

Technical Field

[0001] The present invention belongs to the field of network security technology, and specifically relates to a method, system and storage medium for analyzing and detecting malicious traffic between microservices in cloud scenarios. Background Art

[0002] Microservices is a cloud-native architectural approach that encompasses numerous small, loosely coupled, independently deployable components or services within a single application. These services typically have their own technology stack, including a database and data management model; communicate with each other through a combination of REST APIs, event streams, and message brokers; and are organized by business capabilities, with service demarcations often referred to as bounded contexts. Microservices are characterized by easier code updates; new features or functionality can be added without having to update the entire application, and teams can use different technology stacks and different programming languages ​​for different components. Components can scale independently of each other, reducing the waste and cost associated with having to scale the entire application (because a single function might face excessive load).

[0003] With the rapid development of cloud technology, cloud security is gaining increasing attention from users. As a result of this advancement, microservices, whose security cannot be ignored, are crucial. Protecting east-west traffic within microservices is crucial because east-west traffic refers to traffic entering the internal network from the external network. In a microservices architecture, communication between services typically occurs via east-west traffic. Therefore, protecting east-west traffic within microservices effectively prevents external attackers from exploiting this traffic to enter the internal network and attack microservices.

[0004] Therefore, the present invention provides an analysis and detection solution for malicious traffic between microservices in a cloud scenario, which uses an efficient analysis and detection method to block and release traffic in a specific scenario, wherein the specific scenario is the microservice in a cloud environment. For example, in K8S (Kubernetes), microservices have their own IDs, names, and IPs, which we collectively refer to as unique identifiers. Microservices can use a variety of communication protocols such as GRPC, HTTP, etc. to interact, and their protocol formats are fixed. Therefore, the traffic between microservices is single and fixed relative to the network traffic outside the cluster. The present invention uses the unique identifier of the microservice as the baseline primary key to establish a learning baseline, and the baseline content includes but is not limited to protocol, action, sending time, request characteristics, and response characteristics. The present invention uses a deep learning algorithm to model the established baseline to form a baseline model. After the learning is completed, the baseline model is used to perform malicious analysis and detection on the traffic. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, system and storage medium for analyzing and detecting malicious traffic between microservices in cloud scenarios, aiming to solve the above-mentioned problems.

[0006] The present invention is mainly achieved through the following technical solutions:

[0007] The malicious traffic analysis and detection method between microservices in cloud scenarios includes the following steps:

[0008] Step S1: Collect microservice information in the cluster and establish a time-series baseline based on the unique identifier;

[0009] Step S2: Analyze the traffic and obtain one-dimensional time series data, and construct malicious request traffic of normal samples;

[0010] Step S3: Using the time series data in step S2 to train the network model;

[0011] Step S31: importing the one-dimensional time series data into the sampling device for preprocessing;

[0012] Step S32: Input the sampling characteristics into the corrector, use the corrector to identify the traffic in combination with the environmental parameters of the service process, and intercept the traffic with specific characteristics;

[0013] Step S33: Input the two-dimensional array into the feature model. The feature model checks the traffic between microservices and calculates the 0 / 1 output of whether the traffic is malicious traffic based on the output result in step S32.

[0014] Step S34: The one-dimensional time series data features output from step S31 and the output result from step S33 are input into the time convolution algorithm training model for processing to obtain the final flow detection result; at the same time, the flow time series and frequency are recorded and fed back to the corrector;

[0015] Step S4: Input the monitored data into the trained network model to obtain malicious traffic analysis results.

[0016] In order to better implement the present invention, further, in step S2, the one-dimensional time series data includes protocol, action, occurrence time, request characteristics, response characteristics and malicious traffic characteristics; the protocol includes any one or more of GRPC and HTTP, and the occurrence time is the time when the interactive traffic is generated; the malicious traffic characteristics are determined by the sample traffic itself, normal traffic takes 0, and malicious traffic takes 1.

[0017] In order to better implement the present invention, further, the request characteristics and response characteristics respectively include overall data characteristics, local data characteristics and protocol characteristics, and the overall data characteristics include any one of JSON, XML, and TEXT formats; the local data characteristics include type characteristics of tags or data fields in a fixed format, and in the local data characteristics of the response characteristics, the extracted response key fields include response codes and error codes; the type characteristics include any one or more of integers, strings, MD5, and HASH; the protocol characteristics include fixed fields of the protocol header.

[0018] In order to better implement the present invention, further, when extracting request features and response features, the fuzzy HASH algorithm is used to match the similarity of each feature. If the similarity is within the threshold range, the sample is valid; for traffic with fixed features, the threshold is 80%. If the conditions are met, the matching threshold is set to 1, otherwise it is set to 0.

[0019] In order to better implement the present invention, further, the HTTP request method is extracted in restful, and the actions include POST as creation and PUT as modification; in other modes, if the features in the URL are extracted, the actions include create, add as creation, modify, change, update as modification, and delete, remove as deletion.

[0020] In order to better implement the present invention, further, in step S32, the environmental parameters include the CPU, disk, memory usage and timing characteristics, and request frequency of the service process.

[0021] The present invention is mainly achieved through the following technical solutions:

[0022] A cloud scenario inter-microservice malicious traffic analysis and detection system, which uses the above method, includes a data acquisition module, a model training module, and a detection module. The data acquisition module is used to establish a time-series baseline based on a unique identifier and obtain one-dimensional time series data; the model training module is used to train a network model using the one-dimensional time series data; and the detection module is used to input real-time detection data into the trained network model and output detection results.

[0023] The network model includes a sampling sampler, a corrector, a feature model and a time convolution module arranged in sequence from front to back. The sampling sampler is connected to the corrector and the feature model respectively. The output ends of the corrector and the feature model are connected to the calculation module. The calculation module is used to calculate whether the traffic is malicious traffic and output 0 / 1. The calculation module and the sampling sampler are respectively connected to the time convolution module through a combination module. The time convolution module is connected to the corrector to form negative feedback.

[0024] In order to better implement the present invention, further, the feature module is constructed based on a multi-causal convolution algorithm.

[0025] A computer-readable storage medium stores a computer program, which implements the above method when executed by a processor.

[0026] The beneficial effects of the present invention are as follows:

[0027] The present invention integrates discrete data by extracting one-dimensional time series data, making the results more accurate and reducing missed reports and false positives. Targeting microservices in cloud scenarios, the present invention extracts protocol features, action features, request features, response features, malicious traffic characteristics, and time features, and utilizes the characteristic properties of data in specific scenarios to train network models. The network model of the present invention detects one frame of data from all traffic rather than classifying traffic data. The present invention utilizes a corrector combined with the service process environment to accurately identify normal traffic, and utilizes the principle of negative feedback to find malicious behavior disguised as normal traffic, thus having good practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the data structure of baseline A;

[0029] Figure 2 This is a schematic diagram of the overall model training of the malicious traffic analysis and detection method between microservices in cloud scenarios of the present invention;

[0030] Figure 3 Schematic diagram of output data for the rectifier, feature model, and final network model;

[0031] Figure 4 The data processing principle diagram of the corrector and feature model;

[0032] Figure 5 It is a schematic diagram of the overall structure of the network model of the present invention. DETAILED DESCRIPTION

[0033] Example 1:

[0034] A method for analyzing and detecting malicious traffic between microservices in cloud scenarios, such as Figure 1-Figure 5 As shown, the following steps are included:

[0035] Step S0: Collect microservice information in the cluster and establish a baseline A based on a unique identifier (such as the name of the microservice).

[0036] Step S1: Analyze traffic to obtain baseline model input values, including but not limited to:

[0037] Protocol: such as GRPC, HTTP, etc.

[0038] Action: In RESTful mode, HTTP request methods are used in advance, with POST used for creation and PUT used for modification. In other modes, such as extracting features from the URL, actions include but are not limited to create, add, etc. for creation, modify, change, update, etc. for modification, and delete, remove, etc. for deletion.

[0039] Occurrence time: the time when the interactive traffic is generated

[0040] Request feature extraction: including but not limited to the following features:

[0041] 1. Overall data characteristics, such as JSON, XML, TEXT, etc.;

[0042] 2. Local data features: extracting type features of tags / data fields in fixed formats, such as integers, strings, MD5, HASH, and other specific attributes;

[0043] 3. Protocol features such as fixed fields in the request header

[0044] In summary, each feature is matched using a fuzzy HASH algorithm for similarity. If the similarity is within the threshold range, the sample is considered valid. The threshold range can also be selected through analysis of other learning algorithms. For traffic with fixed features, the empirical value is generally set to 80% or above. If the condition is met, the matching threshold is set to 1, and if not, it is set to 0. The final request feature is (feature HASH, matching threshold)

[0045] Response feature extraction: including but not limited to the following features:

[0046] 1. Overall data characteristics, such as JSON, XML, TEXT, etc.

[0047] 2. Local data features: extract the types of labels / data fields in fixed formats, such as integer, string, MD5, UUID, and HASH; and extract key response fields, such as response codes and error codes.

[0048] 3. Protocol features such as fixed fields in the response header

[0049] In summary, each feature is matched using a fuzzy HASH algorithm for similarity. If the similarity is within the threshold range, the sample is considered valid. The threshold range can also be selected through analysis of other learning algorithms. For traffic with fixed features, the empirical value is generally set to 80% or above. If the condition is met, the matching threshold is set to 1, and if not, it is set to 0. The final request feature is (feature HASH, matching threshold)

[0050] Malicious traffic characteristics: This characteristic is determined by the sample traffic itself. Normal traffic is 0, and malicious traffic is 1.

[0051] Step S2. Combine the features obtained in s1 into a two-dimensional array (protocol features, action features, request features, response features, feature dimensions of malicious traffic characteristics, and time dimensions) and use them as inputs to the deep learning algorithm (here using the multi-causal convolution algorithm) in the order of traffic reception, ultimately obtaining a time-series-based feature model.

[0052] Step S3. Training: construct malicious request traffic of normal samples, such as a large number of requests with the same characteristics within a certain time range, to correct the output results in the model, such as Figure 2 As shown, we collectively refer to the sampling-rectification-feature model as a "computational unit".

[0053] Corrector: The purpose of the corrector is to detect that the traffic itself is normal traffic for the feature model. However, since the attack simulates a large amount of normal traffic to form a DoS-like attack, which causes excessive pressure on the server, the corrector detects environmental parameters including the CPU, disk, memory usage, timing characteristics, and request frequency of the service process to obtain results and intercept traffic with specific characteristics.

[0054] The simulation scenario is as follows: a large amount of traffic data A enters the service and over time causes the service process CPU, disk, memory and other resource consumption to increase, and the request frequency is higher than the learning model. As a result, it is necessary to classify traffic data A under this feature as malicious traffic.

[0055] The feature model in step S2 is used to check the traffic between microservices, and the feature input extracted in step S1 is used to calculate the 0 / 1 output of whether the traffic is malicious traffic through the model.

[0056] Step S4. Negative feedback model training: integrate the discrete baseline sequence, use the data features and output results as model input, and use the time convolution algorithm to train the model to generate a flow detection model. At the same time, the model records the flow timing and frequency and feeds the results back to the corrector to form a negative feedback model.

[0057] This invention targets microservices in cloud scenarios, extracting features from samples and leveraging the specific characteristics of data in specific scenarios. It integrates discrete data through time series, resulting in more accurate results and reducing missed and false positives. The network model detects a single frame of data from all traffic, rather than categorizing traffic data. It utilizes a corrector combined with the service process environment to accurately identify normal traffic and employs the principle of negative feedback to identify malicious behavior disguised as normal traffic, demonstrating its practicality.

[0058] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A method for analyzing and detecting malicious traffic between microservices in cloud scenarios, characterized by: The following steps are involved: Step S1: Collect microservice information in the cluster and establish a time-series baseline based on the unique identifier of the microservice; Step S2: Analyze traffic and obtain one-dimensional time series data, and construct malicious request traffic of normal samples; the one-dimensional time series data includes protocol, action, occurrence time, request characteristics, response characteristics and malicious traffic characteristics; Step S3: Using the one-dimensional time series data in step S2 to train the network model; specifically including: Step S31: importing the one-dimensional time series data into the sampling device for preprocessing; Step S32: input the sampling characteristics into the corrector, and use the corrector to identify and intercept the traffic in combination with the environmental parameters of the service process; Step S33: Combine the feature dimensions of the protocol features, action features, occurrence time features, request features, response features, and malicious traffic characteristics with the time dimension into a two-dimensional array. Input the two-dimensional array into the feature model. The feature model checks the traffic between microservices and calculates a 0 / 1 output indicating whether the traffic is malicious traffic based on the output result in step S32. Step S34: The one-dimensional time series data features output from step S31 and the output result from step S33 are input into the time convolution module for processing to obtain the final flow detection model; at the same time, the flow time series and frequency are recorded and fed back to the corrector to form negative feedback; Step S4: Input the monitored data into the trained network model to obtain malicious traffic analysis results.

2. The method for analyzing and detecting malicious traffic between cloud-based microservices according to claim 1 is characterized in that: In step S2, the protocol includes any one or more of GRPC and HTTP, and the occurrence time is the time when the interactive traffic is generated; the malicious traffic characteristics are determined by the sample traffic itself, normal traffic is 0, and malicious traffic is 1.

3. The method for analyzing and detecting malicious traffic between cloud scenario microservices according to claim 2 is characterized in that: The request features and response features include overall data features, local data features and protocol features respectively; the overall data features include any one of JSON, XML, and TEXT formats; the local data features include type features of tags or data fields in a fixed format, and in the local data features of the response features, the extracted response key fields include response codes and error codes; the type features include any one or more of integer, string, MD5, and HASH; the protocol features include fixed fields of the protocol header.

4. The method for analyzing and detecting malicious traffic between cloud-based microservices according to claim 3 is characterized in that: When extracting request features and response features, the fuzzy HASH algorithm is used to match the similarity of each feature. If the similarity is within the threshold range, the sample is valid. For traffic with fixed features, the threshold is 80%. If the conditions are met, the matching threshold is set to 1, otherwise it is set to 0.

5. The method for analyzing and detecting malicious traffic between cloud scenario microservices according to claim 2 is characterized in that: In restful mode, the HTTP request method is extracted, and the actions include POST for creation and PUT for modification. In other modes, if the features in the URL are extracted, the actions include create and add for creation, modify, change and update for modification, and delete and remove for deletion.

6. The method for analyzing and detecting malicious traffic between cloud-based microservices according to claim 1, characterized in that: In step S32, the environmental parameters include the CPU, disk, memory usage and timing characteristics, and request frequency of the service process.

7. A cloud scenario inter-microservice malicious traffic analysis and detection system, implemented using the method according to any one of claims 1 to 6, characterized in that: It includes a data acquisition module, a model training module and a detection module. The data acquisition module is used to establish a baseline arranged in time sequence according to the unique identifier of the microservice and obtain one-dimensional time series data; the model training module is used to train the network model using the one-dimensional time series data, and the detection module is used to input the real-time detection data into the trained network model and output the detection results; The network model includes a sampling sampler, a corrector, a feature model and a time convolution module arranged in sequence from front to back. The sampling sampler is connected to the corrector and the feature model respectively. The output ends of the corrector and the feature model are connected to the calculation module. The calculation module is used to calculate whether the traffic is malicious traffic and output 0 / 1. The calculation module and the sampling sampler are respectively connected to the time convolution module through a combination module. The time convolution module is connected to the corrector to form negative feedback.

8. The cloud scenario inter-microservice malicious traffic analysis and detection system according to claim 7 is characterized in that: The feature model is constructed based on a multi-causal convolution algorithm.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images