A network flow partitioning method and system

Abstract

An embodiment of the present specification provides a network flow partitioning method, the method comprising: obtaining a network data packet; extracting a quintuple and a corresponding effective load length, a sending timestamp and a receiving timestamp from the network data packet, and dividing the quintuple into a forward quintuple and a backward quintuple, the quintuple comprising at least one of a source IP address, a source port, a destination IP address, a destination port and a transport layer protocol; based on a preset time interval, the forward quintuple and the backward quintuple are respectively blocked to obtain a plurality of forward quintuple blocks and a plurality of backward quintuple blocks; wherein there is a time overlap between two adjacent forward quintuple blocks, and there is a time overlap between two adjacent backward quintuple blocks; and, based on the sending timestamp and the receiving timestamp, determining the transmission delay.

Description

[0001] Description of the case

[0002] This application is a divisional application filed for a Chinese application with a filing date of February 21, 2023, application number 202310146496.2, and invention name “A method and system for event detection based on network flow”. Technical Field

[0003] The present invention relates to the field of communication technology, and in particular to a network flow division method and system. Background Art

[0004] With the continuous development of information technology, various network-based applications have gradually occupied the mainstream position of Internet applications. In such a situation, if you want to manage network security and ensure the efficiency of network operations, you need to accurately identify events in network traffic. For example, network administrators need to monitor and manage different applications based on network flows to ensure the security of network operations.

[0005] For identification based on network traffic, CN114401229A proposed an encrypted traffic identification method based on the Transformer deep learning model. The global and local features of each byte in the data packet and the global features of the bytes between data packets in the data stream and the global features of the statistical information were extracted successively through the improved intra-Transformer model. However, this method does not fully utilize the intrinsic relationship of the data, which may cause large errors in the final identification results.

[0006] Therefore, it is desirable to provide a network flow segmentation method and system so as to provide more accurate event detection results. Summary of the invention

[0007] One or more embodiments of the present specification provide a network flow partitioning method, characterized in that the method includes: obtaining a network data packet; extracting a quintuple and a corresponding effective load length, a sending timestamp, and a receiving timestamp from the network data packet, and dividing the quintuple into a forward quintuple and a backward quintuple, wherein the quintuple includes at least one of a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol; based on a preset time interval, the forward quintuple and the backward quintuple are respectively divided into blocks to obtain a plurality of forward quintuple blocks and a plurality of backward quintuple blocks; wherein there is a time overlap between two adjacent forward quintuple blocks, and there is a time overlap between two adjacent backward quintuple blocks; and, based on the sending timestamp and the receiving timestamp, determining a transmission delay.

[0008] One or more embodiments of the present specification provide a network flow partitioning system, including an acquisition module, an extraction module and a determination module; the acquisition module is used to acquire network data packets; the extraction module is used to extract a quintuple and a corresponding effective load length, a sending timestamp and a receiving timestamp from the network data packet, and divide the quintuple into a forward quintuple and a backward quintuple, the quintuple including at least one of a source IP address, a source port, a destination IP address, a destination port and a transport layer protocol, and based on a preset time interval, the forward quintuple and the backward quintuple are respectively blocked to obtain a plurality of forward quintuple blocks and a plurality of backward quintuple blocks; wherein, there is a time overlap between two adjacent forward quintuple blocks, and there is a time overlap between two adjacent backward quintuple blocks; the sending timestamp and the receiving timestamp are used to determine the transmission delay.

[0009] One or more embodiments of the present specification provide a network flow segmentation device, including a processor, wherein the processor is used to execute any of the network flow segmentation methods described above.

[0010] One or more embodiments of the present specification provide a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the network flow division method as described in any one of the above. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0012] Figure 1 It is a schematic diagram of an application scenario of an event detection system based on network flow according to some embodiments of this specification;

[0013] Figure 2 is an exemplary module diagram of a network flow-based event detection system 200 according to some embodiments of the present specification;

[0014] Figure 3 is an exemplary flow chart of a network flow-based event detection method according to some embodiments of this specification;

[0015] Figure 4 is a schematic diagram of a method for determining an application type according to some embodiments of this specification;

[0016] Figure 5 It is a flow chart of a method for determining user behavior according to some embodiments of this specification. DETAILED DESCRIPTION

[0017] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0018] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0019] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0020] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0021] Figure 1 It is a schematic diagram of an application scenario of a network flow-based event detection system according to some embodiments of this specification.

[0022] like Figure 1 As shown, the components in the application scenario 100 of the network flow-based event detection system may include a terminal 110, a server 120, a storage device 130, and a network 140. In some embodiments, the components in the application scenario 100 may be connected and / or communicated with each other via the network 140 (e.g., wireless connection, wired connection, or a combination thereof).

[0023] In some embodiments, the network flow-based event detection system can be used to detect the type of event that is currently occurring in the network. When a terminal user participates in a network, the network flow-based event detection system can be used to determine the type of event that the terminal user is participating in. For example, when a user participates in a network, the network flow-based event detection system can be used to determine whether the current user is performing network events such as sending emails, browsing web pages, and communicating using chat software.

[0024] Terminal 110 refers to one or more terminals or software with network connection function used by any user. In some embodiments, the user can access the network through terminal 110, conduct network activities, and generate corresponding network events. In some embodiments, terminal 110 can be a mobile device, a tablet computer, a laptop computer, or any combination of other devices with input and / or output functions.

[0025] The server 120 may process data and / or information related to the network flow-based event detection system. In some embodiments, the server 120 accesses information and / or data from the storage device 130 and / or the terminal 110. In some embodiments, the server may obtain network data packets from the storage device 130 and / or the terminal 110.

[0026] The server 120 may be used to manage resources and process data and / or information from at least one component of the system or an external data source (e.g., a cloud data center). In some embodiments, the server may extract a quintuple from the acquired network data packet and determine a time classification based on the quintuple.

[0027] In some embodiments, the server 120 may include a processing device 122. The processing device 122 may process data and / or information obtained from other devices or system components. The processing device 122 may execute program instructions based on the data, information and / or processing results to perform one or more functions described in the present application.

[0028] The storage device 130 can be used to store data and / or instructions. The storage device 130 is implemented in a single central server, multiple servers connected by a communication link, or multiple personal devices. In some embodiments, the storage device 130 can also be used to provide one or more sources of other information for the network flow-based event detection system. The storage device 130 can be one or more devices, one or more application programming interfaces, one or more protocol-based information acquisition interfaces, or other ways to obtain information. The information provided by the storage device 130 may already exist when the information is extracted, or may be temporarily generated when the information is extracted, or may be a combination of the above methods. In some embodiments, the storage device 130 can store network data packets, quintuples extracted from network data packets, etc.

[0029] The network 140 can connect the components of the system and / or connect the system with external resources. The network 140 enables communication between the components and other components outside the system to facilitate the exchange of data and / or information. In some embodiments, the network 140 may include a local area network and other networks within the venue. For example, the network 140 may include a local area network within the venue to which the terminal 110 requests access. For example, when the terminal 110 accesses the network 140, the user can perform network activities through the terminal 110, thereby generating a network event. In some embodiments, the network 140 may also include a cable network, an optical fiber network, a telecommunications network, the Internet, or any combination thereof. The network connection between the components may be in one of the above-mentioned ways or in multiple ways. In some embodiments, the network may be a point-to-point, shared, centralized, or other topological structure or a combination of multiple topological structures. In some embodiments, the network 140 may include one or more network access points. For example, the network 140 may include a wired or wireless network access point, such as a base station and / or a network switching point, through which one or more components of the network flow-based event detection system may be connected to the network 140 to exchange data and / or information.

[0030] It should be noted that the application scenario 100 of the event detection system based on network flow is provided only for the purpose of illustration and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or changes can be made according to the description of this specification. For example, the application scenario 100 of the event detection system based on network flow can be implemented on other devices to achieve similar or different functions. However, changes and modifications will not deviate from the scope of this specification.

[0031] Figure 2 2 is an exemplary module diagram of a network flow-based event detection system 200 according to some embodiments of the present specification. Figure 2 As shown, the network flow-based event detection system 200 may include an acquisition module 210 , an extraction module 220 and a determination module 230 .

[0032] The acquisition module 210 can be used to acquire network data packets. For details about the network data packets, please refer to the description of other contents of this specification, for example, Figure 3 .

[0033] The extraction module 220 can be used to extract a five-tuple from the network data packet, where the five-tuple includes at least one of a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol. For details of the five-tuple, please refer to the description of other contents of this specification, for example, Figure 3 .

[0034] In some embodiments, the extraction module 220 can be used to: divide the quintuple into a forward quintuple and a backward quintuple; based on a preset time interval, the forward quintuple and the backward quintuple are respectively blocked to obtain multiple forward quintuple blocks and multiple backward quintuple blocks; wherein, there is a time overlap between two adjacent forward quintuple blocks, and there is a time overlap between two adjacent backward quintuple blocks. For details about the forward quintuple ring and the backward quintuple ring, please refer to the description of other contents of this specification, for example, Figure 3 .

[0035] The determination module 230 may be used to determine the event classification based on the five-tuple. For details about the event classification, please refer to the description of other contents of this specification, for example, Figure 3 , Figure 4 .

[0036] In some embodiments, the event classification includes an application type, and the determination module 230 can be used to: determine the event classification based on the quintuple includes: based on the forward quintuple block and the backward quintuple block, determine the application type through a classification model, the classification model is a machine learning model; the input data of the classification model is related to the effective load length of the forward quintuple block and the effective load length of the backward quintuple block. For details about the classification model, please refer to the description of other contents of this specification, for example, Figure 4 .

[0037] In some embodiments, event classification also includes user behavior, and the determination module 230 can be used to: determine at least one candidate user behavior based on the application type; calculate the similarity between the quintuple and the candidate quintuple through a similarity model based on the quintuple and the candidate quintuple; determine the probability of occurrence of each candidate user behavior in the at least one candidate user behavior based on the similarity; when the probability difference between at least one candidate user behavior is greater than a threshold, use the candidate user behavior with the highest probability as the target user behavior (i.e., user behavior). For details about the candidate quintuple, similarity model, candidate user behavior, and target user behavior, please refer to the description of other contents of this specification, for example, Figure 4 .

[0038] Figure 3 300 is an exemplary flow chart of an event detection method based on network flow according to some embodiments of this specification. In some embodiments, the process 300 of the event detection method based on network flow can be executed by a processor. Figure 3 As shown, process 300 may include the following steps:

[0039] Step 310, obtaining a network data packet.

[0040] A network data packet may refer to a basic transmission unit related to a network activity performed by a user. A data packet may be a data unit related to a network activity performed by a user in a communication transmission. For example, when a user transmits a video, a network data packet may be a collection of one or more frames of the video. In some embodiments, each network data packet may include its effective load level, a sending timestamp and / or a receiving timestamp, etc. For details about the effective load level, the sending timestamp and / or the receiving timestamp, please refer to the description of the following content of this specification.

[0041] In some embodiments, the processor may obtain network data packets from a terminal or a storage device.

[0042] Step 320, extracting a quintuple from the network data packet.

[0043] A quintuple may refer to a set of fields used to reflect the information characteristics of a network data packet. In some embodiments, a quintuple may include at least one of a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol. The source IP address may be a network address representing the source of a network data packet. The source port may refer to a port number for sending a network data packet. The port number may be used to represent the number of an application for communication on a terminal. Multiple applications may be run simultaneously on the same terminal, for example, the port number may be 21, 22, 80, etc. The destination IP address may refer to a network address for receiving a network data packet. The destination port may refer to a port number for receiving a network data packet. The transport layer protocol may refer to a data format for communication that can be encapsulated and parsed by an application. For example, the transport layer protocol may be a TCP protocol, a UDP protocol, etc. For example, the relevant content of the five-tuple in the data packet may be 192.168.1.1 10000 TCP 121.14.88.76 80, which may indicate that a terminal with a source IP address of 192.168.1.1 is connected to a terminal with a destination IP address of 121.14.88.76 and a destination port of 80 through source port 10000 using the TCP protocol.

[0044] In some embodiments, each network data packet may correspond to a five-tuple.

[0045] In some embodiments, the processor can obtain the parsing result corresponding to the text paragraph classification result through a field extraction method based on a processing algorithm. The processing algorithm may refer to a machine learning model, a machine reading comprehension model, etc. The processor can extract the corresponding parsing result from each network data packet through a machine learning model, a machine reading comprehension model, etc., and determine the corresponding five-tuple related content. Further, the source IP address, source port, destination IP address, destination port and transport layer protocol in the five-tuple all correspond to specific positions and lengths, and the corresponding content can be extracted from the five-tuple related content in sequence. For example, the five-tuple related content is 192.168.1.1 10000 TCP 121.14.88.76 80, and the corresponding source IP address, source port, transport layer protocol, destination IP address, and destination port can be extracted in sequence.

[0046] In some embodiments, extracting a quintuple from a network data packet includes: dividing the quintuple into a forward quintuple and a backward quintuple; based on a preset time interval, respectively, blocking the forward quintuple and the backward quintuple to obtain multiple forward quintuple blocks and multiple backward quintuple blocks; wherein, there is a time overlap between two adjacent forward quintuple blocks, and there is a time overlap between two adjacent backward quintuple blocks.

[0047] The forward quintuple may refer to a quintuple corresponding to a network data packet sent from a source IP address to a destination IP address. The backward quintuple may refer to a quintuple corresponding to a network data packet sent from a destination IP address to a source IP address.

[0048] In some embodiments, the processor can obtain a network data packet sent from the source IP address to the destination IP address, and extract the forward five-tuple therefrom. In some embodiments, the processor can obtain a network data packet sent from the destination IP address to the source IP address, and extract the backward five-tuple therefrom.

[0049] The preset time interval may refer to the length of time occupied by the selected network data packet. The preset time interval may be manually preset. In some embodiments, the preset time intervals may be the same. In some embodiments, the processor may obtain a plurality of forward five-tuple blocks and a plurality of backward five-tuple blocks based on the preset time interval. For example, the preset time interval may be 0.1s, and the forward five-tuple may be 0s-0.10s, 0.05s-0.15s, 0.10s-0.20s, etc.

[0050] In some embodiments, extracting the quintuple from the network data packet further includes: extracting the effective load length corresponding to each quintuple from the network data packet.

[0051] The effective load length can refer to the original data (e.g., network stream) that is often divided into network packets for transmission in order to make the transmission of the original data more reliable, and certain auxiliary identification information (e.g., check bits, etc.) is added to the header and tail of each network packet to prevent the original data from being lost. The original data part in each packet is the effective load in the packet. The effective load length can refer to the bytes occupied by the effective load.

[0052] In some embodiments, the processor may remove other auxiliary identification information in the network data packet, obtain the effective load, and calculate its bytes to determine the effective load length.

[0053] Through some embodiments of the present specification, the effective payload length can be determined, and based on the data volume length of the network data packet, the effective length can be determined to reflect the data volume of the network data packet.

[0054] In some embodiments, extracting the quintuple from the network data packet further includes: extracting a sending timestamp and a receiving timestamp corresponding to each quintuple from the network data packet.

[0055] The sending timestamp may refer to the sending time of the network data packet. The receiving timestamp may refer to the receiving time of the network data packet. The sending timestamp and receiving timestamp corresponding to each quintuple may be extracted from the network data packet. The extraction method may refer to the extraction method of the quintuple.

[0056] Through some embodiments of the present specification, the sending timestamp and receiving timestamp of the quintuple can be determined, and the sending timestamp and receiving timestamp can reflect the time period characteristics of the network data packet.

[0057] In some embodiments, the processor may determine the transmission delay based on the transmit timestamp and the receive timestamp.

[0058] In some embodiments, in some embodiments, the processor may select a corresponding set of forward quintuples and backward quintuples based on the current time and the same preset time interval, and the transmission delay may be determined by formula (1).

[0059] α=(t4-t1)-(t2-t3)…(1)

[0060] Among them, α is the transmission delay, t1 and t2 are the sending timestamp and receiving timestamp of the forward quintuple respectively, and t3 and t4 are the sending timestamp and receiving timestamp of the backward quintuple respectively.

[0061] According to some embodiments of the present specification, the network delay situation can be further determined by sending timestamps and receiving timestamps.

[0062] Step 330, determine the event classification based on the quintuple.

[0063] Event classification may refer to the type of event that the user is currently engaged in. In some embodiments, event classification may include process type. In some embodiments, event classification may include at least one of application type and user behavior, wherein application type may include but is not limited to email, chat software, video software, etc., and user behavior may include but is not limited to email communication, text communication, not participating in a meeting, participating in a voice conference, participating in a video conference, etc.

[0064] In some embodiments, event classification includes application type, and determining event classification based on quintuple includes: determining application type through a classification model based on forward quintuple blocks and backward quintuple blocks, the classification model being a machine learning model; input data of the classification model is related to the effective load length of the forward quintuple blocks and the effective load length of the backward quintuple blocks. For details on determining event classification through the classification model, please refer to the description of other contents of this specification, for example, Figure 4 .

[0065] In some embodiments, event classification also includes user behavior, and determining event classification based on the quintuple includes: determining at least one candidate user behavior based on the application type; calculating the similarity between the quintuple and the candidate quintuple through a similarity model based on the quintuple and the candidate quintuple; determining the probability of occurrence of each candidate user behavior in at least one candidate user behavior based on the similarity; when the probability difference between at least one candidate user behavior is greater than a threshold, taking the candidate user behavior with the highest probability as the target user behavior (i.e., user behavior). For details on determining event classification based on candidate quintuples, please refer to the description of other contents of this specification, for example, Figure 5 .

[0066] According to some embodiments of the present specification, the event classification corresponding to the network data packet is determined by using a quintuple, so that the efficiency and accuracy of determining the event classification can be improved.

[0067] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

[0068] Figure 4 Schematic diagram of a method for determining an application type according to some embodiments of this specification. Figure 4 As shown, the method 400 for determining the application type may include the following contents.

[0069] In some embodiments, the processor 122 may determine the application type through a classification model based on the forward quintuple block and the backward quintuple block, wherein the application type may include but is not limited to email, chat software, video software, etc.

[0070] In some embodiments, the classification model may be a machine learning model. The input data of the classification model is related to the effective sum length of the forward 5-tuple block and the effective load length of the backward 5-tuple block.

[0071] In some embodiments, the input data of the classification model may include a forward five-tuple block, a payload length and a number of packets of the forward five-tuple block, a backward five-tuple block, a payload length and a number of packets of the backward five-tuple block, wherein the payload length refers to the length of the original data portion of the packet corresponding to the five-tuple block, and the number of packets refers to the number of packets corresponding to the five-tuple block.

[0072] In some embodiments, the output of the classification model may include application types, such as email, chat software, video software, etc.

[0073] like Figure 4 As shown, in some embodiments, the input data of the classification model may include a histogram of payload length and a histogram of the number of packets of the forward five-tuple block, a histogram of payload length and a histogram of the number of packets of the backward five-tuple block, and the output may include an application type.

[0074] In some embodiments, the classification model may include a transformer layer 420 and a fully connected layer 440 .

[0075] The transformer layer 420 may also be referred to as a self-attention mechanism layer, and may be used to extract network flow features. The input of this layer may be the input of the classification model, i.e., the effective load length histogram and the number of packets histogram 410-1 of the forward five-tuple block, and the effective load length histogram and the number of packets histogram 410-2 of the backward five-tuple block, and the output of this layer may include network flow features 430. The output of the transformer layer may be used as the input of the fully connected layer.

[0076] The fully connected layer 440 may be used to determine the application type based on the network flow features, and the input of the layer may include the network flow features 430 output by the transformer layer, and the output may include the event classification 450. In some embodiments, the fully connected layer may be a spiking neural network (SNN).

[0077] In some embodiments, the output of the transformer layer can be used as the input of the fully connected layer, and the transformer layer and the fully connected layer can be trained jointly.

[0078] In some embodiments, the first sample data of the joint training may include a histogram of effective load length and a histogram of the number of packets of the sample forward five-tuple block, a histogram of effective load length and a histogram of the number of packets of the sample backward five-tuple block, and the label is a sample event classification. The first sample data is input into the initial transformer layer to obtain the network flow features output by the initial transformer layer, and the network flow features output by the initial transformer layer are input into the initial fully connected layer to obtain the event classification output by the initial fully connected layer. A loss function is constructed based on the sample event classification and the event classification output by the initial fully connected layer, and the parameters of the initial transformer layer and the initial fully connected layer are updated synchronously until the training is completed when the preset conditions are met. Among them, the preset conditions may include that the loss function is less than a threshold, the loss function converges, or the training cycle reaches a threshold.

[0079] In some embodiments, a convolutional layer and a pooling layer (not shown in the figure) may be further included between the transformer layer and the fully connected layer of the classification model. The convolutional layer and the pooling layer are connected in sequence, the output of the transformer layer can be used as the input of the convolutional layer, and the output of the pooling layer can be used as the input of the fully connected layer.

[0080] In some embodiments, the classification model may further include a timestamp embedding layer 460. The input of the timestamp embedding layer may include the forward five-tuple block and all timestamps in the block 470-1, and the backward five-tuple block and all timestamps in the block 470-2. Among them, all timestamps may include a receiving timestamp and a block timestamp, the receiving timestamp is the time point of receiving data, and the block timestamp is the time point for performing block segmentation. The output of this layer may include a time period feature 480.

[0081] In some embodiments, the input of the fully connected layer may also include a time period feature 480 and a transmission delay 490. The time period feature may be obtained by the aforementioned timestamp embedding layer. The transmission delay may be obtained based on the timestamp. For more details on obtaining the transmission delay, see Figure 3 Related description in .

[0082] In some embodiments, the timestamp embedding layer can be trained based on the second training sample. For example, multiple second training samples with labels can be input into the initial timestamp embedding layer, and a loss function can be constructed by using the labels and the result of the initial timestamp embedding layer. The parameters of the initial timestamp embedding layer can be iteratively updated based on the loss function by gradient descent or other methods. When the preset conditions are met, the model training is completed and a trained timestamp embedding layer is obtained. The preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0083] In some embodiments, the second training sample may include the sample forward five-tuple block and all timestamps in the block, the sample backward five-tuple block and all timestamps in the block, and the training label represents the sample period characteristics. The training label can be determined by historical data or obtained through manual annotation.

[0084] In some embodiments, by determining the time period features through a timestamp embedding layer and using the time period features as the input of a fully connected layer, the classification model can learn the sending and receiving time of the network flow corresponding to the forward five-tuple block and / or the backward five-tuple block, thereby better judging the corresponding application type in combination with the time features.

[0085] In some embodiments of the present specification, determining the application type through a classification model based on the effective load length of the forward quintuple block and the effective load length of the backward quintuple can achieve the purpose of determining event classification more intelligently and accurately.

[0086] Figure 5 is a flow chart of a method for determining user behavior according to some embodiments of this specification. Figure 5 As shown, process 500 may include the following contents.

[0087] Step 510: Determine at least one candidate user behavior based on the application type.

[0088] In some embodiments, the application type may refer to application software or application programs that can generate network information flows, such as email, chat software, video software, etc.

[0089] In some embodiments, the application type can be determined by the processor through a classification model. For details about the classification model, please refer to this specification. Figure 3 and its related description.

[0090] In some embodiments, the candidate user behavior may refer to the user's current possible behavior state, for example, email communication, text communication, not participating in a meeting, participating in a voice conference, participating in a video conference, etc.

[0091] In some embodiments, the candidate user behavior can be determined based on the application type. For example, when the application type includes email, the candidate user behavior can include email communication; when the application type includes chat software, the candidate user behavior can include text communication, participating in voice conferences, participating in video conferences, not participating in conferences, etc.; when the application type includes video software, the candidate user behavior can include not participating in conferences, participating in voice conferences, participating in video conferences, etc.

[0092] In some embodiments, the application type may also include other types, and the corresponding candidate user behaviors may also include other types, which may be determined based on actual conditions.

[0093] Step 520 , based on the quintuple and the candidate quintuple, calculate the similarity between the quintuple and the candidate quintuple through a similarity model, and determine the probability of occurrence of each of the at least one candidate user behavior based on the similarity.

[0094] In some embodiments, the candidate five-tuple may refer to five-tuple data extracted from a network data packet corresponding to when the candidate behavior occurs.

[0095] In some embodiments, the candidate quintuple can be obtained by the processor in a variety of ways. For example, the processor can extract historical network data packets and their corresponding historical behaviors from the historical data of the current user, extract historical quintuples from the historical network data packets, and through the matching of candidate user behaviors and historical behaviors, use the historical quintuples corresponding to the historical behaviors consistent with the candidate user behaviors as the candidate quintuples corresponding to the candidate user behaviors. For another example, the processor can obtain different standard quintuples that the user has pre-simulated based on different candidate user behaviors, and use the standard quintuples corresponding to different candidate user behaviors as candidate quintuples.

[0096] In some embodiments, the similarity model can be used to calculate the similarity between the quintuple (i.e., the current quintuple) and the candidate quintuple. The similarity model can be a machine learning model, such as a deep neural network model (DNN), a convolutional neural network (CNN), etc., or other models that can perform the same processing.

[0097] The input of the similarity model may include the network flow features of the current quintuple and the network flow features of the candidate quintuple. The network flow features of the current quintuple and the network flow features of the candidate quintuple can be determined by the transformer layer of the classification model. For more details, please refer to this specification. Figure 4 Related description in .

[0098] The output of the similarity model may be the similarity between the current quintuple and the candidate quintuple, which may be represented by a numerical value between [0,1]. The larger the numerical value, the higher the similarity between the two.

[0099] In some embodiments, the similarity model can be obtained through multiple third training samples with labels. For example, multiple third training samples with labels can be input into the initial similarity model, and a loss function can be constructed through the results of the labels and the initial similarity model. The parameters of the initial similarity model are iteratively updated based on the loss function through gradient descent or other methods. When the preset conditions are met, the model training is completed and a trained similarity model is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0100] In some embodiments, the third training sample may include a sample quintuple and a sample standard quintuple, and the training label indicates whether the sample quintuple and the sample standard quintuple are network flows generated by the same candidate user behavior under the same application type, and may be marked with 0 or 1, where 0 indicates no and 1 indicates yes. The training label may be determined by historical data or obtained through manual annotation.

[0101] In some embodiments, if the candidate quintuple is obtained based on the historical data of the current user, the input of the similarity model may also include time period features. Time period features can be obtained through the timestamp embedding layer of the classification model. For more details, please refer to this specification. Figure 4 Related description in .

[0102] Correspondingly, when the candidate quintuple is obtained based on the historical data of the current user, the third training sample also includes historical period features.

[0103] In some embodiments, the processor may determine the probability of the candidate user behavior occurring based on the output result of the similarity model. For example, the output result of the similarity model may be used as the probability of the candidate user behavior occurring. Specifically, when the output result of the similarity model is 0.7, the probability of the candidate user behavior occurring is 70%.

[0104] Step 530: Determine a target user behavior based on the probability difference between at least one candidate user behavior.

[0105] In some embodiments, when the probability difference between at least one candidate user behavior is greater than a threshold, the candidate user behavior with the highest probability is used as the target user behavior (i.e., user behavior). The probability difference between any two candidate user behaviors can be the absolute value of the difference between the two probabilities. The threshold can be set manually according to actual conditions.

[0106] In some embodiments, when the probability difference between at least one candidate user behavior is not greater than a threshold, the processor may perform a binary classification judgment on each of the at least one candidate user behavior in combination with relevant devices, and determine whether to use the candidate user behavior as the target user behavior based on the result of the binary classification judgment.

[0107] In some embodiments, the processor may perform a binary classification judgment on the candidate user behavior through a judgment model, wherein the judgment model may be a binary classification model.

[0108] The input of the judgment model may include the network flow characteristics of the current five-tuple, the probability of occurrence of the candidate behavior, the current device type, the related device types and the statistical information of the related devices.

[0109] Among them, the network flow characteristics of the current five-tuple can be determined by the transformer layer of the classification model. For more details, please refer to this manual Figure 4 Related description in .

[0110] The probability of the candidate behavior occurring may be determined based on the output of the similarity model. For more details about the similarity model and determining the probability of the candidate behavior occurring based on the similarity, please refer to the relevant description in step 520.

[0111] The current device type may refer to the type of device currently being used by the user, including but not limited to a computer, a mobile phone, etc.

[0112] The related device type can refer to the type of terminal that is related to the device used by the current user. For example, if the current user is using a computer and is also logged into the same account with a mobile phone, then the mobile phone can be a related device; for another example, if the current user is using a computer and a colleague in the same space is also using a computer, then the computer used by the other colleague can be considered a related device. There can be multiple related devices, for example, the devices that appear most frequently in the same time period can be considered related devices.

[0113] In some embodiments, the terminals with correlation may refer to multiple terminals whose time distribution at the same network access point meets a preset condition. For example, the devices that access the same network access point as the current user's device within a certain period of time may be regarded as related devices.

[0114] In some embodiments, the type of the current device and the type of related devices can be determined based on a trained device type judgment model. The device type judgment model can be a machine learning model, such as a deep learning model (DNN), etc., or other models that can complete the corresponding processing. The input of the device type judgment model may include the five-tuple network flow features and application types of the device at multiple historical time points, and the output may include the device type. The device type judgment model can be obtained through multiple labeled training samples through gradient descent or other training methods. The training samples may include sample five-tuple network flow features and sample application types at multiple time points in the historical data, and the training labels may include the device types corresponding to the aforementioned historical data.

[0115] The statistical information of the relevant device may include the statistical frequency of the network flow characteristics of the relevant device being determined to be the candidate user behavior, for example, the relevant device has an average of 3 candidate user behaviors per hour.

[0116] The output of the judgment model can be a binary classification result, which can be represented by 1 or 0.

[0117] In some embodiments, the judgment model can be trained based on multiple fourth training samples with labels. In some embodiments, the fourth training sample can at least include the network flow characteristics of the historical five-tuple, the probability of the historical candidate behavior, the type of historical equipment, the type of historical related equipment, and the statistical information of the historical related equipment. The trained label can indicate whether the historical candidate behavior is consistent with the historical target user behavior, where 1 can be used to indicate consistency and 0 can be used to indicate inconsistency. The label can be obtained based on historical data or through manual annotation.

[0118] In some embodiments, the processor may use the candidate user behavior for which the judgment model output result is 1 as the target user behavior.

[0119] In some embodiments of the present specification, based on related devices, the candidate user behaviors are further classified into two categories through a judgment model, combined with the association between terminal devices, so that a more accurate judgment can be made when the probability difference between at least one candidate user behavior is less than a threshold.

[0120] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0121] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0122] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0123] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0124] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0125] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0126] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A network flow partitioning method, characterized in that: The method comprises: Get network data packets; Extracting a quintuple and corresponding payload length, sending timestamp and receiving timestamp from the network data packet, and dividing the quintuple into a forward quintuple and a backward quintuple, the quintuple including at least one of a source IP address, a source port, a destination IP address, a destination port and a transport layer protocol; Based on a preset time interval, the forward quintuple and the backward quintuple are respectively blocked to obtain a plurality of forward quintuple blocks and a plurality of backward quintuple blocks; wherein two adjacent forward quintuple blocks have a time overlap, and two adjacent backward quintuple blocks have a time overlap; determining a transmission delay based on the transmission timestamp and the reception timestamp; and Based on the five-tuple, determining an event classification, the event classification including an application type and a user behavior; wherein, based on the five-tuple, determining the user behavior includes: Based on the application type, determining at least one candidate user behavior; Based on the quintuple and the candidate quintuple, calculating the similarity between the quintuple and the candidate quintuple through a similarity model, wherein the candidate quintuple refers to quintuple data extracted from a network data packet corresponding to when the candidate behavior occurs, and the similarity model is a machine learning model; Determining a probability of occurrence of each of the at least one candidate user behavior based on the similarity; When the probability difference between the at least one candidate user behavior is not greater than a threshold, for each of the at least one candidate user behavior, a binary classification judgment is performed on the candidate user behavior through a judgment model, and based on the result of the binary classification judgment, it is determined whether to use the candidate user behavior as the user behavior; wherein the input of the judgment model includes the network flow characteristics of the current quintuple, the probability of occurrence of the candidate behavior, the current device type, the related device type and the statistical information of the related devices, the probability of occurrence of the candidate behavior is determined based on the output of the similarity model, the current device type refers to the type of device currently being used by the current user, the related device type refers to the type of terminal that is correlated with the device currently being used by the current user, and the related terminal refers to multiple terminals whose time distribution at the same network access point meets a preset condition.

2. The method according to claim 1, characterized in that: Determining the application type includes: Based on the forward quintuple block and the backward quintuple block, determining the application type through a classification model, wherein the classification model is a machine learning model; The input data of the classification model is related to the payload length of the forward quintuple block and the payload length of the backward quintuple block.

3. The method according to claim 2, characterized in that The determining the user behavior based on the quintuple further includes: When the probability difference between the at least one candidate user behavior is greater than the threshold, the candidate user behavior with the highest probability is used as the user behavior.

4. The method according to claim 1, characterized in that: The similarity model is obtained through multiple third training samples with labels, and the third training samples include a sample quintuple and a sample standard quintuple. The training labels represent whether the sample quintuple and the sample standard quintuple are network flows generated by the same candidate user behavior under the same application type.

5. The method according to claim 1, characterized in that In response to the candidate quintuple being acquired based on the historical data of the current user, the input of the similarity model further includes a time period feature.

6. A network flow partitioning system, characterized in that: The system includes an acquisition module, an extraction module and a determination module; The acquisition module is used to acquire network data packets; The extraction module is used to extract a quintuple and a corresponding effective load length, a sending timestamp and a receiving timestamp from the network data packet, and divide the quintuple into a forward quintuple and a backward quintuple, wherein the quintuple includes at least one of a source IP address, a source port, a destination IP address, a destination port and a transport layer protocol, and based on a preset time interval, the forward quintuple and the backward quintuple are respectively divided into blocks to obtain a plurality of forward quintuple blocks and a plurality of backward quintuple blocks; wherein two adjacent forward quintuple blocks have a time overlap, and two adjacent backward quintuple blocks have a time overlap; The sending timestamp and the receiving timestamp are used to determine transmission delay; The determination module is used for: Based on the five-tuple, determining an event classification, the event classification including an application type and a user behavior; wherein, based on the five-tuple, determining the user behavior includes: Based on the application type, determining at least one candidate user behavior; Based on the quintuple and the candidate quintuple, calculating the similarity between the quintuple and the candidate quintuple through a similarity model, wherein the candidate quintuple refers to quintuple data extracted from a network data packet corresponding to when the candidate behavior occurs, and the similarity model is a machine learning model; Determining a probability of occurrence of each of the at least one candidate user behavior based on the similarity; When the probability difference between the at least one candidate user behavior is not greater than a threshold, for each of the at least one candidate user behavior, a binary classification judgment is performed on the candidate user behavior through a judgment model, and based on the result of the binary classification judgment, it is determined whether to use the candidate user behavior as the user behavior; wherein the input of the judgment model includes the network flow characteristics of the current quintuple, the probability of occurrence of the candidate behavior, the current device type, the related device type and the statistical information of the related devices, the probability of occurrence of the candidate behavior is determined based on the output of the similarity model, the current device type refers to the type of device currently being used by the current user, the related device type refers to the type of terminal that is correlated with the device currently being used by the current user, and the related terminal refers to multiple terminals whose time distribution at the same network access point meets a preset condition.

7. The system according to claim 6, characterized in that Determining the application type includes: Based on the forward quintuple block and the backward quintuple block, determining the application type through a classification model, wherein the classification model is a machine learning model; The input data of the classification model is related to the payload length of the forward quintuple block and the payload length of the backward quintuple block.

8. A network flow segmentation device, comprising a processor, wherein the processor is used to execute the network flow segmentation method according to any one of claims 1 to 5.

9. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the network flow division method according to any one of claims 1 to 5.

Images