Data stream detection method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a data stream detection method and device, computer equipment and a storage medium. Acquiring a data stream in a network to be detected and data stream information thereof; determining a first target hash bucket of the data stream in the first hash bucket array according to the first hash value of the data stream, and determining a second target hash bucket of the data stream in the second hash bucket array according to the second hash value of the data stream; obtaining the occurrence frequency and the window occurrence frequency corresponding to the data stream, and determining a storage position corresponding to the data stream in the first target hash bucket or the second target hash bucket according to the occurrence frequency and the window occurrence frequency; storing data stream information corresponding to the data stream in the storage position to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; and according to the hash bucket data, in the first hash bucket array and the second hash bucket array, determining a target data stream of which the network rate or the occurrence frequency does not meet a preset condition, and performing network security detection on the target data stream.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a data stream detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the explosive growth of network users and network scale, the massive traffic within the network has become unpredictable, posing significant challenges to the operation and management of network infrastructure. Within the network's data flow, there exist network data flows used for cyberattacks. These flows can target servers, network infrastructure, and other systems, ultimately leading to cybersecurity incidents.

[0003] Some network attacks targeting specific network data streams are highly stealthy, tending to continuously and covertly infiltrate the target's database. Because of their persistence, these attacks are difficult to detect. Related technologies address this by calculating the weights of different network data streams; higher weights are more likely to indicate a persistent attack. However, these weights are often based on empirical parameters or manually set, leading to inaccurate detection of persistent attack streams and potentially resulting in cybersecurity incidents such as compromised target databases. Summary of the Invention

[0004] This application provides a data stream detection method, apparatus, computer device, and storage medium, which can improve the accuracy of network security detection of data streams in the network.

[0005] To achieve the above objectives, one aspect of this application provides a data stream detection method, including: Acquire the data stream in the network to be detected and the data stream information of the data stream; The first hash value of the data stream is determined according to the first hash function, and the second hash value of the data stream is determined according to the second hash function; The first target hash bucket corresponding to the data stream is determined in the first hash bucket array based on the first hash value, and the second target hash bucket corresponding to the data stream is determined in the second hash bucket array based on the second hash value. Obtain the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count; The data stream information corresponding to the data stream is stored in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; Based on the hash bucket data in the first hash bucket array and the second hash bucket array, target data streams whose network rate or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on the target data streams.

[0006] To achieve the above objectives, one aspect of this application provides a data stream detection apparatus, including: The first acquisition module is used to acquire the data stream in the network to be detected and the data stream information of the data stream; The first determining module is used to determine the first hash value of the data stream according to the first hash function and the second hash value of the data stream according to the second hash function. The second determining module is used to determine the first target hash bucket corresponding to the data stream in the first hash bucket array based on the first hash value, and to determine the second target hash bucket corresponding to the data stream in the second hash bucket array based on the second hash value; The second acquisition module is used to acquire the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count. The data storage module is used to store the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; The security detection module is used to determine, based on the hash bucket data in the first hash bucket array and the second hash bucket array, target data streams whose network rate or frequency of occurrence does not meet preset conditions, and to perform network security detection on the target data streams.

[0007] In some implementations, the second acquisition module is used for: When the occurrence count and the occurrence window count in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the first target hash bucket based on the occurrence count and the occurrence window count. When the occurrence count and occurrence window count in the first target hash bucket array are both zero, and there is a free storage location in the first target hash bucket, the free storage location in the first target hash bucket is determined as the storage location corresponding to the data stream. When the occurrence count and occurrence window count in the first target hash bucket array are both zero, and there is no free storage location in the first target hash bucket, determine whether there is a data stream to be replaced in the first target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; When there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream.

[0008] In some implementations, the second acquisition module is used for: When the occurrence count and occurrence window count in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the current storage position of the data stream in the first target hash bucket, and obtain the target historical data stream of the target storage position sorted before the current storage position, as well as the target historical occurrence window count corresponding to the target historical data stream; When the number of occurrence windows is greater than the number of occurrence windows in the target history, the target storage location is determined as the storage location corresponding to the data stream; When the number of occurrence windows equals the number of occurrence windows in the target historical data stream, the historical occurrence count of the target historical data stream is obtained; The historical occurrence count is compared with the occurrence count to obtain a comparison result, and the storage location corresponding to the data stream is determined from the current storage location and the target storage location based on the comparison result.

[0009] In some implementations, the second acquisition module is used for: After obtaining the target historical data stream of the target storage location sorted before the current storage location, and the number of target historical occurrence windows corresponding to the target historical data stream, when the number of occurrence windows is less than the number of target historical occurrence windows, the candidate storage location of the previous sorted target storage location is determined. The candidate storage location is determined as the target storage location, the historical data stream corresponding to the candidate storage location is determined as the target historical data stream, and the process returns to the step of obtaining the number of times the target historical occurrence window is obtained corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

[0010] In some implementations, the second acquisition module is used for: When both the occurrence count and the occurrence window count in the first target hash bucket array are not zero, and the window corresponding to the data stream is a new window, the occurrence frequency of the data stream in the current window is obtained, and the ratio between the occurrence count and the occurrence window count is determined. When the occurrence frequency is greater than or equal to the ratio, a first traffic trend value is set for the data stream, and the current storage position of the data stream in the first target hash bucket is determined. The target historical data stream of the target storage position sorted before the current storage position is obtained, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream. When the historical traffic trend value is the second traffic trend value, the target storage location is determined as the storage location corresponding to the data stream; When the target historical traffic trend value is the first traffic trend value, and the number of occurrence windows is greater than the target historical occurrence window number, the target storage location is determined as the storage location corresponding to the data stream; When the occurrence frequency is less than the ratio, the second flow trend value is set for the data stream, and the current storage location is determined as the storage location corresponding to the data stream.

[0011] In some implementations, the second acquisition module is used for: After obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, when the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is less than or equal to the number of historical occurrence windows, the candidate storage location of the previous sorted target storage location is determined. The candidate storage location is determined as the target storage location, the historical data stream corresponding to the candidate storage location is determined as the target historical data stream, and the process returns to the step of obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

[0012] In some implementations, the second acquisition module is used for: After determining whether there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, when there is no data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, it is determined whether the occurrence count and the occurrence window count in the second target hash bucket array are both zero. When the occurrence count and the occurrence window count in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the second target hash bucket based on the occurrence count and the occurrence window count. When the occurrence count and occurrence window count in the second target hash bucket array are both zero, and there is a free storage location in the second target hash bucket, the free storage location in the second target hash bucket is determined as the storage location corresponding to the data stream; When the occurrence count and occurrence window count in the second target hash bucket array are both zero, and there is no free storage location in the second target hash bucket, determine whether there is a data stream to be replaced in the second target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; When there is a data stream to be replaced in the second target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream. When there is no data stream to be replaced in the second target hash bucket whose network rate or frequency does not meet the preset storage conditions, the storage position corresponding to the data stream is determined from the first storage position at the end of the sorting in the first target hash bucket and the second storage position at the end of the sorting in the second target hash bucket.

[0013] In some implementations, the second acquisition module is used for: Determine the first occurrence count of the first data stream at the last first storage location of the sorted first target hash bucket, and determine the second occurrence count of the second data stream at the last second storage location of the sorted second target hash bucket; When the first occurrence count of the first data stream is less than the second occurrence count of the second data stream, the first occurrence count is attenuated to obtain the target first occurrence count; When the first occurrence count of the target is less than the preset occurrence count, the first storage location is determined as the storage location corresponding to the data stream.

[0014] To achieve the above objectives, one aspect of this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the data flow detection method provided in this application.

[0015] To achieve the above objectives, one aspect of this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the data flow detection method provided in this application.

[0016] In this embodiment, the following steps are taken: First, the data stream in the network to be detected and its data stream information are acquired. A first hash value for the data stream is determined using a first hash function, and a second hash value is determined using a second hash function. A first target hash bucket corresponding to the data stream is determined in a first hash bucket array based on the first hash value, and a second target hash bucket corresponding to the data stream is determined in a second hash bucket array based on the second hash value. The occurrence count and occurrence window count of the data stream are acquired, and the storage location corresponding to the data stream is determined in the first or second target hash bucket based on these counts. The data stream information is stored in the storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network rate or occurrence frequency does not meet preset conditions are identified, and network security detection is performed on the target data streams.

[0017] Therefore, by acquiring the data stream and its information, the first hash value of the data stream is determined using a first hash function, and the second hash value is determined using a second hash function. Based on the first hash value, the first target hash bucket corresponding to the data stream is determined in the first hash bucket array, and based on the second hash value, the second target hash bucket corresponding to the data stream is determined in the second hash bucket array. This allows for accurate determination of the data stream's position within the hash bucket array. By using two hash bucket arrays, the number of data streams stored can be increased, and the hash bucket position of the data stream can be flexibly adjusted. Then, the occurrence count and occurrence window count of the data stream are obtained, and based on these counts, the storage location of the data stream is determined in either the first or second target hash bucket. This ensures that the data stream requiring detection is stored in an appropriate storage location within either the first or second target hash bucket, preventing it from being replaced by subsequent updates. Finally, the data stream information corresponding to the data stream is stored in a storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network speed or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on these target data streams. This allows for the storage of some data streams that need to be detected in hash bucket arrays, and subsequent network security detection can be performed on target data streams whose network speed or frequency of occurrence does not meet preset conditions to ensure network security.

[0018] Compared to related technologies that analyze whether a network data stream is a continuous attack by setting corresponding weights for different data streams, this application first stores the relevant information of different data streams in a hash bucket array, and then determines the target data streams that do not meet the preset conditions by network speed or frequency of occurrence. This can more accurately identify target data streams with network security risks and perform security detection on the target data streams, thereby improving the accuracy of network security detection of data streams in the network.

[0019] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram illustrating a scenario for network security detection provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the data stream detection method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the first target hash bucket provided in an embodiment of this application; Figure 4 This is another schematic flowchart of the data stream detection method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the data stream detection device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] It is understood that in the specific implementation of this application, information related to data flow is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0024] The data flow detection method provided in this application relates to the field of artificial intelligence technology. This data flow detection method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the data flow detection method, but is not limited to the above forms.

[0025] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations: Hash Bucket Array: A hash bucket array is an important component of the hash table data structure. A hash table is a data structure used to store key-value pairs, which uses a hash function to calculate the storage location corresponding to the key. The hash bucket array is the array used to store these key-value pairs, and each element in the array is a hash bucket.

[0027] A hash bucket is a basic storage unit in a hash bucket array. It stores data elements that, after being hashed, are mapped to the same position (index). A bucket is where entries are stored. When a hash collision occurs, multiple entries may be stored in the same bucket. For example, in a bucket structured as a linked list, each entry (existing as a key-value pair) is stored as a node in the linked list. If two different keys are hashed and mapped to the same bucket, then the entries corresponding to those two keys will be stored in that bucket.

[0028] Entry: In a hash bucket structure, an "entry" typically refers to a basic unit stored within a hash bucket, also known as an entry, specifically a "key-value pair." It consists of two parts: a key, used to calculate the index of the hash bucket using a hash function; and a value, the data content associated with the key. The storage location of an entry in the hash bucket array is indirectly determined by the buckets. Each entry is assigned an index using a hash function, which points to a bucket located within the hash bucket array. Therefore, an entry is ultimately stored within a specific bucket in the hash bucket array.

[0029] Five-tuple information refers to a combination of five elements used to uniquely identify a network connection in the fields of network communication or security. These five elements include the source IP address, source port, destination IP address, destination port, and protocol type.

[0030] First, let's describe the technical problems existing in the relevant technologies: With the explosive growth of network users and network scale, the massive traffic within the network has become unpredictable, posing significant challenges to the operation and management of network infrastructure. Within the network's data flow, there exist network data flows used for cyberattacks. These flows can target servers, network infrastructure, and other systems, ultimately leading to cybersecurity incidents.

[0031] Some network attacks targeting specific network data streams are highly stealthy, tending to continuously and covertly infiltrate the target's database. Because of their persistence, these attacks are difficult to detect. Related technologies address this by calculating the weights of different network data streams; higher weights are more likely to indicate a persistent attack. However, these weights are often based on empirical parameters or manually set, leading to inaccurate detection of persistent attack streams and potentially resulting in cybersecurity incidents such as compromised target databases.

[0032] To address the technical problems existing in related technologies, this application obtains a data stream and its information, determines a first hash value of the data stream based on a first hash function, determines a second hash value of the data stream based on a second hash function, determines a first target hash bucket corresponding to the data stream in a first hash bucket array based on the first hash value, and determines a second target hash bucket corresponding to the data stream in a second hash bucket array based on the second hash value. This accurately determines the position of the data stream in the hash bucket array. By setting two hash bucket arrays, the number of data streams stored can be increased, and the hash bucket position of the data stream can be flexibly adjusted. Then, the occurrence count and occurrence window count of the data stream are obtained, and the storage location of the data stream is determined in the first or second target hash bucket based on the occurrence count and occurrence window count. This allows the data stream to be detected to be stored in an appropriate storage location in the first or second target hash bucket, preventing the data stream to be detected from being subsequently updated or replaced. Finally, the data stream information corresponding to the data stream is stored in a storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network speed or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on these target data streams. This allows for the storage of some data streams that need to be detected in hash bucket arrays, and subsequent network security detection can be performed on target data streams whose network speed or frequency of occurrence does not meet preset conditions to ensure network security.

[0033] Compared to related technologies that analyze whether a network data stream is a continuous attack by setting corresponding weights for different data streams, this application first stores the relevant information of different data streams in a hash bucket array, and then determines the target data streams that do not meet the preset conditions by network speed or frequency of occurrence. This can more accurately identify target data streams with network security risks and perform security detection on the target data streams, thereby improving the accuracy of network security detection of data streams in the network.

[0034] For a comprehensive understanding of the data stream detection method provided in the embodiments of this application, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of a network security detection scenario provided in an embodiment of this application.

[0035] In this embodiment, for the network to be detected, two hash bucket arrays can be set, namely the first hash bucket array and the second hash bucket array. The first hash bucket array and the second hash bucket array are hash bucket arrays.

[0036] The first hash bucket array contains multiple first hash buckets, and each first hash bucket contains multiple entries. Each entry can be used to store data stream information corresponding to the data stream, such as 5-tuple information, the number of times the data stream appears in the corresponding window, the number of times the data stream appears in the corresponding window, the number of times the data stream appears, etc. This network information can be used as the value, and the hash value of the data stream can be used as the key, thus forming a key-value pair data structure stored in the entry.

[0037] The second hash bucket array contains multiple second hash buckets, and each second hash bucket contains multiple entries. Each entry can be used to store network information corresponding to the data stream, such as quintuple information, the number of times the data stream appears in the corresponding window, the number of times the data stream appears in the corresponding window, the number of times the data stream appears, etc. This network information can be used as the value, and the hash value of the data stream can be used as the key, thus forming a key-value pair data structure stored in the entry.

[0038] The advantage of using two hash bucket arrays in this application is that it reduces hash collisions between hash values ​​corresponding to different data streams, thereby reducing the probability of hash collisions. This method can reduce hash collisions to a certain extent because two different hash functions distribute the data across two different arrays, resulting in a more even data distribution compared to a single hash bucket array. When searching for a data record, the search is first performed in the first hash bucket array; if the record is not found there, it is then searched in the second hash bucket array. Due to the reduced collisions, the average search length is also reduced accordingly, thus improving the efficiency of data retrieval.

[0039] In the network under inspection, when a data stream is detected, a first hash value (hash value 1) can be determined using a first hash function. Then, based on hash value 1, the first target hash bucket is found among multiple first hash buckets in the first hash bucket array. A second hash value (hash value 2) can be determined using a second hash function. Then, based on hash value 2, the second target hash bucket is found among multiple second hash buckets in the second hash bucket array. The data stream information can be stored in either the first or second target hash bucket.

[0040] Then, obtain the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count.

[0041] Store the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network speed or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on these target data streams. For example, in the hash bucket data, data streams with excessively high or low network speeds are detected, and data streams with excessively high or low frequency of occurrence are also detected.

[0042] For these target data streams, subsequent security checks can be performed using appropriate network security detection methods to determine whether the network data streams possess offensive capabilities. Compared to related technologies that analyze whether a network data stream constitutes a sustained attack by assigning weights to different data streams, this application first stores relevant information about different data streams in a hash bucket array. Then, it identifies target data streams that do not meet preset conditions based on network speed or frequency of occurrence. This allows for more accurate identification of target data streams with network security vulnerabilities and enables security checks on these streams, thereby improving the accuracy of network security detection of data streams within the network. This enhances the network security defense capabilities of the network under test and protects its network security.

[0043] As can be seen from the above, the data stream detection method provided in this application embodiment can be applied to the field of network security, such as cloud computing, cloud services and other application scenarios.

[0044] For a more detailed understanding of the data stream detection method provided in the embodiments of this application, please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating the data stream detection method provided in an embodiment of this application. The data stream detection method may include the following steps: Step 110: Obtain the data stream in the network to be detected and the data stream information of the data stream; Step 120: Determine the first hash value of the data stream according to the first hash function, and determine the second hash value of the data stream according to the second hash function; Step 130: Determine the first target hash bucket corresponding to the data stream in the first hash bucket array based on the first hash value, and determine the second target hash bucket corresponding to the data stream in the second hash bucket array based on the second hash value; Step 140: Obtain the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count; Step 150: Store the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; Step 160: Based on the hash bucket data, identify the target data streams in the first and second hash bucket arrays whose network rate or frequency of occurrence does not meet the preset conditions, and perform network security detection on the target data streams.

[0045] Steps 110 to 160 will be described in detail below.

[0046] In step 110, the data stream in the network to be detected and the data stream information of the data stream are obtained.

[0047] There are massive data streams in the network to be detected. The corresponding data streams can be acquired at a certain acquisition frequency, and then the data stream information can be determined, such as the five-tuple information, which is the source IP address, source port, destination IP address, destination port and protocol type of the data stream.

[0048] Data flow information can then be used for access control and intrusion detection. Firewalls can configure rules to allow or block certain network connections based on data flow information. For example, a company's firewall can set rules to only allow specific IP addresses on the internal network to access certain services on the external network through specific ports, thus preventing unauthorized external access. Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) can also utilize data flow information to analyze network traffic and detect abnormal network connections. For example, if a source IP address is found to frequently send large amounts of data to different destination IP addresses, and the ports and protocols used do not conform to normal business logic, it may be a malicious attack.

[0049] In step 120, the first hash value of the data stream is determined according to the first hash function, and the second hash value of the data stream is determined according to the second hash function.

[0050] The first hash function and the second hash function are functions that map a data stream of arbitrary length to a hash value of fixed length. They accept data blocks from the input data stream, perform a series of mathematical and logical operations, and output a fixed-length binary sequence, i.e., the hash value.

[0051] If the hash value of a hash bucket in the preset hash bucket array (first hash bucket array and second hash bucket array) is the same as the hash value of the data stream, then that hash bucket is the target hash bucket corresponding to the hash value of the data stream. If no hash bucket in the preset hash bucket array has the same hash value as the hash value of the data stream, a new target hash bucket can be created for the data stream.

[0052] In some implementations, the first hash function and the second hash function can be different hash functions, so for the same data stream, the first hash value calculated by the first hash function and the second hash value calculated by the second hash function are different.

[0053] Hash functions include MD5 (Message-Digest Algorithm 5), SHA-1 (Secure Hash Algorithm 1), and SHA-256. These hash functions perform complex calculations on the input data, so that even slight differences in the input data can result in significantly different output hash values. The first and second hash functions can be determined from these hash functions.

[0054] Once the first hash value is determined, if the hash value of a first hash bucket in the first hash bucket array is the same as the first hash value of the data stream, then that first hash bucket becomes the first target hash bucket corresponding to the hash value of the data stream. If no first hash bucket in the first hash bucket array has a hash value that matches the hash value of the data stream, a new first target hash bucket can be created for the data stream.

[0055] Once the second hash value is determined, if the hash value of a second hash bucket in the second hash bucket array is the same as the second hash value of the data stream, then that second hash bucket becomes the second target hash bucket corresponding to the hash value of the data stream. If no second hash bucket in the second hash bucket array has a hash value that matches the hash value of the data stream, a new second target hash bucket can be created for the data stream. The target hash bucket includes both the first target hash bucket and the second target hash bucket.

[0056] In step 130, the occurrence count and occurrence window count of the data stream are obtained, and the storage location of the data stream is determined in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count.

[0057] In this embodiment, the occurrence count and occurrence window count of the data stream can be determined. The occurrence count is the number of times the data stream is detected, and the occurrence window count is the number of windows in which the data stream appears within each detection window period. Specifically, the occurrence count and occurrence window count of the data stream can be determined by comparing the 5-tuple information of the data stream with the 5-tuple information stored in the first hash bucket array and the second hash bucket array.

[0058] If both the occurrence count and the occurrence window count of the data stream are zero, then the data stream has not been stored in the first hash bucket array or the second hash bucket array; if both the occurrence count and the occurrence window count of the data stream are not zero, then the data stream has been stored in the first hash bucket array or the second hash bucket array.

[0059] Then, based on the occurrence count and the occurrence window count, the storage location corresponding to the data stream is determined in the first or second target hash bucket.

[0060] For example, the quintuple information can be compared with the quintuple information already stored in the first hash bucket array to obtain the first comparison result.

[0061] If the first comparison result indicates that the quintuple information matches the quintuple information already stored in the first hash bucket array, and the first target hash bucket in the first hash bucket array has a storage location corresponding to the data stream, then the network information corresponding to the data stream can be directly stored in the first hash bucket array.

[0062] If the first comparison result indicates a mismatch between the 5-tuple information and the 5-tuple information already stored in the first hash bucket array, then the 5-tuple information is compared with the 5-tuple information already stored in the second hash bucket array to obtain a second comparison result. Based on this second comparison result, the storage location corresponding to the data stream is determined in the second target hash bucket. In other words, network information such as the 5-tuple information corresponding to the data stream, the average network rate, the number of occurrence windows of the data stream, and the frequency of the data stream's occurrence within the corresponding window are stored in either the first hash bucket array or the second hash bucket array.

[0063] The advantage of using two hash bucket arrays in this application is that it reduces hash collisions between hash values ​​corresponding to different data streams, thereby reducing the probability of hash collisions. This method can reduce hash collisions to a certain extent because two different hash functions distribute the data across two different arrays, resulting in a more even data distribution compared to a single hash bucket array. When searching for a data record, the search is first performed in the first hash bucket array; if the record is not found there, it is then searched in the second hash bucket array. Due to the reduced collisions, the average search length is also reduced accordingly, thus improving the efficiency of data retrieval.

[0064] In some implementations, the storage location corresponding to the data stream is determined in a first target hash bucket or a second target hash bucket based on the occurrence count and the occurrence window count, including: (1.1) When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the first target hash bucket based on the number of occurrences and the number of occurrence windows; (1.2) When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both zero, and there is a free storage location in the first target hash bucket, the free storage location in the first target hash bucket is determined as the storage location corresponding to the data stream; (1.3) When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both zero, and there is no free storage location in the first target hash bucket, determine whether there is a data stream to be replaced in the first target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; (1.4) When there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream.

[0065] Specifically, when the number of occurrences and the number of occurrence windows in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, it means that the data stream has appeared before. At this time, the storage location corresponding to the data stream is determined in the first target hash bucket based on the number of occurrences and the number of occurrence windows.

[0066] It should be noted that the storage location of the data stream is determined within the first target hash bucket based on the occurrence frequency and the occurrence window frequency. The main decision-making factor lies in the data stream detection task, which can include four types: 1. Continuous high-frequency stream detection. If the rate of a data stream is greater than a preset rate, then the data stream is a high-frequency data stream. The rate is the number of times the data stream occurs divided by the number of occurrence windows of the data stream.

[0067] Second, continuous low-frequency stream detection. If the rate of a data stream is less than the preset rate, then the data stream is a low-frequency data stream.

[0068] Third, continuously increasing flow detection. If the frequency of a data flow appearing in the current window is greater than the corresponding rate, then the data flow is a continuously increasing flow. The rate is the number of times the data flow appears divided by the number of windows in which the data flow appears. The frequency of the data flow appearing in the current window is the number of times the data flow appears in the current window.

[0069] Fourth, continuously decreasing flow detection. If the frequency of a data flow appearing in the current window is less than the rate corresponding to that data flow, then that data flow is a continuously decreasing flow.

[0070] For the four types of data stream detection tasks mentioned above, the storage location of the current data stream can be determined according to the different data stream detection tasks.

[0071] In some implementations, if the number of occurrences and the number of occurrence windows in the first target hash bucket array are both zero, and there is an empty storage location in the first target hash bucket, then that empty storage location is directly determined as the storage location of the data stream.

[0072] In some implementations, when the number of occurrences in the first target hash bucket array and the number of occurrences in the window are both non-zero, and the window corresponding to the data stream is not a new window, then the number of occurrences of the data stream can be continued to be accumulated.

[0073] First, let's describe it from the perspective of continuous high-frequency stream detection and continuous low-frequency data stream detection.

[0074] In some implementations, when both the occurrence count and the occurrence window count in the first target hash bucket array are non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined within the first target hash bucket based on the occurrence count and the occurrence window count, including: (1.1.1) When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the current storage position of the data stream in the first target hash bucket, and obtain the target historical data stream of the target storage position sorted before the current storage position, as well as the number of target historical occurrence windows corresponding to the target historical data stream; (1.1.2) When the number of occurrences of a window is greater than the number of occurrences of a target window in history, the target storage location is determined as the storage location corresponding to the data stream; (1.1.3) When the number of occurrence windows equals the number of occurrence windows in the target's historical data stream, obtain the historical occurrence count of the target's historical data stream; (1.1.4) Compare the historical occurrence count with the occurrence count to obtain the comparison result, and determine the storage location corresponding to the data stream between the current storage location and the target storage location based on the comparison result.

[0075] Specifically, when the number of occurrences and the number of occurrence windows in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the current storage position of the data stream in the first target hash bucket is determined, and the target historical data stream of the target storage position sorted before the current storage position, as well as the number of occurrence windows of the target historical data stream corresponding to the target historical data stream, are obtained.

[0076] Please refer to the following for details. Figure 3 , Figure 3 This is a schematic diagram of the structure of the first target hash bucket provided in an embodiment of this application. In the first target hash bucket, entries can be sorted from high to low; after data is stored in the entry corresponding to the previous storage location, new data is stored in the entry corresponding to the next storage location.

[0077] Once the current storage location of the first target hash bucket is determined in an entry of the first target hash bucket, the target historical data stream that is sorted before the current storage location is then determined. The target storage location can be the previous storage location before the current storage location.

[0078] When the number of occurrences of a window exceeds the number of occurrences of a target window in history, the target storage location is determined as the storage location corresponding to the data stream.

[0079] When the number of occurrence windows equals the number of occurrence windows in the target historical data stream, the historical occurrence count of the target historical data stream is obtained, the historical occurrence count is compared with the current occurrence count to obtain the comparison result, and the storage location corresponding to the data stream is determined between the current storage location and the target storage location based on the comparison result.

[0080] For example, when the data stream detection task is a continuous high-frequency stream detection task, if the historical occurrence count is less than the current occurrence count, the target storage location is determined as the storage location corresponding to the data stream. This might involve clearing the data in the target historical data stream at the target storage location and then setting the cleared storage location as the corresponding storage location. If the historical occurrence count is greater than or equal to the current occurrence count, the previous ranked candidate storage location is determined, and this candidate storage location is set as the target storage location. The historical data stream corresponding to this candidate storage location is then set as the target historical data stream, and the process returns to retrieving the target historical occurrence window count for the target historical data stream, continuing until the storage location corresponding to the data stream is determined. The advantage of this approach is that high-frequency data streams can be placed towards the end of the target hash bucket, preserving them as much as possible for subsequent analysis.

[0081] For example, when the data stream detection task is a continuous low-frequency stream detection task, if the historical occurrence count is greater than or equal to the occurrence count, the target storage location is determined as the storage location corresponding to the data stream. This might involve clearing the data in the target historical data stream at the target storage location and then setting the cleared storage location as the corresponding storage location. If the historical occurrence count is less than the occurrence count, the previous ranked candidate storage location is determined, and this candidate storage location is set as the target storage location. The historical data stream corresponding to this candidate storage location is then set as the target historical data stream, and the process returns to the step of obtaining the target historical occurrence window count corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined. The advantage of this approach is that low-frequency data streams can be placed towards the end of the target hash bucket, preserving them as much as possible for subsequent analysis.

[0082] In some implementations, after obtaining the target historical data stream of the target storage location sorted before the current storage location, and the number of target historical occurrence windows corresponding to the target historical data stream, the method further includes: (1.1.5) When the number of occurrences of a window is less than the number of occurrences of a target window in history, determine the previous sorted candidate storage location of the target storage location; (1.1.6) Determine the storage location to be selected as the target storage location, determine the historical data stream corresponding to the storage location to be selected as the target historical data stream, and return to the step of obtaining the number of times the target historical occurrence window is obtained for the target historical data stream, until the storage location corresponding to the data stream is determined.

[0083] When the number of occurrences of a window is less than the number of occurrences of the target historical data stream, it indicates that the target historical data stream is also important, but the data stream cannot occupy the storage location of the target historical data stream. In this case, the storage location of the data stream needs to be found in an earlier storage location.

[0084] For example, determine the candidate storage location in the previous sort of the target storage location, determine the candidate storage location as the target storage location, determine the historical data stream corresponding to the candidate storage location as the target historical data stream, and return to execute the step of obtaining the number of times the target historical occurrence window is obtained corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

[0085] The advantage of doing this is that some data streams in the target hash bucket that do not conform to the data stream detection task can be cleared, thus retaining the data streams that need to be detected.

[0086] The above describes the relevant content of step (1.1) from the perspective of continuous high-frequency stream detection and continuous low-frequency data stream detection.

[0087] The following will describe the relevant content of step (1.1) from the perspective of continuously increasing the flow.

[0088] In some implementations, when both the occurrence count and the occurrence window count in the first target hash bucket array are non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined within the first target hash bucket based on the occurrence count and the occurrence window count, including: (1.2.1) When the number of occurrences in the first target hash bucket array and the number of occurrences in the window are both non-zero, and the window corresponding to the data stream is a new window, obtain the frequency of occurrence of the data stream in the current window, and determine the ratio between the number of occurrences and the number of occurrences in the window; (1.2.2) When the frequency of occurrence is greater than or equal to the ratio, set the first flow trend value for the data stream, determine the current storage position of the data stream in the first target hash bucket, and obtain the target historical data stream of the target storage position sorted before the current storage position, as well as the target historical flow trend value and the number of target historical occurrence windows corresponding to the target historical data stream; (1.2.3) When the historical traffic trend value is the second traffic trend value, the target storage location is determined as the storage location corresponding to the data stream; (1.2.4) When the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is greater than the number of occurrence windows in the target history, the target storage location is determined as the storage location corresponding to the data stream; (1.2.5) When the frequency of occurrence is less than the ratio, set a second flow trend value for the data stream and determine the current storage location as the storage location corresponding to the data stream.

[0089] Specifically, when the number of occurrences in the first target hash bucket array and the number of occurrences in the window are both non-zero, and the window corresponding to the data stream is a new window, the frequency of occurrence of the data stream in the current window is obtained, and the ratio between the number of occurrences and the number of occurrences in the window is determined.

[0090] When the frequency of occurrence is greater than or equal to the ratio, a first flow trend value is set for the data stream, and the current storage position of the data stream in the first target hash bucket is determined. The target historical data stream of the target storage position sorted before the current storage position is obtained, as well as the target historical flow trend value and the number of target historical occurrence windows corresponding to the target historical data stream. When the historical traffic trend value is the second traffic trend value, the target storage location is determined as the storage location corresponding to the data stream. For example, if the second traffic trend value is 0 and the first traffic trend value is 1, then the data of the target historical data stream at the target storage location is cleared, and the target storage location is determined as the storage location corresponding to the data stream.

[0091] When the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is greater than the number of occurrence windows in the target history, the target storage location is determined as the storage location corresponding to the data stream.

[0092] When the frequency of occurrence is less than the ratio, a second flow trend value is set for the data stream, and the current storage location is determined as the storage location corresponding to the data stream. For example, the second flow trend value can be set to 0, and the current storage location can be determined as the storage location corresponding to the data stream.

[0093] The advantage of doing this is that the continuously growing data stream can be saved, making it easier to perform security checks on the ever-increasing data stream later.

[0094] In some implementations, after obtaining the target historical data stream of the target storage location sorted before the current storage location, and the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, the method further includes: (1.2.6) When the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is less than or equal to the number of occurrence windows in history, determine the previous sorted candidate storage location of the target storage location; (1.2.7) Determine the storage location to be selected as the target storage location, determine the historical data stream corresponding to the storage location to be selected as the target historical data stream, and return to execute the steps of obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of times the target historical occurrence window is corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

[0095] Among them, when the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is less than or equal to the number of occurrence windows in history, it means that the storage location of the target historical data stream cannot be replaced. At this time, the previous sorted candidate storage location of the target storage location is determined.

[0096] The process involves determining the candidate storage location as the target storage location, identifying the corresponding historical data stream as the target historical data stream, and then returning to the previous step of retrieving the target historical data stream of the target storage location that is sorted before the current storage location, along with the target historical traffic trend value and the number of times the target historical occurrence window appears. This process continues until the storage location corresponding to the data stream is determined. In other words, it involves identifying target historical data streams from the previous historical data streams that can replace the storage location, storing the data stream at the beginning of the target hash bucket to prevent it from being replaced or deleted, thus enabling subsequent security checks on the data stream.

[0097] The above describes the relevant content of step (1.1) from the perspective of continuously increasing flow.

[0098] Similarly, from the perspective of continuously decreasing flow, only the judgment condition needs to be changed. For example, if the flow trend value corresponding to continuously decreasing flow is the second flow trend value, which is 0, then the target storage location of the target historical data flow with the first flow trend value in the target hash bucket is cleared, and the target storage location is determined as the storage location of the data flow.

[0099] As can be seen from steps (1.1) to (1.4), we can search in the first target hash bucket of the first hash bucket array for a storage location that can store the data stream information. If no storage location for storing the data stream information is found in the first target hash bucket, we need to continue searching in the second target hash bucket array for a storage location for the data stream.

[0100] In some implementations, after determining whether a data stream to be replaced exists in the first target hash bucket due to a network rate or frequency that does not meet preset storage conditions, the method further includes: (2.1) When there is no data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, determine whether the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero; (2.2) When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the second target hash bucket based on the number of occurrences and the number of occurrence windows; (2.3) When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is a free storage location in the second target hash bucket, the free storage location in the second target hash bucket is determined as the storage location corresponding to the data stream; (2.4) When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is no free storage location in the second target hash bucket, determine whether there is a data stream to be replaced in the second target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; (2.5) When there is a data stream to be replaced in the second target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced shall be determined as the storage location corresponding to the data stream. (2.6) When there is no data stream to be replaced in the second target hash bucket whose network rate or frequency does not meet the preset storage conditions, the storage position corresponding to the data stream is determined from the first storage position at the end of the sorting in the first target hash bucket and the second storage position at the end of the sorting in the second target hash bucket.

[0101] Specifically, when there is no data stream to be replaced in the first target hash bucket whose network rate or frequency does not meet the preset storage conditions, it is determined whether the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero.

[0102] In some implementations, if the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is an empty storage location in the second target hash bucket, then that empty storage location is directly determined as the storage location of the data stream.

[0103] In some implementations, when the number of occurrences in the second target hash bucket array and the number of occurrences in the window are both non-zero, and the window corresponding to the data stream is not a new window, the number of occurrences of the data stream can be continued to be accumulated.

[0104] When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the second target hash bucket based on the number of occurrences and the number of occurrence windows.

[0105] For example, when the number of occurrences and the number of occurrence windows in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the current storage position of the data stream in the second target hash bucket, and obtain the target historical data stream of the target storage position sorted before the current storage position, as well as the number of target historical occurrence windows corresponding to the target historical data stream; When the number of occurrences of a window exceeds the number of occurrences of a target window in history, the target storage location is determined as the storage location corresponding to the data stream. When the number of occurrences of a window equals the number of occurrences of a target historical window, obtain the historical occurrence count of the target historical data stream. The historical occurrence count is compared with the current occurrence count to obtain the comparison result. Based on the comparison result, the storage location corresponding to the data stream is determined between the current storage location and the target storage location.

[0106] For example, after obtaining the target historical data stream of the target storage location sorted before the current storage location, and the number of target historical occurrence windows corresponding to the target historical data stream, it also includes: When the number of occurrences of a window is less than the number of occurrences of a target window in history, determine the previous sorted candidate storage location for the target storage location; The process involves determining the storage location to be selected as the target storage location, determining the historical data stream corresponding to the storage location to be selected as the target historical data stream, and returning to the step of obtaining the number of times the target historical occurrence window is obtained for the target historical data stream, until the storage location corresponding to the data stream is determined.

[0107] For example, when both the occurrence count and the occurrence window count in the second target hash bucket array are non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined within the second target hash bucket based on the occurrence count and the occurrence window count, including: When the number of occurrences in the second target hash bucket array and the number of occurrences in the window are both non-zero, and the window corresponding to the data stream is a new window, obtain the frequency of occurrence of the data stream in the current window, and determine the ratio between the number of occurrences and the number of occurrences in the window; When the frequency of occurrence is greater than or equal to the ratio, a second flow trend value is set for the data stream, and the current storage position of the data stream in the second target hash bucket is determined. The target historical data stream of the target storage position sorted before the current storage position is obtained, as well as the target historical flow trend value and the number of target historical occurrence windows corresponding to the target historical data stream. When the historical traffic trend value is the second traffic trend value, the target storage location is determined as the storage location corresponding to the data stream; When the target historical traffic trend value is the second traffic trend value and the number of occurrence windows is greater than the number of occurrence windows in the target history, the target storage location is determined as the storage location corresponding to the data stream. When the frequency of occurrence is less than the ratio, a second flow trend value is set for the data stream, and the current storage location is determined as the storage location corresponding to the data stream.

[0108] For example, after obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, it also includes: When the target historical traffic trend value is the second traffic trend value, and the number of occurrence windows is less than or equal to the number of occurrence windows in history, determine the previous sorted candidate storage location of the target storage location; The process involves determining the storage location to be selected as the target storage location, determining the historical data stream corresponding to the storage location to be selected as the target historical data stream, and returning to execute the steps of obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of times the target historical occurrence window is corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

[0109] When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is a free storage location in the second target hash bucket, the free storage location in the second target hash bucket is determined as the storage location corresponding to the data stream.

[0110] When the occurrence count and occurrence window count in the second target hash bucket array are both zero, and there is no free storage location in the second target hash bucket, determine whether there is a data stream to be replaced in the second target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions.

[0111] For example, when there is a data stream to be replaced in the second target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream. When there is no data stream to be replaced in the second target hash bucket whose network rate or frequency does not meet the preset storage conditions, the storage location corresponding to the data stream is determined from the first storage location at the end of the sorting in the first target hash bucket and the second storage location at the end of the sorting in the second target hash bucket.

[0112] In some implementations, determining the storage location corresponding to the data stream from the last sorted first storage location in the first target hash bucket and the last sorted second storage location in the second target hash bucket includes: (2.6.1) Determine the first occurrence count of the first data stream at the last storage location of the first target hash bucket, and determine the second occurrence count of the second data stream at the last storage location of the second target hash bucket; (2.6.2) When the first occurrence count of the first data stream is less than the second occurrence count of the second data stream, the first occurrence count is attenuated to obtain the target first occurrence count; (2.6.3) When the first occurrence number of the target is less than the preset occurrence number, the first storage location is determined as the storage location corresponding to the data stream.

[0113] Specifically, the first occurrence count of the first data stream at the last first storage location of the first target hash bucket is determined, and the second occurrence count of the second data stream at the last second storage location of the second target hash bucket is determined.

[0114] When the first occurrence count of the first data stream is less than the second occurrence count of the second data stream, it means that the second data stream occurs more often and needs to be retained. In this case, the first occurrence count is attenuated to obtain the target first occurrence count.

[0115] When the number of times the target appears is less than the preset number of times, the first storage location is determined as the storage location corresponding to the data stream. For example, the data corresponding to the first data stream in the first storage location is cleared, and the first storage location is then determined as the storage location corresponding to the data stream.

[0116] The advantage of doing this is that the storage location corresponding to the data stream can be found in the first target hash bucket and the second target hash bucket, and then saved.

[0117] In step 150, the data stream information corresponding to the data stream is stored in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array.

[0118] For example, network information such as the five-tuple information corresponding to the data stream, network speed, number of occurrence windows of the data stream, frequency of occurrence of the data stream in the corresponding window, and number of occurrences of the data stream are stored in the storage location. The hash value of the data stream is used as the key, and the network information of the data stream is used as the value, forming a key-value pair data structure stored in the storage location of the target hash bucket, so as to update the hash bucket data stored in the preset hash bucket array.

[0119] In step 160, based on the hash bucket data in the first hash bucket array and the second hash bucket array, target data streams whose network rate or frequency of occurrence does not meet the preset conditions are identified, and network security detection is performed on the target data streams.

[0120] For example, if the preset condition is that the network speed is lower than the first network speed, the target data stream with the lower network speed can be identified from the first hash bucket array and the second hash bucket array, and network security detection can be performed on the target data stream.

[0121] For example, if the preset condition is that the network speed is greater than the second network speed, the target data stream with a network speed greater than the second network speed can be determined from the first hash bucket array and the second hash bucket array, and network security detection can be performed on the target data stream. The first network speed is less than the second network speed.

[0122] For example, if the preset condition is that the network speed is lower than the first occurrence frequency, the target data stream with an occurrence frequency lower than the first occurrence frequency can be identified in the first hash bucket array and the second hash bucket array, and network security detection can be performed on the target data stream.

[0123] For example, if the preset condition is that the occurrence frequency is greater than the second occurrence frequency, the target data stream with an occurrence frequency greater than the second occurrence frequency can be identified in the first hash bucket array and the second hash bucket array, and network security detection can be performed on the target data stream. The first occurrence frequency is less than the second occurrence frequency.

[0124] This allows for filtering target data streams that meet preset criteria from the hash bucket data, improving the accuracy of target data stream filtering. This, in turn, enhances the accuracy of network security detection of persistent threat network data streams.

[0125] In this embodiment, the following steps are taken: First, the data stream in the network to be detected and its data stream information are acquired. A first hash value for the data stream is determined using a first hash function, and a second hash value is determined using a second hash function. A first target hash bucket corresponding to the data stream is determined in a first hash bucket array based on the first hash value, and a second target hash bucket corresponding to the data stream is determined in a second hash bucket array based on the second hash value. The occurrence count and occurrence window count of the data stream are acquired, and the storage location corresponding to the data stream is determined in the first or second target hash bucket based on these counts. The data stream information is stored in the storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network rate or occurrence frequency does not meet preset conditions are identified, and network security detection is performed on the target data streams.

[0126] Therefore, by acquiring the data stream and its information, the first hash value of the data stream is determined using a first hash function, and the second hash value is determined using a second hash function. Based on the first hash value, the first target hash bucket corresponding to the data stream is determined in the first hash bucket array, and based on the second hash value, the second target hash bucket corresponding to the data stream is determined in the second hash bucket array. This allows for accurate determination of the data stream's position within the hash bucket array. By using two hash bucket arrays, the number of data streams stored can be increased, and the hash bucket position of the data stream can be flexibly adjusted. Then, the occurrence count and occurrence window count of the data stream are obtained, and based on these counts, the storage location of the data stream is determined in either the first or second target hash bucket. This ensures that the data stream requiring detection is stored in an appropriate storage location within either the first or second target hash bucket, preventing it from being replaced by subsequent updates. Finally, the data stream information corresponding to the data stream is stored in a storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network speed or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on these target data streams. This allows for the storage of some data streams that need to be detected in hash bucket arrays, and subsequent network security detection can be performed on target data streams whose network speed or frequency of occurrence does not meet preset conditions to ensure network security.

[0127] Compared to related technologies that analyze whether a network data stream is a continuous attack by setting corresponding weights for different data streams, this application first stores the relevant information of different data streams in a hash bucket array, and then determines the target data streams that do not meet the preset conditions by network speed or frequency of occurrence. This can more accurately identify target data streams with network security risks and perform security detection on the target data streams, thereby improving the accuracy of network security detection of data streams in the network.

[0128] Please see Figure 4 , Figure 4 This is another schematic flowchart of the data stream detection method provided in this application embodiment. The data stream detection method may include the following steps: Step 201: Obtain the data stream in the network to be detected and the data stream information of the data stream; Step 202: Determine the first hash value of the data stream according to the first hash function, and determine the second hash value of the data stream according to the second hash function; Step 203: Determine the first target hash bucket corresponding to the data stream in the first hash bucket array based on the first hash value, and determine the second target hash bucket corresponding to the data stream in the second hash bucket array based on the second hash value; Step 204: Obtain the occurrence count and occurrence window count of the data stream. When the occurrence count and occurrence window count in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the storage location of the data stream in the first target hash bucket based on the occurrence count and occurrence window count. Step 205: When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both zero, and there is a free storage location in the first target hash bucket, the free storage location in the first target hash bucket is determined as the storage location corresponding to the data stream. Step 206: When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both zero, and there is no free storage location in the first target hash bucket, determine whether there is a data stream to be replaced in the first target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; Step 207: When there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream. Step 208: When there is no data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, determine whether the occurrence count and occurrence window count in the second target hash bucket array are both zero; Step 209: When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the storage location corresponding to the data stream in the second target hash bucket based on the number of occurrences and the number of occurrence windows. Step 210: When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is a free storage location in the second target hash bucket, the free storage location in the second target hash bucket is determined as the storage location corresponding to the data stream. Step 211: When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is no free storage location in the second target hash bucket, determine whether there is a data stream to be replaced in the second target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions. Step 212: When there is a data stream to be replaced in the second target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream. Step 213: When there is no data stream to be replaced in the second target hash bucket whose network rate or frequency does not meet the preset storage conditions, determine the storage position corresponding to the data stream from the first storage position at the end of the sorting in the first target hash bucket and the second storage position at the end of the sorting in the second target hash bucket. Step 214: Store the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; Step 215: Based on the hash bucket data, identify the target data streams in the first and second hash bucket arrays whose network rate or frequency of occurrence does not meet the preset conditions, and perform network security detection on the target data streams.

[0129] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not detailed in a particular embodiment, please refer to the detailed description of the data stream detection method above; they will not be repeated here. The figures only depict the individual steps; the order of the steps depends on the actual logic.

[0130] Please see Figure 5 , Figure 5 This is a schematic diagram of the data stream detection device provided in an embodiment of this application. This data stream detection device can perform the data stream detection method described in the above embodiments.

[0131] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0132] One aspect of this application provides a data stream detection device 300, including: The first acquisition module 310 is used to acquire the data stream in the network to be detected and the data stream information of the data stream; The first determining module 320 is used to determine the first hash value of the data stream according to the first hash function and the second hash value of the data stream according to the second hash function. The second determining module 330 is used to determine the first target hash bucket corresponding to the data stream in the first hash bucket array based on the first hash value, and to determine the second target hash bucket corresponding to the data stream in the second hash bucket array based on the second hash value. The second acquisition module 340 is used to acquire the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count. The data storage module 350 is used to store the data stream information corresponding to the data stream in the storage location to update the hash bucket data saved in the first hash bucket array or the second hash bucket array. The security detection module 360 ​​is used to determine, based on the hash bucket data in the first hash bucket array and the second hash bucket array, the target data stream whose network rate or frequency of occurrence does not meet the preset conditions, and to perform network security detection on the target data stream.

[0133] In some implementations, the second acquisition module 340 is used for: When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the first target hash bucket based on the number of occurrences and the number of occurrence windows. When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both zero, and there is a free storage location in the first target hash bucket, the free storage location in the first target hash bucket is determined as the storage location corresponding to the data stream. When the occurrence count and occurrence window count in the first target hash bucket array are both zero, and there is no free storage location in the first target hash bucket, determine whether there is a data stream to be replaced in the first target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; When there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream.

[0134] In some implementations, the second acquisition module 340 is used for: When the number of occurrences and the number of occurrence windows in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the current storage position of the data stream in the first target hash bucket, and obtain the target historical data stream of the target storage position sorted before the current storage position, as well as the number of target historical occurrence windows corresponding to the target historical data stream; When the number of occurrences of a window exceeds the number of occurrences of a target window in history, the target storage location is determined as the storage location corresponding to the data stream. When the number of occurrences of a window equals the number of occurrences of a target historical window, obtain the historical occurrence count of the target historical data stream. The historical occurrence count is compared with the current occurrence count to obtain the comparison result. Based on the comparison result, the storage location corresponding to the data stream is determined between the current storage location and the target storage location.

[0135] In some implementations, the second acquisition module 340 is used for: After obtaining the target historical data stream of the target storage location that is sorted before the current storage location, and the number of target historical occurrence windows corresponding to the target historical data stream, when the number of occurrence windows is less than the number of target historical occurrence windows, determine the candidate storage location of the previous sorted target storage location. The process involves determining the storage location to be selected as the target storage location, determining the historical data stream corresponding to the storage location to be selected as the target historical data stream, and returning to the step of obtaining the number of times the target historical occurrence window is obtained for the target historical data stream, until the storage location corresponding to the data stream is determined.

[0136] In some implementations, the second acquisition module 340 is used for: When the number of occurrences in the first target hash bucket array and the number of occurrences in the window are both non-zero, and the window corresponding to the data stream is a new window, obtain the frequency of occurrence of the data stream in the current window, and determine the ratio between the number of occurrences and the number of occurrences in the window; When the frequency of occurrence is greater than or equal to the ratio, a first flow trend value is set for the data stream, and the current storage position of the data stream in the first target hash bucket is determined. The target historical data stream of the target storage position sorted before the current storage position is obtained, as well as the target historical flow trend value and the number of target historical occurrence windows corresponding to the target historical data stream. When the historical traffic trend value is the second traffic trend value, the target storage location is determined as the storage location corresponding to the data stream; When the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is greater than the number of occurrence windows in the target history, the target storage location is determined as the storage location corresponding to the data stream. When the frequency of occurrence is less than the ratio, a second flow trend value is set for the data stream, and the current storage location is determined as the storage location corresponding to the data stream.

[0137] In some implementations, the second acquisition module 340 is used for: After obtaining the target historical data stream of the target storage location ranked before the current storage location, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, when the target historical traffic trend value is the first traffic trend value and the number of occurrence windows is less than or equal to the number of historical occurrence windows, the candidate storage location of the previous ranking of the target storage location is determined. The process involves determining the storage location to be selected as the target storage location, determining the historical data stream corresponding to the storage location to be selected as the target historical data stream, and returning to execute the steps of obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of times the target historical occurrence window is corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

[0138] In some implementations, the second acquisition module 340 is used for: After determining whether there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, if there is no data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, determine whether the occurrence count and occurrence window count in the second target hash bucket array are both zero. When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the second target hash bucket based on the number of occurrences and the number of occurrence windows. When the number of occurrences and the number of occurrence windows in the second target hash bucket array are both zero, and there is a free storage location in the second target hash bucket, the free storage location in the second target hash bucket is determined as the storage location corresponding to the data stream. When the occurrence count and occurrence window count in the second target hash bucket array are both zero, and there is no free storage location in the second target hash bucket, determine whether there is a data stream to be replaced in the second target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; When there is a data stream to be replaced in the second target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream. When there is no data stream to be replaced in the second target hash bucket whose network rate or frequency does not meet the preset storage conditions, the storage location corresponding to the data stream is determined from the first storage location at the end of the sorting in the first target hash bucket and the second storage location at the end of the sorting in the second target hash bucket.

[0139] In some implementations, the second acquisition module 340 is used for: Determine the first occurrence count of the first data stream at the last first storage location of the first target hash bucket, and determine the second occurrence count of the second data stream at the last second storage location of the second target hash bucket; When the first occurrence count of the first data stream is less than the second occurrence count of the second data stream, the first occurrence count is attenuated to obtain the target first occurrence count; When the number of times the target appears is less than the preset number of times it appears, the first storage location is determined as the storage location corresponding to the data stream.

[0140] In this embodiment, the first acquisition module 310 acquires the data stream and its data stream information in the network to be detected; the first determination module 320 determines the first hash value of the data stream according to the first hash function and the second hash value of the data stream according to the second hash function; the second determination module 330 determines the first target hash bucket corresponding to the data stream in the first hash bucket array according to the first hash value and the second target hash bucket corresponding to the data stream in the second hash bucket array according to the second hash value; the second acquisition module 340 acquires the occurrence count and occurrence window count of the data stream, and determines the storage location of the data stream in the first target hash bucket or the second target hash bucket according to the occurrence count and occurrence window count; the data storage module 350 stores the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; the security detection module 360 ​​determines the target data stream whose network rate or occurrence frequency does not meet the preset conditions in the first hash bucket array and the second hash bucket array according to the hash bucket data, and performs network security detection on the target data stream.

[0141] Therefore, by acquiring the data stream and its information, the first hash value of the data stream is determined using a first hash function, and the second hash value is determined using a second hash function. Based on the first hash value, the first target hash bucket corresponding to the data stream is determined in the first hash bucket array, and based on the second hash value, the second target hash bucket corresponding to the data stream is determined in the second hash bucket array. This allows for accurate determination of the data stream's position within the hash bucket array. By using two hash bucket arrays, the number of data streams stored can be increased, and the hash bucket position of the data stream can be flexibly adjusted. Then, the occurrence count and occurrence window count of the data stream are obtained, and based on these counts, the storage location of the data stream is determined in either the first or second target hash bucket. This ensures that the data stream requiring detection is stored in an appropriate storage location within either the first or second target hash bucket, preventing it from being replaced by subsequent updates. Finally, the data stream information corresponding to the data stream is stored in a storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network speed or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on these target data streams. This allows for the storage of some data streams that need to be detected in hash bucket arrays, and subsequent network security detection can be performed on target data streams whose network speed or frequency of occurrence does not meet preset conditions to ensure network security.

[0142] Compared to related technologies that analyze whether a network data stream is a continuous attack by setting corresponding weights for different data streams, this application first stores the relevant information of different data streams in a hash bucket array, and then determines the target data streams that do not meet the preset conditions by network speed or frequency of occurrence. This can more accurately identify target data streams with network security risks and perform security detection on the target data streams, thereby improving the accuracy of network security detection of data streams in the network.

[0143] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described data stream detection method. This computer device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0144] Please see Figure 6 , Figure 6 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes: The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 using the data flow detection method of the embodiments of this application. Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0145] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data stream detection method.

[0146] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0147] The data stream detection method, apparatus, computer equipment, and storage medium provided in this application embodiment acquire data streams and data stream information in the network to be detected; determine a first hash value of the data stream according to a first hash function, and a second hash value of the data stream according to a second hash function; determine a first target hash bucket corresponding to the data stream in a first hash bucket array according to the first hash value, and determine a second target hash bucket corresponding to the data stream in a second hash bucket array according to the second hash value; acquire the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream corresponding to the data stream in the first target hash bucket or the second target hash bucket according to the occurrence count and occurrence window count; store the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; determine target data streams whose network rate or occurrence frequency does not meet preset conditions in the first hash bucket array and the second hash bucket array according to the hash bucket data, and perform network security detection on the target data streams.

[0148] Therefore, by acquiring the data stream and its information, the first hash value of the data stream is determined using a first hash function, and the second hash value is determined using a second hash function. Based on the first hash value, the first target hash bucket corresponding to the data stream is determined in the first hash bucket array, and based on the second hash value, the second target hash bucket corresponding to the data stream is determined in the second hash bucket array. This allows for accurate determination of the data stream's position within the hash bucket array. By using two hash bucket arrays, the number of data streams stored can be increased, and the hash bucket position of the data stream can be flexibly adjusted. Then, the occurrence count and occurrence window count of the data stream are obtained, and based on these counts, the storage location of the data stream is determined in either the first or second target hash bucket. This ensures that the data stream requiring detection is stored in an appropriate storage location within either the first or second target hash bucket, preventing it from being replaced by subsequent updates. Finally, the data stream information corresponding to the data stream is stored in a storage location to update the hash bucket data stored in the first or second hash bucket array. Based on the hash bucket data in the first and second hash bucket arrays, target data streams whose network speed or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on these target data streams. This allows for the storage of some data streams that need to be detected in hash bucket arrays, and subsequent network security detection can be performed on target data streams whose network speed or frequency of occurrence does not meet preset conditions to ensure network security.

[0149] Compared to related technologies that analyze whether a network data stream is a continuous attack by setting corresponding weights for different data streams, this application first stores the relevant information of different data streams in a hash bucket array, and then determines the target data streams that do not meet the preset conditions by network speed or frequency of occurrence. This can more accurately identify target data streams with network security risks and perform security detection on the target data streams, thereby improving the accuracy of network security detection of data streams in the network.

[0150] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0151] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0153] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0154] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0155] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0157] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0159] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0160] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A data stream detection method, characterized in that, include: Acquire the data stream in the network to be detected and the data stream information of the data stream; The first hash value of the data stream is determined according to the first hash function, and the second hash value of the data stream is determined according to the second hash function; The first target hash bucket corresponding to the data stream is determined in the first hash bucket array based on the first hash value, and the second target hash bucket corresponding to the data stream is determined in the second hash bucket array based on the second hash value. Obtain the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count; The data stream information corresponding to the data stream is stored in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; Based on the hash bucket data in the first hash bucket array and the second hash bucket array, target data streams whose network rate or frequency of occurrence does not meet preset conditions are identified, and network security detection is performed on the target data streams.

2. The data stream detection method according to claim 1, characterized in that, Determining the storage location corresponding to the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and the occurrence window count includes: When the occurrence count and the occurrence window count in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the first target hash bucket based on the occurrence count and the occurrence window count. When the occurrence count and occurrence window count in the first target hash bucket array are both zero, and there is a free storage location in the first target hash bucket, the free storage location in the first target hash bucket is determined as the storage location corresponding to the data stream. When the occurrence count and occurrence window count in the first target hash bucket array are both zero, and there is no free storage location in the first target hash bucket, determine whether there is a data stream to be replaced in the first target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; When there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream.

3. The data stream detection method according to claim 2, characterized in that, When both the occurrence count and the occurrence window count in the first target hash bucket array are not zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the first target hash bucket based on the occurrence count and the occurrence window count, including: When the occurrence count and occurrence window count in the first target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, determine the current storage position of the data stream in the first target hash bucket, and obtain the target historical data stream of the target storage position sorted before the current storage position, as well as the target historical occurrence window count corresponding to the target historical data stream; When the number of occurrence windows is greater than the number of occurrence windows in the target history, the target storage location is determined as the storage location corresponding to the data stream; When the number of occurrence windows equals the number of occurrence windows in the target historical data stream, the historical occurrence count of the target historical data stream is obtained; The historical occurrence count is compared with the occurrence count to obtain a comparison result, and the storage location corresponding to the data stream is determined from the current storage location and the target storage location based on the comparison result.

4. The data stream detection method according to claim 3, characterized in that, After obtaining the target historical data stream of the target storage location sorted before the current storage location, and the number of target historical occurrence windows corresponding to the target historical data stream, the method further includes: When the number of occurrence windows is less than the number of occurrence windows in the target history, the previous sorted candidate storage location of the target storage location is determined; The candidate storage location is determined as the target storage location, the historical data stream corresponding to the candidate storage location is determined as the target historical data stream, and the process returns to the step of obtaining the number of times the target historical occurrence window is obtained corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

5. The data stream detection method according to claim 2, characterized in that, When both the occurrence count and the occurrence window count in the first target hash bucket array are not zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the first target hash bucket based on the occurrence count and the occurrence window count, including: When both the occurrence count and the occurrence window count in the first target hash bucket array are not zero, and the window corresponding to the data stream is a new window, the occurrence frequency of the data stream in the current window is obtained, and the ratio between the occurrence count and the occurrence window count is determined. When the occurrence frequency is greater than or equal to the ratio, a first traffic trend value is set for the data stream, and the current storage position of the data stream in the first target hash bucket is determined. The target historical data stream of the target storage position sorted before the current storage position is obtained, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream. When the historical traffic trend value is the second traffic trend value, the target storage location is determined as the storage location corresponding to the data stream; When the target historical traffic trend value is the first traffic trend value, and the number of occurrence windows is greater than the target historical occurrence window number, the target storage location is determined as the storage location corresponding to the data stream; When the occurrence frequency is less than the ratio, the second flow trend value is set for the data stream, and the current storage location is determined as the storage location corresponding to the data stream.

6. The data stream detection method according to claim 5, characterized in that, After obtaining the target historical data stream of the target storage location sorted before the current storage location, and the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, the method further includes: When the target historical traffic trend value is the first traffic trend value, and the number of occurrence windows is less than or equal to the number of historical occurrence windows, the previous sorted candidate storage location of the target storage location is determined; The candidate storage location is determined as the target storage location, the historical data stream corresponding to the candidate storage location is determined as the target historical data stream, and the process returns to the step of obtaining the target historical data stream of the target storage location sorted before the current storage location, as well as the target historical traffic trend value and the number of target historical occurrence windows corresponding to the target historical data stream, until the storage location corresponding to the data stream is determined.

7. The data stream detection method according to claim 3 or 5, characterized in that, After determining whether there is a data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the method further includes: When there is no data stream to be replaced in the first target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, determine whether the occurrence count and the occurrence window count in the second target hash bucket array are both zero; When the occurrence count and the occurrence window count in the second target hash bucket array are both non-zero, and the window corresponding to the data stream is a new window, the storage location corresponding to the data stream is determined in the second target hash bucket based on the occurrence count and the occurrence window count. When the occurrence count and occurrence window count in the second target hash bucket array are both zero, and there is a free storage location in the second target hash bucket, the free storage location in the second target hash bucket is determined as the storage location corresponding to the data stream; When the occurrence count and occurrence window count in the second target hash bucket array are both zero, and there is no free storage location in the second target hash bucket, determine whether there is a data stream to be replaced in the second target hash bucket whose network rate or occurrence frequency does not meet the preset storage conditions; When there is a data stream to be replaced in the second target hash bucket whose network rate or frequency of occurrence does not meet the preset storage conditions, the storage location corresponding to the data stream to be replaced is determined as the storage location corresponding to the data stream. When there is no data stream to be replaced in the second target hash bucket whose network rate or frequency does not meet the preset storage conditions, the storage position corresponding to the data stream is determined from the first storage position at the end of the sorting in the first target hash bucket and the second storage position at the end of the sorting in the second target hash bucket.

8. The data stream detection method according to claim 7, characterized in that, Determining the storage location corresponding to the data stream from the last storage location in the first target hash bucket and the last storage location in the second target hash bucket includes: Determine the first occurrence count of the first data stream at the last first storage location of the sorted first target hash bucket, and determine the second occurrence count of the second data stream at the last second storage location of the sorted second target hash bucket; When the first occurrence count of the first data stream is less than the second occurrence count of the second data stream, the first occurrence count is attenuated to obtain the target first occurrence count; When the first occurrence count of the target is less than the preset occurrence count, the first storage location is determined as the storage location corresponding to the data stream.

9. A data stream detection device, characterized in that, include: The first acquisition module is used to acquire the data stream in the network to be detected and the data stream information of the data stream; The first determining module is used to determine the first hash value of the data stream according to the first hash function and the second hash value of the data stream according to the second hash function. The second determining module is used to determine the first target hash bucket corresponding to the data stream in the first hash bucket array based on the first hash value, and to determine the second target hash bucket corresponding to the data stream in the second hash bucket array based on the second hash value; The second acquisition module is used to acquire the occurrence count and occurrence window count of the data stream, and determine the storage location of the data stream in the first target hash bucket or the second target hash bucket based on the occurrence count and occurrence window count. The data storage module is used to store the data stream information corresponding to the data stream in the storage location to update the hash bucket data stored in the first hash bucket array or the second hash bucket array; The security detection module is used to determine, based on the hash bucket data in the first hash bucket array and the second hash bucket array, target data streams whose network rate or frequency of occurrence does not meet preset conditions, and to perform network security detection on the target data streams.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the data stream detection method according to any one of claims 1 to 8.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data stream detection method according to any one of claims 1 to 8.

Images