Data processing method and related apparatus

By extracting and aggregating target information in network traffic, the problem of high bandwidth cost in network traffic detection is solved, and efficient and low-impact network traffic detection is achieved.

WO2024244690A1PCT designated stage expired Publication Date: 2024-12-05TENCENT TECHNOLOGY (SHENZHEN) CO LTD

Patent Information

Application Number
PCT/CN2024/086289
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-02
Filing Date
2024-04-07
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The prior art requires the transmission of complete network traffic in network traffic detection, resulting in high bandwidth costs and may have an impact on network communication.

Method used

By extracting target information in network traffic, only valid information is sent to traffic detection equipment for detection, reducing bandwidth cost, and aggregating multiple network traffic into one detection traffic to detect, reducing bandwidth usage.

Benefits of technology

It effectively reduces the bandwidth cost of network traffic detection, avoids the impact on network communication, and improves detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024086289_05122024_PF_FP_ABST
    Figure CN2024086289_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present application discloses a data processing method and a related apparatus. A detection-pending network flow can be acquired first, wherein the detection-pending network flow can refer to a network flow for detection; then, information extraction can be carried out on the detection-pending network flow to obtain target information corresponding to the detection-pending network flow, wherein the target information can refer to information required when performing detection on the detection-pending network flow, that is, the target information is effective information for detection; therefore, after the target information is obtained, the target information can be sent to a flow detection device, and the flow detection device can perform detection on the target information to obtain a detection result corresponding to the detection-pending network flow, so that the objective of detection is achieved. The data volume of the target information is smaller than the data volume of the detection-pending network flow, so that the bandwidth cost required for sending the target information to the flow detection device is smaller than the bandwidth cost required for sending the detection-pending network flow, and thus the bandwidth cost can be reduced while implementing detection of the detection-pending network flow.
Need to check novelty before this filing date? Find Prior Art

Description

A data processing method and related device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 2, 2023, with application number 2023106495327 and application name “A data processing method and related device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of data processing technology, and in particular to data processing. Background Art

[0003] Typically, devices can communicate with each other over a network by sending network traffic, which can carry information required for network communication. For example, in the case of network communication between a first device and a second device, the first and second devices can establish a communication connection over a network. After the communication connection is established, the first and second devices can communicate with each other by sending network traffic.

[0004] In network communications, for purposes such as ensuring network security, it is common to inspect transmitted network traffic to promptly detect anomalies, such as abnormal network traffic and the location of traffic bursts, which helps ensure network security. Furthermore, by inspecting network traffic, it is possible to analyze resource usage (such as bandwidth usage) during network communications, promptly identifying excessive resource usage and effectively preventing network instability caused by excessive resource usage, thereby ensuring normal and stable network operation.

[0005] Summary of the Invention

[0006] In order to solve the above technical problems, the present application provides a data processing method and related devices, which can reduce bandwidth costs while realizing the detection of network traffic.

[0007] The embodiments of this application disclose the following technical solutions:

[0008] In one aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0009] Get the network traffic to be tested;

[0010] Extracting information from the network traffic to be measured to obtain target information corresponding to the network traffic to be measured; the data volume of the target information is smaller than the data volume of the network traffic to be measured;

[0011] The target information is sent to a flow detection device; the flow detection device is used to detect the target information and obtain a detection result corresponding to the network flow to be tested.

[0012] In another aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0013] Get multiple network flows to be tested;

[0014] Aggregating the plurality of network flows to be tested, and encapsulating the plurality of network flows to be tested into one detection flow;

[0015] The detection traffic is sent to a traffic detection device; the traffic detection device is used to detect the detection traffic and obtain detection results corresponding to the multiple network traffic to be tested.

[0016] In another aspect, an embodiment of the present application provides a data processing device, comprising an acquiring unit, an extracting unit, and a sending unit:

[0017] The acquisition unit is used to acquire the network traffic to be tested;

[0018] The extraction unit is used to extract information from the network traffic to be measured to obtain target information corresponding to the network traffic to be measured; the data volume of the target information is smaller than the data volume of the network traffic to be measured;

[0019] The sending unit is used to send the target information to the flow detection device; the flow detection device is used to detect the target information and obtain the detection result corresponding to the network flow to be tested.

[0020] In another aspect, an embodiment of the present application provides a data processing device, comprising an acquiring unit, an aggregating unit, and a sending unit:

[0021] The acquisition unit is used to acquire multiple network flows to be tested;

[0022] The aggregation unit is configured to aggregate the plurality of network flows to be tested and encapsulate the plurality of network flows to be tested into one detection flow;

[0023] The sending unit is used to send the detection traffic to the traffic detection device; the traffic detection device is used to detect the detection traffic and obtain detection results corresponding to the multiple network traffic to be tested.

[0024] On the other hand, an embodiment of the present application provides a computer device, the computer device including a processor and a memory:

[0025] The memory is used to store a computer program and transmit the computer program to the processor;

[0026] The processor is configured to execute the method described in any one of the preceding aspects according to instructions in the computer program.

[0027] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a computer device, the computer device executes the method described in any of the above aspects.

[0028] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which, when executed on a computer device, enables the computer device to execute the method described in any one of the aforementioned aspects.

[0029] It can be seen from the above technical solution that when it is necessary to detect network traffic in network communication, the network traffic to be tested can be obtained first, and the network traffic to be tested can refer to the network traffic used for detection. Then, information extraction can be performed on the network traffic to be tested to obtain target information corresponding to the network traffic to be tested. Among them, the target information can refer to the information required when detecting the network traffic to be tested, that is, the target information is effective information for detection. Therefore, after obtaining the target information, the target information can be sent to the traffic detection device, and the traffic detection device can detect the target information and obtain the detection result corresponding to the network traffic to be tested, thereby achieving the purpose of detection. Since the data volume of the target information is smaller than the data volume of the network traffic to be tested, the bandwidth cost required to send the target information to the traffic detection device is less than the bandwidth cost required to send the network traffic to be tested, thereby reducing the bandwidth cost on the basis of realizing the detection of network traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is a schematic diagram of a network traffic detection scenario provided by an embodiment of the present application;

[0031] FIG2 is a schematic diagram of an application scenario of a data processing method provided in an embodiment of the present application;

[0032] FIG3 is a flow chart of a data processing method provided in an embodiment of the present application;

[0033] FIG4 is a schematic diagram of a microburst scenario;

[0034] FIG5 is a schematic diagram of a non-microburst scenario;

[0035] FIG6 is a schematic diagram of the structure of a binary tree provided in an embodiment of the present application;

[0036] FIG7 is a schematic diagram of a system architecture for network traffic detection provided by an embodiment of the present application;

[0037] FIG8 is a schematic diagram of a double mirroring processing logic provided by an embodiment of the present application;

[0038] FIG9 is a schematic diagram of a processing logic of multicast replication combined with a primary mirroring according to an embodiment of the present application;

[0039] FIG10 is a flowchart of another data processing method provided in an embodiment of the present application;

[0040] FIG11 is a structural diagram of a data processing device provided in an embodiment of the present application;

[0041] FIG12 is a structural diagram of another data processing device provided in an embodiment of the present application;

[0042] FIG13 is a structural diagram of a terminal provided in an embodiment of the present application;

[0043] FIG14 is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The embodiments of the present application are described below with reference to the accompanying drawings.

[0045] In practical applications, devices can communicate with each other by sending network traffic. This network traffic can carry information required for network communication, such as messages. During network communication, for purposes such as ensuring network security, the transmitted network traffic is inspected to promptly detect anomalies, such as abnormal network traffic volume or the location of traffic bursts, and to promptly address these anomalies to ensure network security. Abnormal network traffic can refer to attack network traffic that could impact network security. Network traffic inspection can promptly detect abnormal traffic and locate the source of attack network traffic, contributing to network security.

[0046] Taking the first device and the second device as an example, the first device and the second device can establish a communication connection based on the network. After the communication connection is established, the first device and the second device can use the sending network traffic to realize network communication. In the process of network communication between the first device and the second device, the transmitted network traffic can be detected, which is the network traffic detection scenario in the network communication. This application does not make any restrictions on the first device and the second device. In actual applications, the first device and the second device can be different in different network traffic detection scenarios. For example, for the network traffic detection scenario in the telecommunications-grade network, the first device can be a data access terminal (such as a user terminal), and the second device can be a service processing terminal (such as a network service provider).

[0047] For a better understanding, with respect to the network traffic detection scenario, please refer to FIG1 , which is a schematic diagram of a network traffic detection scenario provided in an embodiment of the present application. It should be noted that FIG1 takes the detection of the network traffic sent by the first device to the second device as an example, and does not impose any limitation. In actual applications, the network traffic sent by the second device to the first device can also be detected. A communication connection can be established between the first device and the second device based on the network. In actual applications, the network traffic sent by the first device to the second device can be distributed to the second device via a switch. Similarly, for the network traffic that needs to be detected, it can be sent to a traffic detection device via a switch. The traffic detection device can detect the received network traffic, specifically by analyzing the information carried in the network traffic to complete the detection. Among them, a communication connection can be established between the first device and the switch based on the network, and a communication connection can be established between the second device and the switch based on the network. Based on this, the communication connection between the first device and the second device can be an indirect connection established through the switch. Similarly, a communication connection can be established between the traffic detection device and the switch based on the network.

[0048] In related technologies, when performing network traffic detection, a switch sends the entire amount of network traffic to be detected to the traffic detection device. Specifically, all information contained in the network traffic is sent to the traffic detection device. It is understood that transmitting network traffic to the traffic detection device consumes a certain amount of bandwidth. For network traffic detection, the traffic detection device can utilize partial information contained in the network traffic to complete the detection, determining whether anomalies exist. In other words, only a portion of the information contained in the network traffic is valid for network traffic detection. Therefore, the bandwidth cost of the methods used in related technologies for network traffic detection is relatively high.

[0049] To this end, embodiments of the present application provide a data processing method and related apparatus. For network traffic detection scenarios, before transmitting network traffic to a traffic detection device, valid information for network traffic detection is extracted, thereby sufficing to simply send the valid information to the traffic detection device. Since valid information is the information required for network traffic detection, the traffic detection device can achieve the purpose of detection by detecting the valid information. Furthermore, since the amount of valid information is less than the amount of network traffic, the bandwidth cost required to send the valid information to the traffic detection device is less than the bandwidth cost required to send network traffic, thereby reducing bandwidth costs while enabling network traffic detection.

[0050] The data processing method provided in the embodiments of the present application can be implemented by a computer device, which can be a terminal or a server, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. Terminals include but are not limited to smart phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, audio and video, assisted driving, etc. The embodiments of the present application can be specifically applied to various network traffic detection scenarios, for example, network traffic detection scenarios in telecommunication-grade networks, network traffic detection scenarios in enterprise networks, etc.

[0051] It should be noted that in the specific implementation of this application, user information and other related data may be involved in the process of data processing. When the above embodiments of this application are applied to specific products or technologies, the user's separate consent or separate permission is required, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0052] The methods provided in the embodiments of this application may involve artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system for using digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. For example, the embodiments of this application may utilize artificial intelligence to achieve the automated execution of the data processing method of this application.

[0053] Artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operating / interactive systems, and mechatronics. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, and smart transportation. In the embodiments of the present application, machine learning may be mainly involved. For example, for the acquired network traffic to be tested, machine learning may be used to automatically extract information, and the target information obtained by information extraction may be automatically sent to the traffic detection equipment. Based on this, it is beneficial to improve the efficiency of data processing.

[0054] Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" appear to users to be infinitely scalable, and can be obtained at any time, used on demand, expanded at any time, and paid for on a per-use basis. As a basic capability provider for cloud computing, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an Infrastructure as a Service (IaaS) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. For example, in an embodiment of the present application, the resources in the "cloud" can be network traffic, and network communication is achieved by sending network traffic, and by obtaining network traffic, detection can be performed to detect abnormal situations in a timely manner.

[0055] FIG2 illustrates an application scenario of the data processing method provided by an embodiment of the present application. The scenario illustrated in FIG2 may include a switch 100 and a flow detection device 200, wherein the switch 100 and the flow detection device 200 are connected in communication. It should be noted that the switch 100 and the flow detection device 200 illustrated in FIG2 are merely examples and do not limit the present invention in any way. The following description will be made using the switch 100 as an example of the aforementioned computer device, in conjunction with the scenario illustrated in FIG2 :

[0056] When network traffic needs to be tested, the switch 100 can obtain the network traffic to be tested, which can refer to the network traffic used for testing. In actual applications, taking the aforementioned network traffic detection scenario where a first device sends network traffic to a second device as an example, the network traffic to be tested can be determined based on the network traffic sent by the first device to the second device. Based on the detection of the network traffic to be tested, the network traffic sent by the first device to the second device can be detected.

[0057] The switch 100 can extract information from the acquired network traffic to obtain target information corresponding to the network traffic to be tested. Target information refers to information required for testing the network traffic to be tested, that is, target information is effective information for testing.

[0058] After obtaining the target information, the switch 100 can send the target information to the traffic detection device 200, which can then detect the target information. Since the target information is valid information for detection, the traffic detection device 200 can detect the target information and obtain the detection result corresponding to the network traffic to be tested, thereby achieving the purpose of detection.

[0059] The data volume of the target information is smaller than the data volume of the network traffic to be tested. That is, the data volume of the network traffic can be reduced through information extraction, so that the bandwidth cost required for the switch 100 to send the target information to the traffic detection device 200 is smaller than the bandwidth cost required to send the network traffic to be tested. Moreover, the target information is effective information for detection. Therefore, on the basis of realizing the detection of network traffic, the bandwidth cost can be reduced.

[0060] It should be noted that this application does not impose any restrictions on the configuration of switch 100. In practical applications, to facilitate the implementation of the data processing method provided in this application, switch 100 can be configured as a programmable switch. A programmable switch not only forwards network traffic but also has programmable features. This programmable feature allows for customizing the network traffic processing process, facilitating the implementation of the data processing method provided in this application.

[0061] FIG3 is a flow chart of a data processing method provided in an embodiment of the present application, which is illustrated by taking a switch as an example of the aforementioned computer device. The method includes S301-S303:

[0062] S301: Obtain the network traffic to be tested.

[0063] When network traffic needs to be tested, the switch can obtain the network traffic to be tested, which can refer to the network traffic used for testing. In actual applications, taking the aforementioned network traffic detection scenario where a first device sends network traffic to a second device as an example, the network traffic to be tested can be determined based on the network traffic sent by the first device to the second device. Based on the detection of the network traffic to be tested, the network traffic sent by the first device to the second device can be detected.

[0064] It should be noted that this application does not impose any restrictions on the network traffic to be measured. For ease of understanding, this application will explain the network traffic to be measured from the following three aspects:

[0065] On the one hand, there is no limitation on the relationship between the network traffic to be tested and the network traffic sent between devices. The network traffic to be tested may refer to the network traffic used for testing, the purpose of which is to ensure network security, etc. The network traffic sent between devices may refer to the network traffic used for network communication, the purpose of which is to conduct network communication. It is understandable that changes in the information carried by the network traffic will affect the network communication. Therefore, in order to avoid the impact of network traffic detection on network communication, network traffic detection is usually performed on the basis of the network traffic obtained by copying the network traffic used for network communication. In other words, the network traffic to be tested may be obtained by copying the network traffic used for network communication, and the network traffic used for network communication may refer to the network traffic sent when the devices communicate with each other. For example, the network traffic to be tested may refer to the network traffic copied based on the network traffic sent by the first device to the second device. Based on this, the detection will not cause changes to the network traffic sent by the first device to the second device, thereby avoiding the impact on the network communication.

[0066] On the other hand, there is no limitation on the type of network traffic to be tested. For example, in actual network communications, taking the network traffic sent between devices as messages as an example, the type of network traffic to be tested can also be in the form of messages.

[0067] On the other hand, there is no limit on the number of network flows to be tested. In actual network flow detection scenarios, the volume of network flows sent between devices is very large, and accordingly, the number of network flows to be tested can also be multiple.

[0068] S302: Extract information from the network traffic to be measured to obtain target information corresponding to the network traffic to be measured.

[0069] After obtaining the network traffic to be tested, the switch can extract information from the network traffic to be tested to obtain target information corresponding to the network traffic to be tested. The target information may refer to the information required for detecting the network traffic to be tested, that is, the target information is effective information for detection. Under different detection purposes, the information required for detection may be different.

[0070] Because the target information is used for detection, the switch does not need to extract information unrelated to the detection when extracting the target information from the network traffic under test, effectively reducing the amount of target information data. The target information data volume is smaller than the network traffic data volume under test. Based on this, information extraction can reduce the data volume.

[0071] It should be noted that this application does not impose any restrictions on how to extract information. In actual applications, the types of network traffic to be measured are different, and the methods for extracting information may be different. For ease of understanding, this embodiment of the application takes the form of packets as an example of the type of network traffic to be measured, and provides the following methods as examples:

[0072] When the network traffic to be tested is in the form of a message, the network traffic to be tested can be referred to as a message to be tested. Typically, a message to be tested may include multiple fields, and different fields may carry different information. Therefore, in one possible implementation, information extraction may be performed by extracting relevant fields in the message to be tested that carry valid information (relative to detection), thereby achieving the purpose of extracting valid information. For example, multiple fields in the message to be tested may carry message quintuple information, message length information, message version information, and the like. The message five-tuple information may include the source Internet Protocol Address (IP address), destination IP address, source port number, destination port number, and protocol type of the message to be tested. Specifically, taking the example of a first device sending network traffic to a second device, the source IP address may refer to the IP address of the first device, the destination IP address may refer to the IP address of the second device, the source port number may refer to the device port number of the first device, the destination port number may refer to the device port number of the second device, and the protocol type may refer to the protocol used by the message to be tested. For example, the protocol used may be the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), etc. The message length information may identify the length of the message to be tested, and the message version information may identify the version of the message to be tested. For detection, the message quintuple information and message length information can be used to detect whether the IP address is legal and valid, and whether the length of the message to be tested is abnormal, so as to evaluate whether the message to be tested is abnormal (such as the IP address is illegal and invalid, indicating that it may be the aforementioned attack network traffic). Therefore, in actual applications, the message quintuple information and message length information can be determined as valid information related to this detection. When extracting information, the relevant fields carrying the message quintuple information and message length information can be extracted.

[0073] It can be understood that, corresponding to the case where the network traffic to be tested is the message to be tested, in order to facilitate the subsequent sending of the target information, after extracting the message quintuple information and the message length information, this information can be reorganized so that the type of valid information still conforms to the form of the message, which is convenient for subsequent sending, etc.

[0074] It should also be noted that this application does not impose any restrictions on the form of data volume. Data volume can be used to represent data size, such as the data volume of target information is used to represent the data size of target information, and the data volume of the network traffic to be measured is used to represent the data size of the network traffic to be measured. In practical applications, data volume can be in the form of data length, and its unit can be bytes (Byte, B), kilobytes (Kilobyte, KB), etc., which can intuitively represent the data size. For example, the data volume of the network traffic to be measured may be 512B, and the valid information is 40B of it. Accordingly, the data volume of the target information may be 40B.

[0075] S303: Send target information to the traffic detection device.

[0076] After obtaining the target information, the switch can send the target information to the flow detection device, which can be used to detect the target information. Since the target information is valid information for detection, the flow detection device can detect the target information to obtain the detection result corresponding to the network traffic to be tested, thereby achieving the purpose of detection. Since the data volume of the target information is smaller than the data volume of the network traffic to be tested, the smaller the data volume, the lower the bandwidth cost required when sending between devices. Therefore, the data volume is reduced through information extraction, so that the bandwidth cost required for the switch to send the target information to the flow detection device is less than the bandwidth cost required to send the network traffic to be tested. At the same time, the information extraction extracts valid information for detection, so it can reduce the bandwidth cost while ensuring the accuracy of detection. Taking the aforementioned network traffic to be tested as the test message as an example, based on information extraction, the relevant fields carrying valid information are retained and other fields are filtered, thereby reducing the field length of the message and reducing the bandwidth cost.

[0077] It should be noted that this application does not impose any restrictions on how to send target information to the traffic detection device. In most network traffic detection scenarios, the volume of network traffic sent between devices is very large, and accordingly, the number of network traffic to be tested can also be multiple. For a better understanding, the embodiment of this application will take the example that the network traffic to be tested can include n network traffic to be tested, where n is an integer greater than 1, and provide the following method as an example:

[0078] In an actual network traffic detection scenario, for the target information corresponding to n network traffic to be tested, in a possible implementation method, the switch can directly send the target information corresponding to the n network traffic to be tested to the traffic detection device. Corresponding to this method, the traffic detection device can receive n target information corresponding to the n network traffic to be tested, and accordingly, the traffic detection device can detect the n target information respectively, and obtain the detection results corresponding to the n network traffic to be tested. Based on this, the traffic detection device can detect the n network traffic to be tested by detecting n times, and obtain the detection results corresponding to each network traffic to be tested. In this method, the detection results obtained can usually also be n, which is more intuitive. In this method of directly sending the target information corresponding to the n network traffic to be tested to the traffic detection device, the switch does not perform other processing on the n target information, but directly sends it to the traffic detection device. Therefore, it can also be called a sending method based on non-aggregation mode (Normal Mirror).

[0079] It is understandable that the flow detection device consumes its performance to detect target information. In practical applications, more detections place greater demands on the performance of the flow detection device. Therefore, in another possible implementation, the switch can aggregate the target information corresponding to each of the n network flows to be tested before sending it. Aggregation can be a processing method that aggregates multiple target information into a single aggregated message. In this way, the switch can send the aggregated information to the flow detection device, thereby achieving the purpose of sending multiple target information to the flow detection device. Accordingly, the flow detection device can detect the aggregated information to achieve the detection of multiple target information. Based on this, the detection of multiple network flows to be tested can be completed in a single test, reducing the number of detections and thus reducing the performance loss of the flow detection device. In a specific implementation, the switch can aggregate the target information corresponding to each of the n network flows to be tested, and then send the aggregated information to the flow detection device. In response, the flow detection device can detect the received aggregated information to obtain a detection result. Based on this, aggregation can effectively reduce the number of detections performed by the flow detection device, thereby reducing the performance loss of the flow detection device.

[0080] This method of first aggregating and then sending the aggregated information obtained by aggregation can also be called a sending method based on an aggregation mode (Collescing Mirror). When adopting a sending method based on an aggregation mode, the data processing method provided in the embodiment of the present application can be considered as a method of compressing the data volume based on information extraction, and aggregating multiple target information into one aggregated information based on aggregation, thereby reducing the number of detection times, especially for network traffic detection scenarios with very large volume of network traffic, it can greatly reduce bandwidth costs and reduce the loss of equipment performance of traffic detection equipment, and is a high-performance network traffic detection technology. Taking the aforementioned network traffic to be tested as a message as an example, the target information can also be in the form of a message, and the aggregated information can also be message information. After adopting this application, it can be considered that multiple messages are compressed and aggregated into the same message for detection.

[0081] In the sending method based on the aggregation mode, it should be noted that this application does not impose any restrictions on how to aggregate the target information corresponding to the n network flows to be tested. In actual applications, the purpose of aggregation is to determine multiple target information as one aggregate information. Therefore, in a possible implementation method, the specific method of aggregation can be splicing, that is, multiple target information can be spliced ​​together to obtain one aggregate information. And in actual applications, it is also possible to flexibly set how many specific target information to aggregate into one aggregate information. Accordingly, when aggregating the target information corresponding to the n network flows to be tested, it can be possible to aggregate to obtain one aggregate information every time a sufficient number of target information is generated. In other words, the target information corresponding to the n network flows to be tested can be aggregated to obtain multiple aggregate information.

[0082] Corresponding to the transmission method based on the aggregation mode, multiple target information is aggregated into a single aggregate information. To facilitate aggregation, the target information corresponding to each of the n network flows to be tested can be cached and stored before subsequent aggregation. In practical applications, a cache address can be configured in the switch. The cache address can refer to a storage space with cache storage capabilities. In the switch, the cache address can be a session control object (session). The cache address can be used to cache and store the target information corresponding to each of the n network flows to be tested. In a specific implementation, before aggregating the target information corresponding to each of the n network flows to be tested, the switch can write the target information corresponding to each of the n network flows to be tested into the cache address. Based on this, the target information corresponding to each of the n network flows to be tested can be cached and stored. Accordingly, a specific implementation method of aggregating the target information corresponding to each of the n network flows to be tested can include: the switch can aggregate the target information corresponding to each of the n network flows to be tested written to the cache address; and a specific implementation method of sending the aggregated information to the traffic detection device can include: the switch can read the aggregated information from the cache address and then send the read aggregated information to the traffic detection device. Based on this, the cache address can be used to cache and store the target information corresponding to multiple network flows to be tested, and then aggregate and send them.

[0083] It should be noted that this application does not impose any restrictions on the method of writing the target information corresponding to the n network flows to be tested into the cache address. It is understandable that the method of writing the cache address is different, and the method of aggregating the target information corresponding to the n network flows to be tested written into the cache address may also be different. In actual applications, the aforementioned cache address may include multiple cache addresses, so that the aforementioned n target information can be cached and stored using multiple cache addresses. For ease of understanding, the following will be explained using the example of a cache address including multiple cache addresses:

[0084] In practical applications, one of the functions of a switch is to forward received information. To facilitate forwarding, a data packetization method is provided in related technologies. Specifically, received information is cached and stored using a cache address, and then the information is read from the cache address and forwarded. Typically, this method includes an information queue, a cache queue, and an egress queue. The information queue can contain received information, the cache queue can include multiple cache addresses, and the egress queue can contain information read from the cache address. The read information can be forwarded via the switch's egress port.

[0085] It is understood that any of the multiple cache addresses can include both write and read processing. In practical applications, the cache write rate can typically be used to represent the rate at which information in the information queue is written to the cache address, while the cache read rate can be used to represent the rate at which information in the cache queue is read into the egress queue. The cache read rate can depend on the processing rate of the egress queue. If the cache write rate is greater than the cache read rate, a continuous accumulation of cache writes may occur. For example, if the cache address is full, the cache write rate is greater than the cache read rate, resulting in a continuous accumulation of cache writes. This scenario can also be referred to as a microburst scenario, as shown in Figure 4. Figure 4 shows a schematic diagram of a microburst scenario, which specifically includes the aforementioned information queue, cache queue, and egress queue. Taking the first cache address among the multiple cache addresses as an example, the cache write rate can be considered to be 2 writes per unit time, while the cache read rate can be considered to be 1 read per unit time. Since the first cache address does not have sufficient cache space, packet loss will occur. In other words, after a microburst occurs, packet loss will occur due to insufficient cache space. If the cache write rate equals the cache read rate, the aforementioned accumulation of cache writes will not occur, meaning no microbursts will occur. For this scenario, see Figure 5, which shows a schematic diagram of a non-microburst scenario. The cache write rate equals the cache read rate, so microbursts will not occur.

[0086] If the data packet sending method provided by the relevant technology is adopted, the n target information in this application is written into the cache address. First, the micro-burst situation shown in Figure 4 may occur, and in actual applications, since the cache space of the switch chip is generally relatively small, correspondingly, the cache space allocated to a single cache address in the switch is usually relatively small, a single cache address corresponds to a cache pipeline, and multiple cache addresses can correspond to multiple cache pipelines. For example, the chip of a programmable switch can be an integrated circuit (Application Specific Integrated Circuit, ASIC) chip, and the cache space allocated to a single cache address can be 122KB. A small cache space is more likely to cause micro-bursts, which often leads to packet loss of target information due to insufficient cache, which will reduce the detection accuracy.

[0087] Secondly, even in the absence of microbursts, as shown in Figure 5, since the main purpose of writing n target information into the cache address in this application is to facilitate the aggregation of multiple target information into one aggregated information, for any cache address, a sufficient amount of target information must be written to meet the purpose of aggregation to obtain an aggregated information, and then the aggregated information can be read and sent. If a sufficient amount of target information is not written, the purpose of aggregation to obtain an aggregated information cannot be achieved. Using the method in the related art, multiple cache addresses meet the requirement of writing a sufficient amount of target information almost simultaneously. Before this is met, any cache address is cache occupied, but no aggregated information can be obtained, resulting in low aggregation performance.

[0088] It can be seen that regardless of whether a microburst occurs, the data packet sending method in the related technology cannot meet the requirements of high-performance aggregation, and the detection accuracy will also be affected by the microburst.

[0089] To this end, an embodiment of the present application provides another data packet sending method. For n target information, the target information can first be continuously written to one of the multiple cache addresses until this cache address can no longer be written, and then the target information can be written to the next cache address until the n target information is written to the cache address. Based on this, a cache address can be given priority to write a sufficient amount of target information, and then aggregation can be performed to obtain an aggregated information, which is beneficial to improving aggregation performance. At the same time, since writing to a cache address is given priority until this cache address can no longer be written, and then writing to the next cache address, cache occupancy can be reduced. In actual applications, the reason for not being able to continue writing to the current cache address is usually that the remaining cache capacity of the current cache address is insufficient to store the target information. Based on this, it can be ensured that writing will not be done to a cache address with insufficient cache, thereby avoiding micro-bursts and packet loss problems, which is beneficial to ensuring detection accuracy. During specific implementation, for each target information among the n target information, it can be determined in turn whether to continue writing to the current cache address or to the next cache address.

[0090] In actual applications, for any cache address among multiple cache addresses, its cache capacity can represent the amount of data that can be stored. In order to facilitate the judgment of whether the remaining cache capacity of the current cache address is sufficient to store the target information, taking the data amount as the aforementioned data length as an example, the remaining cache length can be used to represent the remaining cache capacity of the cache address. Accordingly, in the process of writing to the cache address, the specific implementation method for judging whether to continue writing to the current cache address or to the next cache address can be determined based on the comparison of the data length corresponding to the data amount of the target information and the remaining cache length of the current cache address. Specifically, if the remaining cache length is greater than or equal to the data length corresponding to the data amount of the target information, it indicates that the remaining cache capacity of the cache address can store the target information, and the target information can continue to be written to the current cache address. Correspondingly, if the remaining cache length is less than the data length corresponding to the data amount of the target information, it indicates that the remaining cache capacity of the cache address is insufficient to store the target information, and the target information can be written to the next cache address.

[0091] Correspondingly, the aforementioned specific implementation method of aggregating the target information corresponding to the n network flows to be tested written to the cache address can be that when the current cache address is insufficient to continue storing the target information, it is considered that the storage of the current cache address has reached the upper limit (e.g., it is full). At this time, the target information stored in the current cache address can be aggregated, and the aggregated information obtained can be sent to the traffic detection device for detection until the target information corresponding to the n network flows to be tested is aggregated. Based on this, writing and detection can be performed simultaneously, which is more efficient.

[0092] For better understanding, the embodiment of the present application will take the example that the multiple cache addresses may include two cache addresses, specifically a first cache address and a second cache address, and the data amount is the aforementioned data length as an example for explanation:

[0093] In practical applications, target information is valid information for detection. Typically, the data size of the target information can be fixed, and the data size of the target information can be a preset data length, such as the aforementioned 40 bytes. When writing to a cache address, the first cache address can refer to the aforementioned current cache address, and the second cache address can refer to the aforementioned next cache address. In a specific implementation, for the i-th target information corresponding to the i-th network flow under test among n network flows under test, if the remaining cache length of the first cache address is determined to be greater than or equal to the preset data length, it indicates that the first cache address is sufficient to store the i-th target information. In this case, the i-th target information can be written to the first cache address. Where i is an integer greater than or equal to 2 and less than or equal to n. Since the first cache address can be the current cache address, the first cache address can store the i-1th target information corresponding to the i-1th network flow under test among the n network flows under test. If the remaining cache length of the first cache address is determined to be less than the preset data length, it indicates that the first cache address is insufficient to store the i-th target information. In this case, the i-th target information can be written to the second cache address. This process continues until all n target information corresponding to the n network flows to be tested are written to the cache addresses. By first writing to the first cache address and then writing to the second cache address when the first cache address is no longer capable of writing, the system can more quickly store enough target information to aggregate into a single aggregated information, reducing cache usage and improving aggregation performance. Furthermore, target information will not be written to cache addresses with insufficient buffer space, thus avoiding microbursts, reducing packet loss, and ensuring detection accuracy.

[0094] It should be noted that this application does not impose any restrictions on how to determine the remaining cache length of the first cache address. In actual applications, for any cache address, there is an upper limit to its cache capacity, that is, the maximum amount of data that can be stored, which can usually be represented by the maximum cache length of the cache address. In the process of storing information, the cache address can also use the historical cache length to represent the sum of the data lengths of the information already stored in the cache address. In this way, the difference between the maximum cache length and the historical cache length can be used to determine the remaining cache length. Corresponding to the first cache address, its remaining cache length can be the difference between the maximum cache length of the first cache address and the historical cache length.

[0095] In order to better understand the specific implementation method of aggregating the target information corresponding to the n network flows to be tested written into the cache address, the first cache address and the second cache address are still used as examples for explanation:

[0096] In practical applications, information can be read from the cache address only after the cache address is full. That is, reading can be temporarily suspended until the cache address is full, and only after the cache address is full. Therefore, when using a first cache address and a second cache address to store n pieces of target information, to improve the storage efficiency of the first cache address, the maximum cache length of the first cache address can include m preset data lengths, where m is an integer greater than 1. That is, the first cache address can exactly store m pieces of target information. The same applies to the second cache address. Typically, the first and second cache addresses can be configured with the same maximum cache length. Accordingly, when it is determined that the first cache address cannot store any more data, it can be considered full. At this point, the m pieces of target information stored in the first cache address can be aggregated, and the aggregated information can be read from the first cache address for transmission to the traffic detection device. Simultaneously, target information can be written to the second cache address. When the second cache address is full, the m pieces of target information stored in the second cache address can similarly be aggregated, and the aggregated information can be read from the second cache address for transmission to the traffic detection device. The process continues in this way until the aggregation of target information corresponding to the n measured flows is completed.

[0097] In a specific implementation, the switch can determine the cache address storing m target information in the first cache address and the second cache address as the target cache address during the process of writing n network flows to be tested into the cache address. In other words, the full cache address is determined as the target cache address. The target cache address is full, so it can be read. At this time, aggregation can be performed based on the m target information stored in the target cache address, and the aggregated information obtained by aggregation can be read from the target cache address, and the aggregated information obtained by aggregation can be sent to the traffic detection device for easy detection. It can be understood that after reading the aggregated information, the target cache address can change from full to not full, and the target information can continue to be written. And so on, until the aggregation of the target information corresponding to the n flows to be tested is completed.

[0098] It should also be noted that this application does not impose any restrictions on the method for determining the size relationship between the remaining cache length of the first cache address and the preset data length. For ease of understanding, this embodiment of the application takes the example that the maximum cache length of the first cache address can include m preset data lengths, where m is an integer greater than 1, and provides the following method as an example:

[0099] Since the first cache address can store exactly m pieces of target information, in one possible implementation, a judgment can be made based on the number of stored pieces. In a specific implementation, if i ≤ m, it indicates that the first cache address has not yet stored enough pieces of target information, so it can be determined that the remaining cache length of the first cache address is greater than or equal to the preset data length. If i > m, it indicates that the first cache address has already stored enough pieces of target information, so it can be determined that the remaining cache length of the first cache address is less than the preset data length. Based on this, the judgment based on the number of stored pieces is relatively simple.

[0100] In practical applications, the problem of determining whether to continue writing to the first cache address or the second cache address can be abstracted into a binary tree structure. In the binary tree, each parent node can correspond to two child nodes, each corresponding to a cache address. After receiving the target information, the parent node can write the target information to the cache address corresponding to the child node by controlling which child node to use to send the target information. Therefore, in another possible implementation, a binary tree structure can be used to implement the determination and writing. First, in the binary tree, the first cache address can correspond to the first child node, and the second cache address can correspond to the second child node. The first and second child nodes correspond to the same parent node, which can be the node included in the binary tree that received the i-th target information. Second, in the binary tree, the maximum cache length of the first child node can be used to represent the upper limit of the length of the target information that the parent node can send through the first child node. The length of the historical information sent by the parent node through the first child node can be used to represent the length of the target information that the parent node has sent through the first child node. The same applies to the second child node. It should be noted that the maximum cache length of the first child node can be determined based on the number of levels of the parent node in the binary tree and the maximum cache length of the first cache address. Therefore, the upper limit of the length of the target information that can be sent by the first child node matches the maximum cache length of the first cache address. The length of the historical information sent by the first child node can reflect the length already stored in the first cache address. Therefore, the size relationship between the maximum cache length of the first child node and the length of the historical information of the first child node can be used to make a judgment. In addition, the maximum cache length of the first cache address can include m preset data lengths, where m is an integer greater than 1. Therefore, the size relationship between the maximum cache length of the first child node and the length of the historical information of the first child node can be used to determine whether the first cache address corresponding to the first child node stores all m target information. In actual applications, since the maximum cache length of the first child node can include an integer number of preset data lengths, the specific number can be determined based on the number of levels of the parent node in the binary tree and m, and can be an integer multiple of m.

[0101] Specifically, if the length of the historical information sent by the parent node through the first child node is less than the maximum cache length of the first child node, it indicates that the target information can continue to be sent through the first child node, the first cache address is not full, and the target information can continue to be stored. At this time, it can be determined that the remaining cache length of the first cache address is greater than or equal to the preset data length. Then, the parent node can be used to send the i-th target information to the first child node, and the i-th target information is written to the first cache address through the first child node. Correspondingly, if the length of the historical information sent by the parent node through the first child node is equal to the maximum cache length of the first child node, it indicates that the upper limit has been reached and the first cache address is full. At this time, it can be determined that the remaining cache length of the first cache address is less than the preset data length. Then, the parent node can be used to send the i-th target information to the second child node, and the i-th target information is written to the second cache address through the second child node. Based on this, the purpose of writing the target information into the cache address can be achieved by using the binary tree structure. In a binary tree, to facilitate judgment, each parent node can record the length of historical information sent by two child nodes respectively, so that it can quickly determine which child node the current target information goes through. If it goes through different child nodes, the corresponding cache address written will be different.

[0102] Based on the binary tree structure, a parent node will not use a child node that has reached its upper limit to send target information, thereby ensuring that target information is not written to a full cache address and avoiding microbursts. Furthermore, until a child node reaches its upper limit, it will continue to use that child node to send target information, prioritizing writing target information to the same cache address, reducing cache usage and improving aggregation performance.

[0103] It should be noted that this application does not impose any restrictions on how to implement the binary tree structure. In practical applications, registers in a switch can be used to control the sending of information to a specified cache address. Therefore, the binary tree structure can be implemented using registers in the switch. Because a binary tree structure has multiple layers, such as parent nodes and child nodes, in specific implementations, the binary tree structure can be implemented using multiple levels of register control in the switch.

[0104] In a binary tree structure, the depth of a binary tree can refer to the number of layers of the deepest node among all the nodes included in the binary tree, or it can refer to the number of layers of the root node. In actual applications, the depth of the binary tree will vary depending on the number of cache addresses. Specifically, the relationship between the depth of the binary tree and the number of cache addresses can be expressed by the following formula: A = log2(B)

[0105] Among them, A can represent the depth of the binary tree, and B can represent the number of cache addresses.

[0106] For example, still taking the aforementioned example where the number of cache addresses is two, specifically including a first cache address and a second cache address, the depth of the binary tree is 1, that is, the number of layers of the root node in the binary tree is 1.

[0107] For ease of understanding, the embodiment of the present application takes the cache address as the aforementioned session as an example, and takes the number of cache addresses as more than two, specifically including 8, as an example, and provides a binary tree structure, which can be seen in Figure 6. Figure 6 shows a schematic diagram of the structure of a binary tree. Specifically, the 8 cache addresses can be session 1, session 2, session 3, session 4, session 5, session 6, session 7 and session 8, corresponding to child nodes 4-1, 4-2, 4-3, 4-4, 4-5, 4-6, 4-7 and 4-8, respectively. Among them, the parent node corresponding to 4-1 and 4-2 is 3-1, the parent node corresponding to 4-3 and 4-4 is 3-2, the parent node corresponding to 4-5 and 4-6 is 3-3, and the parent node corresponding to 4-7 and 4-8 is 3-4. The parent node corresponding to 3-1 and 3-2 is 2-1, and the parent node corresponding to 3-3 and 3-4 is 2-2. The parent node corresponding to 2-1 and 2-2 is 1. Node 1 can be the deepest node in the binary tree. Corresponding to B=8, the depth of the binary tree is A=3. Accordingly, node 1 is at level 3 in the binary tree. Similarly, nodes 2-1 and 2-2 are at level 2 in the binary tree, nodes 3-1, 3-2, 3-3, and 3-4 are at level 1 in the binary tree, and nodes 4-1, 4-2, 4-3, 4-4, 4-5, 4-6, 4-7, and 4-8 are at level 0 in the binary tree, corresponding to the nodes directly corresponding to the session.

[0108] For better understanding, the maximum buffer length for each session can be denoted as max_buffer, which can be equal to m preset data lengths. Based on the binary tree structure shown in Figure 6, it can be seen that the maximum buffer length of node 4-1 can be equal to max_buffer, and the maximum buffer lengths corresponding to nodes 4-2, ..., 4-7, and 4-8 can also be equal to max_buffer. The maximum buffer length of node 3-1 can be the sum of the maximum buffer lengths of its corresponding child nodes 4-1 and 4-2, specifically equal to 2*max_buffer. The same applies to nodes 3-2, 3-3, and 3-4. The maximum buffer length of node 2-1 can be the sum of the maximum buffer lengths of its corresponding child nodes 3-1 and 3-2, specifically equal to 4*max_buffer. The same applies to node 2-2. The maximum buffer length of node 1 can be the sum of the maximum buffer lengths of its corresponding child nodes 2-1 and 2-2, specifically equal to 8*max_buffer. Since node 1 is the root node, any target information must first reach node 1 and be transmitted through node 1.

[0109] It can be seen that based on the binary tree structure shown in Figure 6, the maximum cache length for any child node can be determined based on the number of levels of the parent node corresponding to the child node in the binary tree and the maximum cache length of the session corresponding to the child node. Specifically, it can be expressed by the following formula: max_node = 2 a-1 *max_buffer

[0110] Among them, max_node can represent the maximum cache length of any child node, a can represent the number of layers of the parent node corresponding to the child node in the binary tree, and max_buffer can represent the maximum cache length of the session corresponding to the child node.

[0111] By adopting the data packet sending method provided by the present application, since the target information will not be written to a full session, it can be ensured that no micro-burst scenario will occur, and thus it will not be affected by micro-bursts. At the same time, it can reduce cache occupancy. Specifically, if the data packet sending method in the related art is adopted, multiple sessions are almost filled at the same time, and then the aggregate information can be read. It will not be read before the aggregate information is obtained, resulting in cache occupancy, and the occupied cache is greater than or equal to the number of sessions multiplied by the data length of one aggregate information. After adopting the data packet sending method provided by the present application, it is preferred to fill one session first and then write to the next session. In this way, the fullness can be met faster, and then the aggregate information can be aggregated and read, etc., reducing cache occupancy.

[0112] Through the above embodiments, the data processing method provided by the embodiment of the present application is described in detail. It should be noted that in the above embodiments, the present application does not impose any restrictions on how to obtain the network traffic to be measured, that is, the specific implementation of the aforementioned S301. For ease of understanding, the embodiment of the present application takes the example of the network traffic to be measured being copied, and provides the following method as an example:

[0113] In specific applications, the switch can determine the incoming transmission network traffic. Transmission network traffic can refer to network traffic that the switch needs to forward and process. The determination can specifically refer to determining whether the transmission network traffic is copied. If it is copied, the transmission network traffic can be considered as network traffic used for detection, and it can be determined as network traffic to be tested for subsequent processing and forwarded to the traffic detection device to achieve detection. If it is not copied, it indicates that the transmission network traffic is network traffic used for network communication, and there is no need to determine it as network traffic to be tested, thereby ensuring network communication. Therefore, in one possible implementation, when the aforementioned S301 is specifically implemented, the switch can first obtain the transmission network traffic and determine the transmission network traffic. If it is determined that it is copied, the transmission network traffic can be determined as network traffic to be tested. Correspondingly, if it is determined that it is not copied, there is no need to determine it as network traffic to be tested, thereby ensuring network communication.

[0114] It should be noted that this application does not impose any limitations on how a switch determines transmitted network traffic. In practical applications, network traffic may carry a type identifier, which can be used to indicate whether the network traffic is duplicated. Therefore, in one possible implementation, the switch may use the type identifier to determine transmitted network traffic. In a specific implementation, the type identifier may include a duplication identifier and a service identifier. The duplication identifier can be used to indicate that the network traffic is duplicated and is used for testing, while the service identifier can be used to indicate that the network traffic is not duplicated and is used for network communication. In accordance with this approach, in the specific implementation of S301, the switch may first obtain transmitted network traffic. Transmitted network traffic may refer to network traffic that the switch needs to forward. Next, the switch may parse the transmitted network traffic to obtain a type identifier for the transmitted network traffic. The type identifier for the transmitted network traffic can be used to indicate whether the transmitted network traffic is duplicated. If the type identifier is a duplication identifier, indicating that the transmitted network traffic is duplicated, the switch may determine the transmitted network traffic as the network traffic to be tested. Correspondingly, if the type identifier is a business identifier, it indicates that the transmitted network traffic is not copied, but is network traffic used for network communication, so it does not need to be determined as the network traffic to be tested. Based on this, the type identifier can be used to determine the transmitted network traffic.

[0115] It should be noted that this application does not impose any restrictions on how to obtain the transmission network traffic. First of all, the transmission network traffic may refer to the network traffic that the switch needs to forward and process, and in actual applications, the network traffic used for network communication is forwarded via the switch, and the network traffic used for detection is also forwarded via the switch. It can be seen that the transmission network traffic can include both network traffic used for detection and network traffic used for network communication. Among them, the network traffic used for detection is obtained by copying the network traffic used for network communication. The method of obtaining the transmission network traffic may be different depending on the method of copying. For ease of understanding, the embodiments of this application provide the following two methods as examples:

[0116] In actual applications, the switch can first obtain the initial network traffic, which can refer to the network traffic used for network communication. Correspondingly, the type identifier of the initial network traffic is the service identifier. In one possible implementation, after obtaining the initial network traffic, the switch can directly multicast replicate the initial network traffic to obtain replicated network traffic, and the type identifier of the replicated network traffic is the replication identifier. Then, the switch can respectively determine the initial network traffic and the replicated network traffic as transmission network traffic. Among them, multicast replication is a flexible replication method that can flexibly control the number of replications. The use of multicast replication is conducive to improving the flexibility of implementing the method provided in the present application. Based on this, first replicating the initial network traffic can be considered as a replication-pre-position method, so that the transmission network traffic includes both the initial network traffic and the replicated network traffic. The use of the replication-pre-position method allows detection to be performed after receiving the transmission network traffic, which is conducive to improving detection efficiency.

[0117] Different from the aforementioned pre-copy approach, another possible implementation method can employ a post-copy approach. Specifically, the switch does not replicate the acquired initial network traffic, but instead replicates it after a subsequent determination for subsequent detection. In a specific implementation, the switch can first acquire the initial network traffic and identify it as transmission network traffic, with the type identifier of the initial network traffic being the service identifier. Based on this, the transmission network traffic received by the switch can initially be the initial network traffic. Then, after determining the type identifier of the transmission network traffic as the service identifier, to enable detection without impacting network communication, the switch can replicate the transmission network traffic and identify the replicated network traffic as transmission network traffic, with the type identifier of the replicated network traffic being the replication identifier. Accordingly, the transmission network traffic subsequently received by the switch can be the replicated network traffic and can be used for subsequent detection. Based on this, the post-copy approach allows the transmission network traffic to initially be the initial network traffic and subsequently be the replicated network traffic, thereby ensuring both network communication and detection. The post-copy approach allows replication to be performed after the determination, providing greater flexibility.

[0118] It should be noted that this application does not impose any restrictions on how to obtain the initial network traffic. In actual applications, the initial network traffic may refer to the network traffic used for network communication, and specifically may refer to the network traffic sent between devices. Taking the aforementioned first device sending network traffic to the second device as an example, the switch can obtain the initial network traffic from the network traffic sent during the process of the first device sending network traffic to the second device. In actual applications, the first device can send network traffic to the second device with the device interface as the basic unit. Correspondingly, the switch can obtain the initial network traffic from the device interface of the first device. Based on this, the detection can also be considered as a network traffic detection with the device interface as the basic unit, which is conducive to locating abnormal device interfaces, etc. In addition, the network traffic sent from the first device to the second device can be classified according to the detection requirements, and the classification can be used to distinguish whether the sent network traffic needs to be detected. The network traffic that needs to be detected is screened out and determined as the initial network traffic for subsequent detection. The network traffic that does not need to be detected can be directly forwarded to the second device.

[0119] In network traffic detection, network traffic detection can be evaluated mainly from two dimensions: detection accuracy and required bandwidth cost. It is understandable that the higher the proportion of sent network traffic detected, the more conducive it is to ensuring detection accuracy, and the lower the proportion of sent network traffic detected, the more conducive it is to reducing the required bandwidth cost. Different proportions of sent network traffic detected refer to different proportions of initial network traffic obtained from the sent network traffic. It is understandable that different network traffic detection scenarios may have different corresponding detection requirements. For example, some network traffic detection scenarios pay more attention to detection accuracy, while some network traffic detection scenarios pay more attention to bandwidth cost. Therefore, by adjusting the proportion of obtaining initial network traffic, the detection can be flexibly applied to various network traffic detection scenarios.

[0120] In actual applications, the proportion of obtaining initial network traffic can be adjusted by setting a sampling ratio. During specific implementation, in the process of the first device sending network traffic to the second device, the switch can sample the sent network traffic according to the sampling ratio to obtain the initial network traffic. Among them, the sampling ratio can refer to the proportion of obtaining the initial network traffic from the sent network traffic, and specifically can refer to the ratio of the amount of sent network traffic to the amount of initial network traffic. In actual applications, the sFlow network traffic detection technology supported by the Request For Comments (RFC) of the Internet standard can be used to sample the sent network traffic according to the sampling ratio. Based on this, the detection ratio can be adjusted by setting the sampling ratio, so that the detection is more in line with the actual detection needs.

[0121] It should be noted that this application does not impose any restrictions on the setting of the sampling ratio. For ease of understanding, the embodiments of this application provide the following methods as examples:

[0122] In order to ensure the accuracy of detection, in one possible implementation, the sampling ratio can be set to 1:1, that is, the ratio of the number of network traffic sent to the number of initial network traffic is 1:1. Based on this, all network traffic sent can be detected. In this way, comprehensive detection is achieved, and network traffic can be finely detected and analyzed, effectively reducing the omission of stateful network traffic (i.e., abnormal network traffic), thereby ensuring the accuracy of detection. It should be noted that for this sampling method with a sampling ratio of 1:1, if the method of sending the full amount to the traffic detection device in the related art is adopted, although the detection accuracy can be guaranteed, the required bandwidth cost is high. Compared with the related art, after adopting the method provided by this application, only the target information (i.e., valid information) needs to be sent to the traffic detection device. Therefore, while ensuring the accuracy of detection, the bandwidth cost can be reduced. In particular, for network traffic detection scenarios where the volume of network traffic is very large, after adopting this application, the bandwidth cost of network traffic detection can be greatly reduced on the basis of ensuring the accuracy of detection. In actual applications, in the same network traffic detection scenario, for the same initial network traffic and the same amount of initial network traffic, the bandwidth cost required for using this application is about 8% of the original, which is reduced by about 92%, greatly reducing the bandwidth cost of network traffic detection.

[0123] In practical applications, for bandwidth cost considerations, the sampling ratio can also be set to N:1, where N is an integer greater than 1, that is, the ratio of the number of network flows sent to the number of initial network flows is N:1. In specific implementation, one initial network flow can be sampled for every N network flows sent. Based on this, the proportion of detection can be effectively reduced, which is conducive to reducing bandwidth costs. It should be noted that for this sampling method with a sampling ratio of N:1, if the method of sending the full amount to the flow detection device in the related art is adopted, the required bandwidth cost can be reduced to a certain extent by reducing the number. Compared with the related art, after adopting the method provided by the present application, since only the target information (i.e., valid information) needs to be sent to the flow detection device, the required bandwidth cost can be further reduced. That is to say, when N is the same, the bandwidth cost required by the present application is lower. When the bandwidth cost consumed is the same, the present application can set N to a smaller value, increase the proportion of network flows for detection, and achieve higher detection accuracy.

[0124] The present application has been introduced in detail through the above embodiments. In order to better understand the implementation of the method provided by the present application, the following takes the switch as the aforementioned programmable switch as an example to detect the scenario in which the first device sends network traffic to the second device. Correspondingly, please refer to Figure 7, which shows a schematic diagram of a system architecture for network traffic detection, which may specifically include a first device, a second device, a programmable switch, and a traffic detection device. The data processing method provided in the embodiment of the present application can be executed by a programmable switch. Specifically, the programmable switch may include a classification unit, a replication unit, a reorganization unit, an aggregation unit, and a service unit:

[0125] The classification unit can be used to classify network traffic sent from the first device to the second device, filter the network traffic that needs to be detected, and sample it according to a sampling ratio to obtain initial network traffic. In practical applications, the execution logic of the classification unit can be called inbound processing logic, which can be programmed based on the programmability of the programmable switch.

[0126] The replication unit can be used to replicate the initial network traffic. It should be noted that the output of the transmission network traffic will vary depending on the timing of the replication unit replicating the initial network traffic. In practical applications, the programmable switch's Packet Replication Engine (PRE) can be used to replicate the network traffic that needs to be replicated.

[0127] The reassembly unit can be used to parse the transmission network traffic, determine the type identifier, distinguish the duplicate network traffic as the network traffic to be tested, and extract information from the network traffic to be tested to obtain the target information. Accordingly, the distinguished initial network traffic can be sent to the business unit. In actual applications, the outbound processing logic can be configured in the reassembly unit to achieve the purpose of distinguishing the duplicate network traffic from the initial network traffic based on the outbound processing logic, and the distinguished duplicate network traffic can be modified to retain valid information, that is, the target information. Taking the network traffic as a message as an example, the extracted message fields carrying valid information can also be reassembled to obtain the target information in the same message form.

[0128] The execution statement of the outbound processing logic in the reorganization unit may be as follows:

[0129] If the packet is mirrored:

[0130] Make modifications and retain valid information;

[0131] Else:

[0132] No modification is made.

[0133] Here, packet may refer to transmitted network traffic.

[0134] In the reassembly unit, after obtaining the target information, if it corresponds to the aforementioned non-aggregation mode, the reassembly unit can send the obtained target information to the traffic detection device, specifically, it can send the target information to the traffic detection device through the port of the programmable switch. Corresponding to the aforementioned aggregation mode, the reassembly unit can send the obtained target information to the aggregation unit. In actual applications, the process of obtaining target information based on information extraction can also be considered as the process of copying target information from the network traffic to be tested. Specifically, the network traffic to be tested can be modified first, and valid information can be retained. Then, the underlying capability of the Collescing Mirror of the programmable switching chip of the programmable switch can be used to copy the content of the modified network traffic to be tested as is to obtain the target information. At the same time, the modified network traffic to be tested can be discarded. In actual applications, the reassembly unit can include mirror logic, based on which the adopted mode can be judged. Corresponding to different modes, the target information can be sent with different sending processes, and the differentiated initial network traffic can be sent to the business unit. The execution statement of the mirror logic can be as follows:

[0135] If mode is collescing mirror:

[0136] Write the target information into the cache address;

[0137] If mode is normal mirror:

[0138] Send target information to business units;

[0139] Else:

[0140] Sends initial network traffic to the business unit.

[0141] Here, mode may refer to the currently used processing mode, specifically the aforementioned aggregation mode, i.e., mode is collescing mirror, or the aforementioned non-aggregation mode, i.e., mode is normal mirror. In other cases, it may refer to sending the original network traffic to the service unit without modification.

[0142] The aggregation unit can be used to aggregate the received target information, aggregate multiple target information into one aggregate information, and then send the aggregate information to the flow detection device, specifically, it can be sending the aggregate information to the flow detection device through the port of the programmable switch. The aggregation unit can include two parts: packet sending and caching. Among them, the packet sending part can be used to control the sending method of sending multiple target information to the corresponding cache address. The packet sending part can be implemented using the aforementioned binary tree structure, specifically, it can be implemented based on the multi-level register of the programmable switch, which can reduce cache occupancy and improve cache utilization. The cache part can refer to writing the target information into the corresponding cache address, and after the cache address is full, the aggregate information of the specified length can be read from it for sending. Corresponding to the reorganization unit using the underlying capability of the Collescing Mirror of the programmable switching chip to copy the target information, the aggregation unit can write the target information into the cache address corresponding to the Collescing Mirror of the programmable switching chip, such as the on-chip storage of the programmable switching chip, specifically the static random-access memory (SRAM). In this way, after the aggregation reaches the specified length, the stored SARM information can be read out, specifically, the aggregation information of the specified length can be read, and the cache space can be released at the same time.

[0143] The business unit is used to send the network traffic that does not need to be detected output by the classification unit and the initial network traffic output by the reassembly unit to the second device. Specifically, these network traffic can be sent to the second device through the port of the programmable switch to achieve forwarding and ensure network communication.

[0144] Corresponding to the two different methods of post-copy and pre-copy, the present application provides two corresponding processing logic diagrams. Specific details are as follows:

[0145] The post-copy approach will be explained in conjunction with Figure 8, which shows a schematic diagram of the processing logic for two mirroring operations. In Figure 8, the initial network traffic obtained after processing by the inbound processing logic enters the queue scheduling unit (as shown in Figure 1). The queue scheduling unit can be used to schedule network traffic and, based on the queue scheduling unit, can determine which port to use to send the network traffic. In practical applications, the scheduling unit can be a traffic manager. Specifically, the queue scheduling unit can schedule the transmission network traffic to enter the outbound processing logic (as shown in Figure 2). The outbound processing logic determines that the type identifier of the transmission network traffic is a service identifier, so no modification is performed. For detection, the mirroring logic unit can then enter (as shown in Figure 3). Based on the mirroring logic, the transmission network traffic is replicated and the replicated network traffic is determined as transmission network traffic and enters the scheduling unit (as shown in Figure 4). The queue scheduling unit then schedules the transmission network traffic to enter the outbound processing logic (as shown in Figure 5). The outbound processing logic determines that the transmission network traffic is replicated. At this point, the traffic can be modified, and valid information, i.e., the target information, is retained based on the modification. Next, the mirroring logic (as shown in step 6 in the figure) is entered. If the aggregation mode is determined, the underlying capabilities of Collescing Mirror can be leveraged to create a copy of the modified content, retaining it intact, and then send it to the aggregation unit. The aggregation unit can write the target information into a cache address (as shown in step 7 in the figure). When the cache is full or the aggregate reaches a specified length, it can read aggregate information of the specified length from the cache address (as shown in step 8 in the figure). It is understood that the read aggregate information can also be encapsulated, primarily by encapsulating the address of the traffic detection device with the aggregate information. Since the aggregate information needs to be sent to the traffic detection device, the read aggregate information can be queued. The queue scheduling determines the corresponding switch port for subsequent transmission to the traffic detection device. In actual applications, this encapsulation can be implemented by programming the outbound processing logic. Accordingly, the aggregate information that has undergone queue scheduling can enter the outbound processing logic (as shown in step 9 in the figure). Based on the outbound processing logic, the encapsulation is completed and the encapsulated aggregate information can be sent to the port (as shown in step 10 in the figure). Finally, the port can be used to send the aggregate information to the corresponding traffic detection device. If the non-aggregation mode is determined, the target information is pushed to the programmable switch port. Similarly, for transmission network traffic whose type identifier is the service identifier, that is, initial network traffic, it is pushed to the port after the mirroring logic determines the traffic. In this post-replication method, two replications are performed after the outbound processing logic determines the traffic. The first replication is shown in Figure 4, and the second replication is shown in Figure 7. These two replications can be called double mirroring, and can also be called a double mirroring method.

[0146] It should be noted that in Figure 8, in order to facilitate distinction, dotted arrows are used to represent the flow of copied related information (such as copied network traffic, copied target information, etc.), and solid arrows are used to represent the flow of initial network traffic.

[0147] The method of pre-copying will be explained in conjunction with Figure 9, which shows a schematic diagram of the processing logic of multicast replication combined with a single mirror. In Figure 9, first, the initial network traffic obtained after being processed by the inbound processing logic enters the multicast replication (as shown in Figure 1), that is, the initial network traffic is directly multicast replicated by the replication unit to obtain the corresponding replicated network traffic. The multicast replication can be completed by the replication unit based on PRE. Then, the initial network traffic and the replicated network traffic can be determined as transmission network traffic and enter the queue scheduling (as shown in Figure 2). Since the transmission network traffic at this time includes both the initial network traffic and the replicated network traffic, in order to facilitate distinction in the queue scheduling entering the outbound processing logic and subsequent processing, similar to Figure 8, solid arrows and dotted arrows are shown in Figure 9, with solid arrows representing the flow of the initial network traffic and dotted arrows representing the flow of the related information obtained by replication. Next, the transmitted network traffic enters the outbound processing logic (shown in Figure 3). Based on this, the outbound processing logic determines whether the traffic type is a service identifier and remains unchanged. If the type is a copy identifier, the traffic is modified, retaining the valid information, i.e., the target information, based on the modification. Next, the unmodified and modified portions of the transmitted network traffic enter the mirroring logic (shown in Figure 4). If the aggregation mode is determined, the underlying capabilities of Collescing Mirror are leveraged to create a copy of the modified content, which is then sent to the aggregation unit. The aggregation unit writes the target information to a cache address (shown in Figure 5). When the cache is full or reaches a specified length, it reads aggregate information of a specified length from the cache address (shown in Figure 6). It is understood that the read aggregate information can also be encapsulated, primarily by encapsulating the address of the traffic detection device with the aggregate information. Since the aggregate information needs to be sent to the traffic detection device, it can be queued. The queue scheduling determines the corresponding switch port for subsequent transmission to the traffic detection device. In practical applications, outer encapsulation can be achieved by programming outbound processing logic. Accordingly, aggregated information that has undergone queue scheduling can enter the outbound processing logic (as shown in Figure 7). Based on the outbound processing logic, the outer encapsulation is completed, and the encapsulated aggregated information can enter the port (as shown in Figure 8). Finally, the port can be used to send the aggregated information to the corresponding traffic detection device. If the non-aggregation mode is determined, the target information is pushed to the port of the programmable switch. Similarly, for transmission network traffic whose type identifier is the service identifier, that is, the initial network traffic, it is pushed to the port after determination based on the mirroring logic.In this pre-replication approach, multicast replication is performed before the outbound processing logic (as shown in Figure 2), and only one replication is required after the outbound processing logic (as shown in Figure 5). Therefore, it can be called a multicast replication combined with a single mirroring approach.

[0148] It can be seen from the above technical solution that when it is necessary to detect network traffic in network communication, the network traffic to be tested can be obtained first, and the network traffic to be tested can refer to the network traffic used for detection. Then, information extraction can be performed on the network traffic to be tested to obtain target information corresponding to the network traffic to be tested. Among them, the target information can refer to the information required when detecting the network traffic to be tested, that is, the target information is effective information for detection. Therefore, after obtaining the target information, the target information can be sent to the traffic detection device, and the traffic detection device can detect the target information and obtain the detection result corresponding to the network traffic to be tested, thereby achieving the purpose of detection. Since the data volume of the target information is smaller than the data volume of the network traffic to be tested, the bandwidth cost required to send the target information to the traffic detection device is less than the bandwidth cost required to send the network traffic to be tested, thereby reducing the bandwidth cost on the basis of realizing the detection of network traffic.

[0149] In an actual network traffic detection scenario, when the switch forwards the received network traffic to the traffic detection device, it will first encapsulate the network traffic. The encapsulation can usually be to encapsulate other relevant information together with the network traffic. For example, the IP address of the network traffic receiving end (such as the IP address of the traffic detection device) can be encapsulated together with the network traffic. In this way, it can be sent based on the IP address carried after encapsulation. In the related art, encapsulation is performed for each network traffic. In the network traffic detection scenario where the network traffic volume is very large, the number of network traffic that needs to be detected is large. The encapsulation method in the related art is used to encapsulate each network traffic that needs to be detected and then send it. Each network traffic that needs to be detected needs to be sent once, and each sent network traffic carries the encapsulated IP address, etc. In this way, more sending resources will be occupied and the required bandwidth cost will be high.

[0150] In this regard, embodiments of the present application also provide a data processing method that can be applied to a switch. The switch can aggregate multiple network flows that need to be detected and encapsulate the multiple network flows into a single detection flow based on the aggregation. The encapsulated detection flow can include multiple network flows, so the switch can send the detection flow to the flow detection device for detection. Because encapsulation can encapsulate multiple network flows into a single detection flow, only one encapsulation is required. Accordingly, multiple network flows can only encapsulate one IP address, and this single detection flow contains multiple network flows. Therefore, sending this single detection flow can achieve the purpose of sending multiple network flows, which can reduce the required bandwidth cost.

[0151] Correspondingly, FIG10 is a flowchart of a data processing method provided in an embodiment of the present application, which is illustrated by taking a switch as an example of the aforementioned computer device. The method includes S1001-S1003:

[0152] S1001: Obtain multiple network flows to be tested.

[0153] When network traffic needs to be tested, the switch can obtain multiple network flows to be tested. The network flows to be tested can refer to the network flows used for testing. In practical applications, the switch can obtain the network flows to be tested from a port of a device sending the network flows (such as the aforementioned first device). For details on how to obtain the multiple network flows to be tested, please refer to the aforementioned implementation of S301 and will not be further elaborated here.

[0154] S1002: Aggregate multiple network flows to be tested, and encapsulate the multiple network flows to be tested into one detection flow.

[0155] For the multiple network flows to be tested obtained, the switch can aggregate the multiple network flows to be tested and encapsulate the multiple network flows to be tested into one detection flow. The detection flow can contain multiple network flows to be tested, and the number of detection flows is one. That is, multiple network flows to be tested only need to be encapsulated once, which is conducive to reducing the required bandwidth cost.

[0156] It should be noted that this application does not impose any restrictions on how to aggregate and encapsulate the detection traffic. In actual applications, different aggregation and encapsulation methods may result in different detection traffic. The difference in detection traffic may specifically refer to the different content of the detection traffic. The content of the detection traffic is directly related to the detection result, and detection traffic with different content can obtain different detection results. For ease of understanding, the embodiments of this application provide the following methods as examples:

[0157] In actual applications, a switch, as a forwarding device for network traffic, is an intermediate device compared to the sending and receiving devices of network traffic. In a network traffic detection scenario, a switch, as an intermediate device, is often responsible for forwarding and detecting network traffic from multiple sending devices. That is, the aforementioned multiple network traffic to be tested can be network traffic to be tested from multiple sources. One of the main purposes of detection is to locate the source of the anomaly. In order to ensure that the source of the anomaly can be located based on the detection results, the sources of each network traffic to be tested can be aggregated and encapsulated in the detection traffic. Among them, the source of the network traffic to be tested can be used to indicate which port of the device the network traffic to be tested comes from, which device it comes from, etc. In actual applications, for multiple network traffic to be tested, the multiple network traffic to be tested can be processed separately first to determine the source of each network traffic to be tested. Then, a detection traffic can be obtained based on the aggregate encapsulation, and the sources of each network traffic to be tested can be encapsulated in a detection traffic based on the aggregate encapsulation. Based on this, the obtained detection traffic can reflect the source of each network traffic to be tested. In this way, after testing the detection traffic, once an abnormality is detected, the abnormal port, device, etc. can be located based on the source.

[0158] Furthermore, the amount of network traffic under test originating from the same source can be aggregated. Accordingly, the amount of network traffic under test from each source can be encapsulated within this test traffic. This resulting test traffic can then reflect the amount of network traffic under test originating from the same source. Thus, after testing the test traffic, the anomaly ratio of network traffic under test from the same source can be used to assess the network security of multiple sources. For example, the higher the anomaly ratio for a particular source, the less secure the network from that source is.

[0159] S1003: Send the detection traffic to the traffic detection device.

[0160] The traffic detection device can be used to detect the test traffic and obtain the test results corresponding to the multiple network traffic to be tested. Based on this, the test is completed. Since the encapsulation can encapsulate multiple network traffic into one test traffic, it only needs to be encapsulated once. Correspondingly, multiple network traffic can only encapsulate one IP address. Since this one test traffic contains multiple network traffic, sending this one test traffic can achieve the purpose of sending multiple network traffic, which can reduce the bandwidth cost required for the test while achieving the test.

[0161] For a better understanding, let's take the aforementioned programmable switch as an example. To facilitate the aggregation and encapsulation of multiple network flows to be tested into a single test flow, the underlying Collescing Mirror capability of the programmable switch's programmable switching chip can be utilized to write the multiple network flows to be tested into the corresponding cache address of the programmable switching chip. Based on this, the cache address can be used to cache and store the multiple network flows to be tested. Typically, once the cache address is full, aggregation and encapsulation can be performed to obtain the corresponding test flow, which helps save switch hardware performance. For example, this can save the performance of the programmable switch's programmable switching chip.

[0162] It can be seen from the above technical solution that when network traffic needs to be detected in network communication, multiple network traffic to be tested can be aggregated and encapsulated into one detection traffic. This detection traffic can contain multiple network traffic to be tested. Therefore, the method of detecting this one detection traffic can ensure the detection of multiple network traffic to be tested. At the same time, since multiple network traffic to be tested is encapsulated into one detection traffic, it only needs to be encapsulated once. In actual applications, taking the encapsulation of IP addresses as an example, after adopting this application, multiple network traffic to be tested can be encapsulated with only one IP address. Therefore, sending this one detection traffic can achieve the purpose of sending multiple network traffic, which can reduce the bandwidth cost required for detection on the basis of achieving detection.

[0163] It should be noted that, based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

[0164] Based on the data processing method provided in the embodiment corresponding to FIG3 , the embodiment of the present application further provides a data processing device 1100, which includes an acquisition unit 1101, an extraction unit 1102, and a sending unit 1103:

[0165] The acquisition unit 1101 is used to acquire the network traffic to be tested;

[0166] The extraction unit 1102 is configured to extract information from the network traffic to be measured to obtain target information corresponding to the network traffic to be measured; the data volume of the target information is smaller than the data volume of the network traffic to be measured;

[0167] The sending unit 1103 is used to send the target information to the traffic detection device; the traffic detection device is used to detect the target information and obtain a detection result corresponding to the network traffic to be tested.

[0168] In a possible implementation, the network traffic to be measured includes n network traffic to be measured, where n is an integer greater than 1, and the sending unit is further configured to:

[0169] Aggregating target information corresponding to the n network flows to be measured;

[0170] The aggregated information is sent to the traffic detection device; the traffic detection device is used to detect the received aggregated information to obtain a detection result.

[0171] In a possible implementation, the device further includes a writing unit:

[0172] The writing unit is used to write the target information corresponding to the n network flows to be tested into a cache address;

[0173] The sending unit is further configured to:

[0174] Aggregating target information corresponding to the n network flows to be tested written into the cache address;

[0175] The aggregated information obtained by aggregation is read from the cache address, and the read aggregated information is sent to the traffic detection device.

[0176] In a possible implementation, the cache address includes a first cache address and a second cache address, the data volume of the target information is a preset data length, and the writing unit is further configured to:

[0177] For the i-th target information corresponding to the i-th network flow to be tested among the n network flows to be tested, if it is determined that the remaining cache length of the first cache address is greater than or equal to the preset data length, the i-th target information is written to the first cache address; i is an integer greater than or equal to 2 and less than or equal to n, and the first cache address stores the i-1-th target information corresponding to the i-1-th network flow to be tested among the n network flows to be tested;

[0178] If it is determined that the remaining cache length of the first cache address is less than the preset data length, writing the i-th target information into the second cache address;

[0179] Until the n target information corresponding to the n network flows to be tested are written into the cache address.

[0180] In a possible implementation, the maximum cache length of the first cache address includes m preset data lengths, where m is an integer greater than 1, and the writing unit is further configured to:

[0181] If i≤m, determining that the remaining cache length of the first cache address is greater than or equal to the preset data length;

[0182] If i>m, it is determined that the remaining cache length of the first cache address is less than the preset data length.

[0183] In a possible implementation, the first cache address corresponds to a first child node, the second cache address corresponds to a second child node, the first child node and the second child node correspond to the same parent node, and the parent node is a node included in the binary tree that receives the i-th target information. The writing unit is further configured to:

[0184] If the length of the historical information sent by the parent node through the first child node is less than the maximum cache length of the first child node, determining that the remaining cache length of the first cache address is greater than or equal to the preset data length; the maximum cache length of the first child node is determined based on the number of levels of the parent node in the binary tree and the maximum cache length of the first cache address, the maximum cache length of the first cache address including m of the preset data lengths, where m is an integer greater than 1;

[0185] Using the parent node to send the i-th target information to the first child node, and writing the i-th target information into the first cache address through the first child node;

[0186] If the length of the historical information sent by the parent node through the first child node is equal to the maximum cache length of the first child node, determining that the remaining cache length of the first cache address is less than the preset data length;

[0187] The parent node is used to send the i-th target information to the second child node, and the i-th target information is written into the second cache address through the second child node.

[0188] In a possible implementation, the sending unit is further configured to:

[0189] In the process of writing the n network flows to be tested into the cache address, determining the cache address storing m target information among the first cache address and the second cache address as the target cache address;

[0190] Aggregating based on the m target information stored in the target cache address;

[0191] Until the aggregation of target information corresponding to the n measured flows is completed.

[0192] In a possible implementation, the acquiring unit is further configured to:

[0193] Get transmission network traffic;

[0194] Parsing the transmission network traffic to obtain a type identifier of the transmission network traffic;

[0195] If the type identifier is a copy identifier, the transmission network traffic is determined as the network traffic to be tested.

[0196] In a possible implementation, the acquiring unit is further configured to:

[0197] Acquire initial network traffic; the type identifier of the initial network traffic is a service identifier;

[0198] Performing multicast replication on the initial network traffic to obtain replicated network traffic; the type identifier of the replicated network traffic is the replication identifier;

[0199] The initial network traffic and the duplicate network traffic are respectively determined as the transmission network traffic.

[0200] In a possible implementation, the acquiring unit is further configured to:

[0201] Acquire initial network traffic, and determine the acquired initial network traffic as the transmission network traffic; the type identifier of the initial network traffic is a service identifier;

[0202] After determining that the type identifier of the transmission network traffic is the service identifier, the transmission network traffic is copied, and the copied network traffic obtained by the copy is determined as the transmission network traffic; the type identifier of the copied network traffic is the copy identifier.

[0203] In a possible implementation, the acquiring unit is further configured to:

[0204] In the process of the first device sending network traffic to the second device, the sent network traffic is sampled according to the sampling ratio to obtain the initial network traffic.

[0205] It can be seen from the above technical solution that when it is necessary to detect network traffic in network communication, the network traffic to be tested can be obtained first, and the network traffic to be tested can refer to the network traffic used for detection. Then, information extraction can be performed on the network traffic to be tested to obtain target information corresponding to the network traffic to be tested. Among them, the target information can refer to the information required when detecting the network traffic to be tested, that is, the target information is effective information for detection. Therefore, after obtaining the target information, the target information can be sent to the traffic detection device, and the traffic detection device can detect the target information and obtain the detection result corresponding to the network traffic to be tested, thereby achieving the purpose of detection. Since the data volume of the target information is smaller than the data volume of the network traffic to be tested, the bandwidth cost required to send the target information to the traffic detection device is less than the bandwidth cost required to send the network traffic to be tested, thereby reducing the bandwidth cost on the basis of realizing the detection of network traffic.

[0206] Based on the data processing method provided in the embodiment corresponding to FIG10 , the embodiment of the present application further provides another data processing device 1200, which includes an acquisition unit 1201, an aggregation unit 1202, and a sending unit 1203:

[0207] The acquisition unit 1201 is used to acquire multiple network flows to be tested;

[0208] The aggregation unit 1202 is configured to aggregate the plurality of network flows to be tested and encapsulate the plurality of network flows to be tested into one detection flow;

[0209] The sending unit 1203 is used to send the detection traffic to the traffic detection device; the traffic detection device is used to detect the detection traffic and obtain detection results corresponding to the multiple network traffic to be tested.

[0210] It can be seen from the above technical solution that when network traffic needs to be detected in network communication, multiple network traffic to be tested can be aggregated and encapsulated into one detection traffic. This detection traffic can contain multiple network traffic to be tested. Therefore, the method of detecting this one detection traffic can ensure the detection of multiple network traffic to be tested. At the same time, since multiple network traffic to be tested is encapsulated into one detection traffic, it only needs to be encapsulated once. In actual applications, taking the encapsulation of IP addresses as an example, after adopting this application, multiple network traffic to be tested can be encapsulated with only one IP address. Therefore, sending this one detection traffic can achieve the purpose of sending multiple network traffic, which can reduce the bandwidth cost required for detection on the basis of achieving detection.

[0211] The present application also provides a computer device, which may be a terminal. For example, a smartphone is used as the terminal.

[0212] FIG13 shows a block diagram of a partial structure of a smartphone provided in an embodiment of the present application. Referring to FIG13 , the smartphone includes components such as a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390. The input unit 1330 may include a touch panel 1331 and other input devices 1332, the display unit 1340 may include a display panel 1341, and the audio circuit 1360 may include a speaker 1361 and a microphone 1362. Those skilled in the art will appreciate that the smartphone structure shown in FIG13 does not constitute a limitation on the smartphone, and may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0213] The memory 1320 can be used to store software programs and modules. The processor 1380 executes the various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 1320. The memory 1320 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created based on the use of the smartphone (such as audio data, a phone book, etc.). In addition, the memory 1320 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0214] Processor 1380 is the control center of the smartphone, connecting all components of the smartphone using various interfaces and circuits. It executes software programs and / or modules stored in memory 1320 and accesses data stored in memory 1320 to perform various smartphone functions and process data. Optionally, processor 1380 may include one or more processing units. Preferably, processor 1380 integrates an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 1380.

[0215] In this embodiment, the steps executed by the processor 1380 in the smartphone may be implemented based on the structure shown in FIG. 13 .

[0216] The computer device provided in the embodiment of the present application can also be a server, as shown in Figure 14, which is a structural diagram of the server 1400 provided in the embodiment of the present application. The server 1400 may have relatively large differences due to different configurations or performances, and may include one or more processors, such as a central processing unit (CPU) 1422, and a memory 1432, one or more storage media 1430 (such as one or more mass storage devices) storing application programs 1442 or data 1444. Among them, the memory 1432 and the storage medium 1430 can be temporary storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1422 can be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the server 1400.

[0217] The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0218] In the data processing method embodiment provided based on FIG. 3 , the central processing unit 1422 in the server 1400 may perform the following steps:

[0219] Get the network traffic to be tested;

[0220] Extracting information from the network traffic to be measured to obtain target information corresponding to the network traffic to be measured; the data volume of the target information is smaller than the data volume of the network traffic to be measured;

[0221] The target information is sent to a flow detection device; the flow detection device is used to detect the target information and obtain a detection result corresponding to the network flow to be tested.

[0222] In the data processing method embodiment provided in FIG10 , the central processing unit 1422 in the server 1400 may perform the following steps:

[0223] Get multiple network flows to be tested;

[0224] Aggregating the plurality of network flows to be tested, and encapsulating the plurality of network flows to be tested into one detection flow;

[0225] The detection traffic is sent to a traffic detection device; the traffic detection device is used to detect the detection traffic and obtain detection results corresponding to the multiple network traffic to be tested.

[0226] According to one aspect of the present application, a computer-readable storage medium is provided, which is used to store a computer program. When the computer program is executed by a computer device, the computer device executes the data processing method described in the above embodiments.

[0227] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional implementations of the above-described embodiments.

[0228] The descriptions of the processes or structures corresponding to the above figures have different focuses. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0229] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0230] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0231] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0232] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0233] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0234] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technical members in this field should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, the method being executed by a computer device, the method comprising: Get the network traffic to be tested; Extracting information from the network traffic to be tested to obtain target information corresponding to the network traffic to be tested; The data volume of the target information is smaller than the data volume of the network traffic to be measured; Sending the target information to a flow detection device; The flow detection device is used to detect the target information and obtain the detection result corresponding to the network flow to be tested.

2. According to the method of claim 1, the network traffic to be tested includes n network traffic to be tested, n is an integer greater than 1, and the sending of the target information to the traffic detection device includes: Aggregating target information corresponding to the n network flows to be tested; Sending aggregated information obtained by aggregation to the traffic detection device; The flow detection device is used to detect the received aggregation information to obtain a detection result.

3. The method according to claim 2, before aggregating the target information respectively corresponding to the n network flows to be tested, the method further comprises: Writing the target information corresponding to the n network flows to be tested into the cache address; The aggregating target information respectively corresponding to the n network flows to be tested includes: Aggregating target information corresponding to the n network flows to be tested written into the cache address; The sending the aggregated information to the traffic detection device includes: The aggregated information is read from the cache address, and the aggregated information is sent to the traffic detection device.

4. The method according to claim 3, wherein the cache address comprises a first cache address and a second cache address, the data volume of the target information is a preset data length, and the writing of the target information corresponding to the n network flows to be tested into the cache address comprises: For the i-th target information corresponding to the i-th network flow to be tested among the n network flows to be tested, if it is determined that the remaining cache length of the first cache address is greater than or equal to the preset data length, the i-th target information is written into the first cache address; i is an integer greater than or equal to 2 and less than or equal to n, and the first cache address stores the i-1th target information corresponding to the i-1th network flow to be tested among the n network flows to be tested; If it is determined that the remaining cache length of the first cache address is less than the preset data length, writing the i-th target information into the second cache address; Until the n target information corresponding to the n network flows to be tested are written into the cache address.

5. The method according to claim 4, wherein the maximum cache length of the first cache address includes m of the preset data lengths, where m is an integer greater than 1, and the determining that the remaining cache length of the first cache address is greater than or equal to the preset data length comprises: If i≤m, determining that the remaining cache length of the first cache address is greater than or equal to the preset data length; The determining that the remaining cache length of the first cache address is less than the preset data length includes: If i>m, it is determined that the remaining cache length of the first cache address is less than the preset data length.

6. The method according to claim 4, wherein the first cache address corresponds to a first child node, the second cache address corresponds to a second child node, the first child node and the second child node correspond to the same parent node, the parent node is a node included in the binary tree that receives the i-th target information, and determining that the remaining cache length of the first cache address is greater than or equal to the preset data length, and writing the i-th target information into the first cache address, comprises: If the length of the historical information sent by the parent node through the first child node is less than the maximum cache length of the first child node, determine that the remaining cache length of the first cache address is greater than or equal to the preset data length; the maximum cache length of the first child node is determined according to the number of layers of the parent node in the binary tree and the maximum cache length of the first cache address, and the maximum cache length of the first cache address includes m preset data lengths, where m is an integer greater than 1; Using the parent node to send the i-th target information to the first child node, and writing the i-th target information into the first cache address through the first child node; If it is determined that the remaining cache length of the first cache address is less than the preset data length, writing the i-th target information into the second cache address includes: If the length of the historical information sent by the parent node through the first child node is equal to the maximum cache length of the first child node, determining that the remaining cache length of the first cache address is less than the preset data length; The parent node is used to send the i-th target information to the second child node, and the i-th target information is written into the second cache address through the second child node.

7. The method according to claim 5 or 6, wherein the step of aggregating target information corresponding to the n network flows to be tested written into the cache address comprises: In the process of writing the n network flows to be tested into the cache address, determining a cache address storing m target information among the first cache address and the second cache address as a target cache address; Aggregating based on the m target information stored in the target cache address; Until the aggregation of target information corresponding to the n measured flows is completed.

8. According to any one of claims 1 to 6, the step of obtaining the network traffic to be tested comprises: Get transmission network traffic; Parsing the transmission network traffic to obtain a type identifier of the transmission network traffic; If the type identifier is a copy identifier, the transmission network traffic is determined as the network traffic to be tested.

9. The method according to claim 8, wherein obtaining the transmission network traffic comprises: Get initial network traffic; The type identifier of the initial network traffic is a service identifier; Performing multicast replication on the initial network traffic to obtain replicated network traffic; The type identifier of the duplicate network traffic is the duplication identifier; The initial network traffic and the duplicate network traffic are respectively determined as the transmission network traffic.

10. The method according to claim 8, wherein obtaining the transmission network traffic comprises: Acquire initial network traffic, and determine the acquired initial network traffic as the transmission network traffic; The type identifier of the initial network traffic is a service identifier; After determining that the type identifier of the transmission network traffic is the service identifier, copying the transmission network traffic, and determining the copied network traffic as the transmission network traffic; The type of replication network traffic The copy identifier is identified.

11. The method according to claim 9 or 10, wherein obtaining initial network traffic comprises: In the process of the first device sending network traffic to the second device, the sent network traffic is sampled according to the sampling ratio to obtain the initial network traffic.

12. A data processing method, the method comprising: Get multiple network flows to be tested; Aggregating the multiple network flows to be tested, and encapsulating the multiple network flows to be tested into one detection flow; Sending the detection flow to a flow detection device; The flow detection device is used to detect the detection flow and obtain detection results corresponding to the multiple network flows to be tested.

13. A data processing device, comprising an acquisition unit, an extraction unit and a sending unit: The acquisition unit is used to acquire the network traffic to be tested; The extraction unit is used to extract information from the network traffic to be tested to obtain target information corresponding to the network traffic to be tested; The data volume of the target information is smaller than the data volume of the network traffic to be measured; The sending unit is used to send the target information to the flow detection device; The flow detection device is used to detect the target information and obtain the detection result corresponding to the network flow to be tested.

14. A data processing device, comprising an acquisition unit, an aggregation unit and a sending unit: The acquisition unit is used to acquire multiple network flows to be tested; The aggregation unit is used to aggregate the multiple network flows to be tested and encapsulate the multiple network flows to be tested into one detection flow; The sending unit is used to send the detection flow to the flow detection device; The flow detection device is used to detect the detection flow and obtain detection results corresponding to the multiple network flows to be tested.

15. A computer device, comprising a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1 to 12 according to the instructions in the computer program.

16. A computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a computer device, the computer device executes the method according to any one of claims 1 to 12.

17. A computer program product, comprising a computer program, which, when executed on a computer device, causes the computer device to execute the method according to any one of claims 1 to 12.

Citation Information

Patent Citations

  • Traffic feature extraction method and system, storage medium and electronic equipment

    CN114024758A

  • Traffic visualization method and device, equipment and storage medium

    CN114884882A

  • Data processing method and related device

    CN116366503A

  • Endpoint network traffic analysis

    US20200287920A1

Cited By

  • Memory management method and device for large language model

    CN119576805A