Network packet processing method and apparatus, storage medium, and electronic device

Abstract

The application discloses a network packet processing method and device, a storage medium and an electronic device, relates to the technical field of smart home / smart home, and the network packet processing method comprises the following steps: obtaining a first request network packet associated with a historical request from a plurality of candidate network packets; determining at least one second request network packet matched with the request acknowledgement number of the first request network packet from the plurality of candidate network packets; determining a response acknowledgement number based on the sequence number of the first request network packet, the first data length of the first request network packet and the second data length of the at least one second request network packet, and obtaining a network packet corresponding to the response acknowledgement number; and storing the first request network packet, the at least one second request network packet and at least one response network packet. The application solves the technical problem of low processing efficiency of network packet data in the prior art.

Description

Technical Field

[0001] This application relates to the field of smart home / intelligent home technology, and more specifically, to a network packet processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] Traffic data is currently receiving widespread attention in the testing field, leading to the emergence of a series of packet capture and decryption tools. However, existing tools can only view recorded traffic data (network packet data) and cannot store network packet data for proprietary business purposes in a timely and effective manner, resulting in low processing efficiency for network packet data. Summary of the Invention

[0003] This application provides a network packet processing method, apparatus, storage medium, and electronic device to at least solve the technical problem of low processing efficiency of network packet data in related technologies.

[0004] According to one aspect of the embodiments of this application, a network packet processing method is provided, comprising: obtaining a first request network packet associated with a historical request from a plurality of candidate network packets, wherein the historical request is associated with a plurality of request network packets and at least one response network packet, the plurality of request network packets including the first request network packet; determining at least one second request network packet from the plurality of candidate network packets that matches the request acknowledgment number of the first request network packet; determining a response acknowledgment number based on the sequence number of the first request network packet, a first data length of the first request network packet, and a second data length of the at least one second request network packet, and obtaining the network packet corresponding to the response acknowledgment number, wherein the at least one response network packet includes the network packet corresponding to the response acknowledgment number; and storing the first request network packet, the at least one second request network packet, and the at least one response network packet.

[0005] According to another aspect of the embodiments of this application, a network packet processing apparatus is also provided, comprising: an acquisition unit, configured to acquire a first request network packet associated with a historical request from a plurality of candidate network packets, wherein the historical request is associated with a plurality of request network packets and at least one response network packet, the plurality of request network packets including the first request network packet; a first determination unit, configured to determine at least one second request network packet from the plurality of candidate network packets that matches the request confirmation number of the first request network packet; a second determination unit, configured to determine a response confirmation number based on the sequence number of the first request network packet, a first data length of the first request network packet, and a second data length of the at least one second request network packet, and acquire the network packet corresponding to the response confirmation number, wherein the at least one response network packet includes the network packet corresponding to the response confirmation number; and a storage unit, configured to store the first request network packet, the at least one second request network packet, and the at least one response network packet.

[0006] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the network packet processing method described above.

[0007] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the network packet processing method described above through the computer program.

[0008] In this embodiment, by utilizing the network packet processing method described above, the association information of the current first request network packet can be used to quickly determine multiple other request network packets belonging to the same historical request as the current first request network packet and at least one corresponding response network packet. All request network packets and response packets belonging to the same historical request are merged and stored, thereby achieving the goal of quickly and efficiently acquiring and storing all request network packets and response packets associated with each historical request. This achieves the technical effect of improving the processing efficiency of network packet data and solves the technical problem of low processing efficiency of network packet data in related technologies. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the hardware environment for a network packet processing method according to an embodiment of this application;

[0012] Figure 2 This is a schematic diagram of the flow of an optional network packet processing method according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of an optional network packet processing method according to an embodiment of this application;

[0014] Figure 4 This is a schematic diagram of another optional network packet processing method according to an embodiment of this application;

[0015] Figure 5 This is a schematic diagram of another optional network packet processing method according to an embodiment of this application;

[0016] Figure 6 This is a schematic diagram of an optional information processing apparatus according to an embodiment of the present invention;

[0017] Figure 7 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] According to one aspect of the embodiments of this application, an interaction method for smart home devices is provided. This interaction method for smart home devices is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligencehouse ecosystems. Optionally, in this embodiment, the above-mentioned interaction method for smart home devices can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0021] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0022] Alternatively, as an optional implementation, such as Figure 2 As shown, network packet processing methods include:

[0023] S202, Obtain a first request network packet associated with a historical request from a plurality of candidate network packets, wherein the historical request is associated with a plurality of request network packets and at least one response network packet, and the plurality of request network packets include the first request network packet;

[0024] S204, determine at least one second request network packet from a plurality of candidate network packets that matches the request acknowledgment number of the first request network packet;

[0025] S206, based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of at least one second request network packet, determine the response confirmation number and obtain the network packet corresponding to the response confirmation number, wherein at least one response network packet includes the network packet corresponding to the response confirmation number;

[0026] S208, store the first request network packet, at least one second request network packet, and at least one response network packet.

[0027] Optionally, in this embodiment, the above-described network packet processing method can be applied, but is not limited to, in scenarios involving the recording and storage of traffic data. Specifically, in this scenario, request and response data from proprietary services are monitored, such as acquiring network data packets from multiple protocols (corresponding to multiple ports) of a service under test, where the network data packets include request network packets and response network packets. Further, after recording the traffic data (i.e., network data packets), the data is categorized according to different protocols, and the resulting data is persistently stored. This data is then used for playback by other services, thereby providing real user traffic data for regression testing and performance stress testing in various version iterations.

[0028] Optionally, in this embodiment, the above-described network packet processing method can be applied, but is not limited to, in multi-environment reuse scenarios. Specifically, the network data packets corresponding to each historical request initiated in the first environment are obtained and classified and stored according to protocol type and request order. A historical request can be associated with multiple network request packets and at least one network response packet. The multiple network request packets and at least one network response packet associated with a historical request can be stored as a whole unit.

[0029] Furthermore, for the second environment, which is different from the first environment, the ordered network data packets stored for the first environment are directly used to test the second environment, thereby achieving the purpose of multi-environment reuse (restoration test) of traffic data.

[0030] It should be noted that the first environment and the second environment mentioned above may refer to different external operating environments of a product, and may also refer to different internal test versions of a product, but this embodiment does not limit this.

[0031] Optionally, in this embodiment, the first request network packet may be, but is not limited to, a network packet associated with a historical request among a plurality of candidate network packets. The plurality of candidate network packets may be, but is not limited to, obtained based on recorded traffic data stored in a cloud database. The cloud database may be, but is not limited to, used to store traffic data obtained by recording network packets with multiple network cards, multiple protocols, and multiple ports in a certain environment.

[0032] Optionally, in this embodiment, a historical request may be associated with, but is not limited to, multiple request network packets and at least one response network packet, wherein the first request network packet may be, but is not limited to, the request network packet with the earliest sequence number among the multiple request network packets.

[0033] It is understandable that the request data corresponding to historical requests often exceeds the amount of data that a single request network packet can carry, thus requiring multiple request network packets to carry it together.

[0034] Optionally, in this embodiment, the sequence numbers of multiple request network packets associated with historical requests are different, but the request confirmation numbers are the same.

[0035] Optionally, in this embodiment, when a first request network packet associated with a historical request is obtained, other request network packets identical to the first request network packet are determined from multiple candidate network packets based on the acknowledgment number of the first request network packet.

[0036] Optionally, in this embodiment, the response confirmation number of at least one response network packet associated with the historical request can be determined based on, but is not limited to, the sequence number of the first request network packet and the data length of all request network packets, or based on, but is not limited to, the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of at least one second request network packet.

[0037] Optionally, in this embodiment, after obtaining the above-mentioned response confirmation number, the network packet corresponding to the above-mentioned response confirmation number is obtained from multiple candidate network packets, wherein at least one response network packet includes the network packet corresponding to the above-mentioned response confirmation number.

[0038] Optionally, in this embodiment, after obtaining multiple request network packets and at least one response network packet associated with historical requests, the multiple network request packets and at least one network response packet can be stored as a whole unit, but is not limited to. Alternatively, the multiple network request packets can be stored as a first whole unit, and the at least one network response packet can be stored as a second whole unit, and a mapping relationship between the first whole unit and the second whole unit can be established.

[0039] It should be noted that the above storage can be, but is not limited to, persistent storage, and can be, but is not limited to, used for data replay in other services, thereby achieving the purpose of using real user traffic data for regression testing and performance stress testing in each version iteration.

[0040] To further illustrate, the above network packet processing method can be applied to a scenario of packet-based storage of traffic data. The specific steps are as follows:

[0041] Step S302: Obtain the first request network packet associated with the historical request from multiple candidate networks;

[0042] Step S304: Obtain the request confirmation number of the first request network packet;

[0043] Step S306: Based on the request confirmation number, obtain at least one second request network packet that matches the request network packet from multiple candidate network packets;

[0044] Step S308: Determine the response confirmation number based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of at least one second request network packet;

[0045] Step S310: Using the response confirmation number, obtain the network packet corresponding to the response confirmation number from multiple candidate network packets, and determine at least one response network packet based on the network packet corresponding to the response confirmation number;

[0046] Step S312: Store the first request network packet, at least one second request network packet, and at least one response network packet into the target database.

[0047] The embodiments provided in this application involve obtaining a first request network packet associated with a historical request from multiple candidate network packets, wherein the historical request is associated with multiple request network packets and at least one response network packet, and the multiple request network packets include the first request network packet; determining at least one second request network packet from the multiple candidate network packets that matches the request acknowledgment number of the first request network packet; determining a response acknowledgment number based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of the at least one second request network packet, and obtaining the network packet corresponding to the response acknowledgment number, wherein the at least one response network packet includes the network packet corresponding to the response acknowledgment number; and storing the first request network packet, the at least one second request network packet, and the at least one response network packet. Using the above method, the association information of the current first request network packet can be used to quickly determine other multiple request network packets and at least one corresponding response network packet belonging to the same historical request as the current first request network packet, and all request network packets and response packets belonging to the same historical request can be stored together, thereby achieving the goal of quickly and efficiently obtaining and storing all request network packets and response packets associated with each historical request, thus realizing the technical effect of improving the processing efficiency of network packet data.

[0048] As an optional approach, after determining at least one second request network packet from multiple candidate network packets that matches the request acknowledgment number of the first request network packet, the method further includes:

[0049] S1, determine the position of the first request network packet in the initial queue as the head position of the initial queue, and determine at least one second request network packet as the first position after the head position;

[0050] S2, place the first request network packet in the initial queue according to the header position, and place at least one second request network packet in the initial queue according to the first position to obtain the first queue.

[0051] Optionally, in this embodiment, after determining at least one second request network packet that matches the request acknowledgment number of the first request network packet, the first request network packet and at least one second request network packet may be sorted to form a first queue, wherein the first request network packet is located at the head of the first queue and at least one second request network packet is located at a first position after the head position.

[0052] Optionally, in this embodiment, at least one second request network packet may, but is not limited to, determine the sorting order in the first position based on the sequence number of each request network packet, wherein the request network packet with the earlier sequence number is sorted first, and the request network packet with the later sequence number is sorted last.

[0053] It should be noted that the sequence number of the request network packet can, but is not limited to, indicate the order of the local request data carried by the request network packet within the total request data corresponding to the historical request. It can be understood that by arranging multiple request network packets in an ordered manner in the first queue, the local request data associated with each request network packet is arranged in its original order. Therefore, the stored request network packets can directly reproduce (restore) historical requests, thereby achieving the technical effect of improving the data storage efficiency of network packet data while further ensuring the data storage and reproducibility of network packet data.

[0054] The embodiments provided in this application determine the position of the first request network packet in the initial queue as the head position of the initial queue, and determine at least one second request network packet as the first position after the head position; the first request network packet is placed in the initial queue according to the head position, and at least one second request network packet is placed in the initial queue according to the first position, resulting in a first queue. Improving the form of the first queue by merging and storing multiple request network packets associated with historical requests saves storage space, improves storage space utilization, and thus improves the storage efficiency of traffic data. Furthermore, while improving the data storage efficiency of traffic data, it further ensures the data storage restoreability of traffic data.

[0055] As an optional approach, the response acknowledgment number is determined based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of at least one second request network packet, including:

[0056] S1, determine the sum of the first data length and the second data length as the third data length of the first queue;

[0057] S2, determine the sum of the sequence number and the third data length as the response confirmation number.

[0058] Optionally, in this embodiment, the first data length of the first request network packet and the second data length of at least one second request network packet are summed to obtain the third data length of the first queue, and the sequence number and the third data length are summed to obtain the response confirmation number.

[0059] Optionally, in this embodiment, the first data length of the first request network packet, the second data length of at least one second request network packet, and the sequence number are summed to obtain the response confirmation number.

[0060] It should be noted that, compared to accumulating the sequence number and the data length of each request network packet one by one, directly adding the sequence number and the length of the first queue composed of each request network packet results in fewer calculations (and fewer iterations), reducing the computational complexity of the response acknowledgment number, improving the computational efficiency of the response acknowledgment number, and thus improving the efficiency of data acquisition and storage of network packet data.

[0061] The embodiments provided in this application determine the third data length of the first queue by summing the first data length and the second data length; and determine the response acknowledgment number by summing the sequence number and the third data length. This achieves the goal of reducing the computational complexity of the response acknowledgment number and improving its computational efficiency, thereby enhancing the technical effect of improving the data acquisition and storage efficiency of network packet data.

[0062] As an optional approach, storing the first request network packet, at least one second request network packet, and at least one response network packet includes:

[0063] S1, determine the position of at least one response network packet in the first queue as the second position of the first queue, wherein the second position is after the first position;

[0064] S2, according to the second position, place at least one response network packet in the first queue to obtain the second queue;

[0065] S3, store the second queue in the local database.

[0066] Optionally, in this embodiment, the first queue and at least one response network packet may be sorted to form a second queue, wherein the at least one response network packet is located after at least one first response network packet in the first queue.

[0067] To further illustrate, such as Figure 4 As shown, the second queue includes a first request network packet 402 located at the header position, at least one second request network packet 404 located at the first position, and at least one response network packet 406 located at the second position, wherein the first position is adjacent to and follows the header position, and the second position is adjacent to and follows the first position.

[0068] It should be noted that, but not limited to, at least one response network packet corresponding to each of the multiple first queues can be stored in a third queue, and a mapping relationship can be established between each first queue and the corresponding at least one response network packet.

[0069] The embodiments provided in this application determine the position of at least one response network packet in a first queue as a second position of the first queue, wherein the second position is located after the first position; at least one response network packet is placed in the first queue according to the second position to obtain a second queue; the second queue is stored in a local database. By encapsulating multiple request network packets and at least one response network packet associated with each historical request in the form of a second queue, the purpose of encapsulating request data and response data in the traffic data associated with historical requests is achieved, thereby realizing the technical effect of improving the processing efficiency of network packet data.

[0070] As an alternative approach, before retrieving the first request network packet associated with the historical request from multiple candidate network packets, the method further includes:

[0071] S1, upon obtaining the recording information, perform network packet recording operations involving multiple network cards, multiple protocols, and multiple ports to obtain all recorded network packets, including multiple candidate network packets;

[0072] All recorded network packets are stored in a cloud database.

[0073] Optionally, in this embodiment, the recording information may be initiated through one or more of the following methods: active triggering, timed triggering, external scheduling triggering, or other triggering methods.

[0074] Optionally, in this embodiment, the network packet recording operation can be performed by using a target command line, and all recorded network packets are stored in the corresponding PCAP file and then stored in the cloud database.

[0075] It should be noted that, considering the cloud host where the online service is located, it is necessary to avoid affecting online business and to adhere to the principle of minimizing resource consumption. Therefore, the command-line operation mode is adopted. After the recorded PCAP file is stored according to the specified data packet size, it is promptly synchronized to the cloud database and the local disk space is released.

[0076] To further illustrate, here is an optional command line:

[0077] sudo tcpdump"host 10.10.10.100and tcp and(port 8888or port 9999)"-wtest.pcap

[0078] In the command line, “host 10.10.10.100” is one of the aforementioned network cards, “tcp” is one of the aforementioned protocols, “port 8888 or port 9999” is one of the aforementioned ports, and “test.pcap” is a PCAP file used to store traffic data.

[0079] Through the embodiments provided in this application, when recording information is obtained, network packet recording operations are performed using multiple network cards, multiple protocols, and multiple ports to obtain all recorded network packets, wherein all recorded network packets include multiple candidate network packets; and all recorded network packets are stored in a cloud database.

[0080] As an alternative approach, after storing all recorded network packets in a cloud database, the method also includes:

[0081] S1. Using the configuration information of the target environment, determine multiple candidate network packets of the target type from all recorded network packets. The target type includes: target network card type, target protocol type, and target port type.

[0082] Optionally, in this embodiment, the configuration information of the target environment may be used, but is not limited to, to determine multiple candidate network packets of the target type from all recorded network packets containing multiple types. The target type may include, but is not limited to, the target network card type, the target protocol type, and the target port type.

[0083] Optionally, in this embodiment, the configuration information of the target environment can be determined based on, but is not limited to, the communication quintuple information.

[0084] It should be noted that multiple candidate network packets of the target type selected using the configuration information of the target environment can be used, but are not limited to, for functional testing of the target environment.

[0085] It should be noted that functional testing utilizes targeted network packet data adapted to the target environment, without processing all network packet data, thereby improving data processing efficiency during the functional testing process.

[0086] The embodiments provided in this application utilize the configuration information of the target environment to determine multiple candidate network packets of a target type from all recorded network packets. The target type includes: target network interface card type, target protocol type, and target port type. For all recorded online traffic data, candidate data within a specific gateway range, protocol range, or port range is initially filtered using the target environment's configuration information. Furthermore, complete network packets associated with historical requests are obtained, thereby achieving the technical effect of efficient storage of traffic data from different gateways, protocols, and ports.

[0087] As an optional approach, after storing the first request network packet, at least one second request network packet, and at least one response network packet, the method further includes:

[0088] S1, retrieve multiple target queues associated with multiple historical requests from the local database;

[0089] S2 utilizes multiple target queues to perform functional testing on the target environment.

[0090] Optionally, in this embodiment, the target queue may be, but is not limited to, the corresponding second queue.

[0091] Optionally, in this embodiment, the local database may, but is not limited to, store multiple second queues, wherein each second queue is associated with a corresponding historical request, and each second queue includes multiple request network packets associated with each historical request, and at least one response network packet.

[0092] It should be noted that the complete request network packets and response network packets associated with multiple historical requests indicated by multiple target queues are used to perform functional testing on the target environment. The target environment may be, but is not limited to, the first environment (which is equivalent to repeating the test of the first environment), or it may be, but is not limited to, a second environment that is different from the first environment (which is equivalent to testing a new environment).

[0093] It should be noted that, for the second environment, which is different from the first environment, the ordered network data packets stored for the first environment are directly used to test the second environment, thereby achieving the purpose of multi-environment reuse (restoration test) of traffic data.

[0094] It should be noted that the first environment and the second environment mentioned above may refer to different external operating environments of a product, and may also refer to different internal test versions of a product, but this embodiment does not limit this.

[0095] The embodiments provided in this application retrieve multiple target queues associated with multiple historical requests from a local database; these target queues are then used to perform functional testing on the target environment. The complete traffic data stored persistently can, but is not limited to, be used for data replay in subsequent services, thereby achieving the purpose of using real user traffic data for regression testing and performance stress testing in various version iterations.

[0096] As an optional approach, the above network packet processing method can be applied to a multi-protocol parsing scenario for online traffic to encode and decode multiple protocols in TCP traffic and persistently store the parsed traffic data for use in subsequent test scenarios. The multiple protocols mentioned above may include, but are not limited to, HTTP, Dubbo, and Thrift protocols.

[0097] To further illustrate, a data storage system based on the aforementioned network packet processing method, applied to multi-protocol parsing scenarios of online traffic, is as follows: Figure 5 As shown, the data storage system supports encoding and decoding of HTTP / Thrift / Dubbo protocols in network protocols, and persistently stores the parsing results of different protocols for use by other services. The data storage system includes a central service module 502, a recording service module 504, an OSS service module 506, a Handle service module 508, and a storage service module 510. The specific contents of each of the above service modules are as follows:

[0098] (I) Central Service Module 502

[0099] The central service module 502 is responsible for scheduling the execution of recording service tasks.

[0100] Considering the lengthy process flow, it was broken down into microservices. The central service module 502 centrally schedules and controls the entire process, providing feedback on the flow of process status and the status of any abnormal situations at each stage. The central service is the "brain" of the entire system.

[0101] (II) Recording Service Module 504

[0102] The recording service module 504 is responsible for recording traffic in the corresponding online environment according to the specified recording information. This is achieved by using the tcpdump command on the target machine to monitor traffic, and the recorded data files are stored in the OSS service module 506.

[0103] Considering that online traffic includes data from multiple gateways, protocols, and ports, the system can initially filter candidate data within a specific gateway range, protocol range, or port range by configuring all recorded online traffic data. This enables efficient storage of traffic data from different gateways, protocols, and ports.

[0104] To illustrate further, the tcpdump monitoring command can be used to implement this. By writing filters based on the port number and protocol type of the service being tested, the recording information can be specified, thereby accurately locating the range of business protocol traffic.

[0105] For example, if a service under test supports multiple protocols, and each protocol corresponds to specific port information, then each protocol will be monitored separately and its corresponding pcap file will be stored.

[0106] Command line reference: sudo tcpdump"host 10.10.10.100and tcp and(port 8888orport9999)"-w test.pcap.

[0107] In the command line, “host 10.10.10.100” corresponds to the gateway filtering configuration information, “tcp” corresponds to the protocol filtering configuration information, “port 8888 or port 9999” corresponds to the port filtering configuration information, and “test.pcap” is the PCAP file used to store traffic data.

[0108] (III) OSS Service Module 506

[0109] OSS service module 506 is responsible for storing the recorded data files in the cloud.

[0110] Considering that the online service is hosted on a cloud server and cannot affect online business, based on the principle of minimizing resource consumption, the command-line operation method is adopted. After storing the monitored pcap file according to the specified data packet size, it is promptly synchronized to the OSS system and the local disk space is released.

[0111] (iv) Handle service module 508

[0112] The Handle service module 508 is responsible for downloading, cleaning, and decoding the recorded data. As the core service of the system, it includes core decoding logic and cleaning logic (request and response pairing) for multiple protocols. The following text uses the HTTP protocol as an example for illustration.

[0113] 1. Core decoding logic:

[0114] The protocol encoding and decoding implementation includes parsing the fields according to the protocol, and it is necessary to handle the parsing of various abnormal data during the process.

[0115] It should be noted that the captured packets are individual network packets. The TCP layer protocol can be parsed using the pcap4j open-source package to obtain the corresponding IPv4 packet, TCP packet, and data packet. Then, the data packet can be parsed according to different protocols.

[0116] This section focuses on describing the process of packet merging. For different protocols, this is achieved by parsing the request packets (different protocols determine the protocol type differently; please refer to the protocol specification). For example, the HTTP protocol parses packets starting with GET, POST, or PUT, then obtains the corresponding TCP sequence number and acknowledgment number. Packets with the same acknowledgment number in the request packets are placed in a list. When a FIN packet is received, the data in this list is parsed, and the data portion is merged, thus achieving data merging of multiple packets and ensuring that no data is lost.

[0117] It should be noted that the implementation of packet merging technology varies depending on the protocol.

[0118] 2. Data cleaning logic:

[0119] For packets captured at the transport layer, data parsing and cleaning operations need to be performed, namely, the pairing logic of requests and responses.

[0120] The solution involves iterating through the Elasticsearch storage data of the recorded task, calculating the acknowledgment number for messages with request_type 1 (request), referring to formula (1), retrieving the matching response packet based on the calculated acknowledgment number, and updating the request_id of the response packet to the request_id of the request packet so that the corresponding response packet can be directly queried based on the request_id of the request packet in subsequent use, thus completing the data matching.

[0121] Acknowledgment number = sequence number from the last received message + len (data length) (1)

[0122] If a SYN or FIN packet is received, the sequence number is changed to the sequence number of the previously received packet plus 1. The scheme pre-filters packets in special cases.

[0123] (V) Storage Service Module 510

[0124] The storage service module 510 is responsible for storing the parsed traffic data by protocol and group. Considering the large amount of data, Elasticsearch (ES) storage is used. This service is responsible for the interface implementation of ES storage and retrieval related businesses.

[0125] Since the amount of data recorded is enormous, and the amount of data continues to grow as the recording time increases, the biggest problem we face is parsing and storage performance.

[0126] The solution employs concurrency optimization methods such as thread pools and batch storage. Specifically addressing the requirement that Elasticsearch batch storage requires a single data submission of less than 100MB, the code was optimized from using concurrent record counts to using data size as the basis for submission. That is, even when the data packet size is unknown, the size of the data to be submitted is calculated in real-time, and submission is initiated only when the size exceeds 80MB, thus circumventing the single request size limit of Elasticsearch.

[0127] It should be noted that the recording task can be triggered in the following ways: active triggering, timed triggering, and external scheduling triggering.

[0128] It should be noted that the data storage system based on the above network packet processing method, applied to multi-protocol parsing scenarios of online traffic, has the following beneficial effects:

[0129] (1) Supports encoding and decoding of multiple service protocols. For online traffic data, it can perform encoding and decoding operations according to different protocols to complete the plaintext parsing of business traffic data;

[0130] (2) Supports packet splicing for various protocols. When network communication traffic is transmitted at the transport layer, it will face the problem of packet splitting and splicing when making requests and responses for large amounts of data. This embodiment adapts the packet splicing function logic to all supported protocols. The implementation method adopts the communication, sequence number and acknowledgment number in the TCP package header information.

[0131] (3) The extension protocol is convenient. In this embodiment, the main process code logic design for multi-protocol support has been completed. When extending new protocols in the future, only the protocol's own code and decode functions need to be implemented.

[0132] (4) Traffic data application: After online traffic data is decoded, analyzed and stored in the database, the service under test can be deployed. The traffic data can be replayed and reused in multiple environments, truly achieving the use of real traffic for testing and reuse. It can be used for different scenarios such as functional testing, problem reproduction or performance stress testing.

[0133] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0134] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0135] According to another aspect of the embodiments of this application, a network packet processing apparatus for implementing the above-described network packet processing method is also provided. For example... Figure 6 As shown, the device includes:

[0136] The acquisition unit 602 is used to acquire a first request network packet associated with a historical request from a plurality of candidate network packets, wherein the historical request is associated with a plurality of request network packets and at least one response network packet, and the plurality of request network packets include the first request network packet;

[0137] The first determining unit 604 is used to determine at least one second request network packet from a plurality of candidate network packets that matches the request acknowledgment number of the first request network packet;

[0138] The second determining unit 606 is used to determine a response confirmation number based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of at least one second request network packet, and to obtain the network packet corresponding to the response confirmation number, wherein at least one response network packet includes the network packet corresponding to the response confirmation number.

[0139] Storage unit 608 is used to store a first request network packet, at least one second request network packet, and at least one response network packet.

[0140] As an optional solution, the above-mentioned device further includes:

[0141] The first determining module is configured to, after determining at least one second request network packet that matches the request acknowledgment number of the first request network packet from a plurality of candidate network packets, determine the position of the first request network packet in the initial queue as the head position of the initial queue, and a first position after determining the at least one second request network packet as the head position;

[0142] The first placement module is configured to, after determining at least one second request network packet from a plurality of candidate network packets that matches the request acknowledgment number of the first request network packet, place the first request network packet in an initial queue according to its header position, and place at least one second request network packet in the initial queue according to a first position, thereby obtaining a first queue.

[0143] As an optional solution, the second determining unit 606 mentioned above includes:

[0144] The second determining module is used to determine the sum of the first data length and the second data length as the third data length of the first queue;

[0145] The third determination module is used to determine the sum of the sequence number and the third data length as the response confirmation number.

[0146] As an optional solution, the aforementioned storage unit 608 includes:

[0147] The fourth determining module is used to determine the position of at least one response network packet in the first queue as the second position of the first queue, wherein the second position is located after the first position;

[0148] The second placement module is used to place at least one response network packet into the first queue according to the second position, thereby obtaining the second queue;

[0149] The first storage module is used to store the second queue in the local database.

[0150] As an optional solution, the above-mentioned device further includes:

[0151] The recording module is used to perform network packet recording operations on multiple network cards, multiple protocols, and multiple ports before obtaining the first request network packet associated with the historical request from multiple candidate network packets, and to obtain all recorded network packets, wherein the all recorded network packets include multiple candidate network packets.

[0152] The second storage module is used to store all recorded network packets in a cloud database before retrieving the first request network packet associated with the historical request from multiple candidate network packets.

[0153] As an optional solution, the above-mentioned device further includes:

[0154] The fifth determination module is used to determine multiple candidate network packets of the target type from all recorded network packets after storing all recorded network packets in the cloud database, using the configuration information of the target environment. The target type includes: target network card type, target protocol type, and target port type.

[0155] As an optional solution, the device also includes:

[0156] The acquisition module is used to acquire multiple target queues associated with multiple historical requests from a local database after storing a first request network packet, at least one second request network packet, and at least one response network packet.

[0157] The testing module is used to perform functional testing on the target environment using multiple target queues after storing the first request network packet, at least one second request network packet, and at least one response network packet.

[0158] For specific implementation examples, please refer to the examples shown in the above network packet processing method; these examples will not be repeated here.

[0159] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described network packet processing method is also provided, such as... Figure 7 As shown, the electronic device includes a memory 702 and a processor 704. The memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments via the computer program.

[0160] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0161] Optionally, in this embodiment, the processor can be configured to execute the steps in the network packet processing method via a computer program.

[0162] Alternatively, as those skilled in the art will understand, Figure 7 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 7 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 7 The different configurations shown.

[0163] The memory 702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the network packet processing method and apparatus in this embodiment. The processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, thereby implementing the aforementioned network packet processing method. The memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include memory remotely located relative to the processor 704, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 702 may be used, but is not limited to, to store information such as a first data length and a second data length. As an example, such as... Figure 7 As shown, the memory 702 may include, but is not limited to, the acquisition unit 602, the first determination unit 604, the second determination unit 606, and the storage unit 608 in the network packet processing device. Furthermore, it may include, but is not limited to, other module units in the network packet processing device, which will not be elaborated upon in this example.

[0164] Optionally, the transmission device 706 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 706 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 706 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0165] In addition, the above-mentioned electronic device also includes: a display 708 for displaying information such as the first data length and the second data length; and a connection bus 710 for connecting the various module components in the above-mentioned electronic device.

[0166] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions provided in embodiments of this application.

[0167] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0168] It should be noted that the computer system of the electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0169] A computer system includes a Central Processing Unit (CPU), which performs various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) or loaded from RAM. ROM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output interfaces (I / O interfaces) are also connected to the bus.

[0170] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs various functions defined in the system of this application.

[0171] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of a computer device reads computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the network packet processing method provided in the various optional implementations described above.

[0172] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the steps in the network packet processing method described above.

[0173] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0174] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0175] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0176] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A network packet processing method, characterized in that, include: Obtain a first request network packet associated with a historical request initiated in a first environment from multiple candidate network packets, wherein the historical request is associated with multiple request network packets and at least one response network packet, and the multiple request network packets include the first request network packet; From the plurality of candidate network packets, at least one second request network packet is determined that matches the request acknowledgment number of the first request network packet; Based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of the at least one second request network packet, a response confirmation number is determined, and the network packet corresponding to the response confirmation number is obtained, wherein the at least one response network packet includes the network packet corresponding to the response confirmation number; The first request network packet and the at least one second request network packet are stored as a first whole unit, and the at least one response network packet is stored as a second whole unit, and a mapping relationship is established between the first whole unit and the second whole unit. Using the first overall unit and the second overall unit, a restoration test is performed in a second environment, wherein the first environment and the second environment are different external operating environments or different internal test versions.

2. The method according to claim 1, characterized in that, After determining at least one second request network packet from the plurality of candidate network packets that matches the request acknowledgment number of the first request network packet, the method further includes: The position of the first request network packet in the initial queue is determined as the head position of the initial queue, and the at least one second request network packet is determined as the first position after the head position; The first request network packet is placed in the initial queue according to the header position, and the at least one second request network packet is placed in the initial queue according to the first position to obtain the first queue.

3. The method according to claim 2, characterized in that, The step of determining the response acknowledgment number based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of the at least one second request network packet includes: The sum of the first data length and the second data length is determined as the third data length of the first queue; The sum of the sequence number and the third data length is determined as the response confirmation number.

4. The method according to claim 2, characterized in that, After determining the response acknowledgment number based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of the at least one second request network packet, and obtaining the network packet corresponding to the response acknowledgment number, the method further includes: The position of the at least one response network packet in the first queue is determined as the second position of the first queue, wherein the second position is located after the first position; According to the second position, the at least one response network packet is placed in the first queue to obtain the second queue; Store the second queue in the local database.

5. The method according to any one of claims 1-4, characterized in that, Before retrieving the first request network packet associated with the historical request from multiple candidate network packets, the method further includes: Once the recording information is obtained, a network packet recording operation is performed using multiple network cards, multiple protocols, and multiple ports to obtain all recorded network packets, wherein the all recorded network packets include the multiple candidate network packets; All recorded network packets are stored in a cloud database.

6. The method according to claim 5, characterized in that, After storing all the recorded network packets in a cloud database, the method further includes: Using the configuration information of the target environment, a plurality of candidate network packets of the target type are determined from all recorded network packets, wherein the target type includes: target network card type, target protocol type, and target port type.

7. The method according to any one of claims 1 to 4, characterized in that, After storing the first request network packet and the at least one second request network packet as a first whole unit, and storing the at least one response network packet as a second whole unit, the method further includes: Retrieve multiple target queues associated with multiple historical requests from the local database; The target environment is functionally tested using the multiple target queues.

8. A network packet processing apparatus, characterized in that, include: The acquisition unit is configured to acquire a first request network packet associated with a historical request initiated in a first environment from a plurality of candidate network packets, wherein the historical request is associated with a plurality of request network packets and at least one response network packet, and the plurality of request network packets include the first request network packet; The first determining unit is configured to determine, from the plurality of candidate network packets, at least one second request network packet that matches the request acknowledgment number of the first request network packet; The second determining unit is configured to determine a response confirmation number based on the sequence number of the first request network packet, the first data length of the first request network packet, and the second data length of the at least one second request network packet, and to obtain the network packet corresponding to the response confirmation number, wherein the at least one response network packet includes the network packet corresponding to the response confirmation number; The apparatus is further configured to store the first request network packet and the at least one second request network packet as a first overall unit, and store the at least one response network packet as a second overall unit, and establish a mapping relationship between the first overall unit and the second overall unit; and use the first overall unit and the second overall unit to perform a restoration test in a second environment, wherein the first environment and the second environment are different external operating environments or different internal test versions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images