Message processing method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the present application provide a message processing method and device, electronic equipment and readable storage medium, comprising: obtaining a to-be-processed message and a first quantity of the to-be-processed message, dividing the to-be-processed message according to a preset value and the first quantity of the to-be-processed message, obtaining a plurality of division regions about the to-be-processed message, adding a tracking identifier to the to-be-processed message in any division region, determining a preset cache space corresponding to the to-be-processed message according to the tracking identifier, obtaining a processing result of the to-be-processed message of the division region by a node pair in a preset node graph for any division region, and recording the processing result to the preset cache space corresponding to the to-be-processed message, in the case of determining that the processing is completed, clearing the processing result in the preset cache space after the processing result in the preset cache space corresponding to the to-be-processed message is imported into a preset log library, thereby reducing the occupied storage space.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a message processing method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] The DPDK (Data Plane Development Kit) Node Graph framework is inspired by the open-source technology VPP (Vector Packet Processing). This framework adopts a vector packet processing architecture, dividing the packet processing flow into independent processing nodes. Each node processes a batch of packets, which then sequentially flow to the next node for further batch processing. DPDK's Node Trace function provides relevant information about packet processing. This design effectively solves the instruction cache miss problem caused by complex changes in traditional pipelines, significantly improving system throughput.

[0003] However, in scenarios with high traffic, DPDK's Node Trace function requires the tracing library to collect a large amount of data, which consumes a lot of storage space and leads to a decrease in system performance. Summary of the Invention

[0004] This invention provides a message processing method, apparatus, electronic device, and readable storage medium to solve the problem that in scenarios with high traffic, the existing DPDK Node Trace function requires the tracing library to collect a large amount of data, which consumes a lot of storage space and leads to a decrease in system performance.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0006] In a first aspect, the present invention provides a message processing method, the method comprising:

[0007] Obtain the messages to be processed and a first number of the messages to be processed;

[0008] According to a preset value and a first number of messages to be processed, the messages to be processed are divided to obtain several division regions for the messages to be processed.

[0009] For any of the defined regions, add a tracking identifier to the message to be processed within the defined region;

[0010] The preset cache space corresponding to the message to be processed is determined based on the tracking identifier;

[0011] For any of the aforementioned partitioned regions, the processing results of the nodes in the preset node graph for the message to be processed in the partitioned region are obtained, and the processing results are recorded in the preset cache space corresponding to the message to be processed.

[0012] Once the processing is complete, the processing results in the preset cache space corresponding to the message to be processed are imported into the preset log library, and then the processing results in the preset cache space are cleared.

[0013] Optionally, adding a tracking identifier to the unprocessed message within any of the defined regions includes:

[0014] For any of the defined regions, determine a second number of packets to be processed within the defined region;

[0015] Obtain the target integer value from 0 to the second quantity minus one;

[0016] The target integer value is used as a tracking identifier and added to the message to be processed in the divided region one by one.

[0017] Optionally, before determining the preset buffer space corresponding to the message to be processed based on the tracking identifier, the method further includes:

[0018] Obtain the identity identifier of the preset node graph;

[0019] A preset number of preset cache spaces are set for the preset node graph, and the preset cache spaces carry the identity identifier of the preset node graph;

[0020] Sort the preset cache space and obtain the sorting position value of the preset cache space;

[0021] Based on the sorting position value, different position identifiers are added to different preset cache spaces.

[0022] Optionally, determining the preset buffer space corresponding to the message to be processed based on the tracking identifier includes:

[0023] The tracking identifier is moduloed according to the preset quantity to obtain the modulo value;

[0024] The location identifier is searched based on the modulo operation value;

[0025] If a target location identifier that matches the modulo operation value is found from the location identifiers, then the target preset cache space corresponding to the target location identifier is obtained;

[0026] The target preset cache space is determined as the preset cache space corresponding to the message to be processed corresponding to the tracking identifier.

[0027] Optionally, before obtaining the processing results of the nodes within the preset node graph for the message to be processed in the divided region, the method further includes:

[0028] Obtain the tracking policy set by the user for the message to be processed;

[0029] According to the tracking strategy, a target valid node is determined from the nodes in the preset node graph, wherein the target valid node is used to obtain the processing result of the message to be processed in the divided region.

[0030] Optionally, obtaining the tracking policy set by the user for the message to be processed further includes:

[0031] The tracking strategy is pre-configured with a parameter adjustment page, which includes at least one of the following parameters to be adjusted: network communication protocol, source Internet Protocol address, destination Internet Protocol address, source port, destination port, target node identifier that allows acquisition of message content, and dropped packet tracking switch.

[0032] In response to the user's settings for the parameters to be adjusted on the parameter adjustment page, a tracking strategy for the message to be processed is generated.

[0033] Optionally, the step of obtaining the processing results of the nodes in the preset node graph for the unprocessed message in any of the partitioned regions, and recording the processing results in the preset cache space corresponding to the unprocessed message, further includes:

[0034] For any of the aforementioned partitioned regions, if it is determined that any node in the preset node graph processes the message to be processed in the partitioned region, the node identifier of the node is obtained.

[0035] The node identifier is used as the processing result of the message to be processed;

[0036] The processing result is recorded in the preset cache space corresponding to the message to be processed.

[0037] In a second aspect, the present invention provides a message processing apparatus, the apparatus comprising:

[0038] The first acquisition module is used to acquire the message to be processed and the first number of the message to be processed;

[0039] The segmentation module is used to segment the messages to be processed according to a preset value and a first number of messages to be processed, so as to obtain several segmentation regions for the messages to be processed.

[0040] The tracking identifier adding module is used to add a tracking identifier to the message to be processed within any of the defined regions.

[0041] The first determining module is used to determine the preset cache space corresponding to the message to be processed based on the tracking identifier;

[0042] The recording module is used to obtain the processing results of the nodes in the preset node graph for the message to be processed in any of the partitioned regions, and record the processing results in the preset cache space corresponding to the message to be processed.

[0043] The clearing module is used to import the processing results in the preset cache space corresponding to the message to be processed into the preset log library after determining that the processing is completed, and then clear the processing results in the preset cache space.

[0044] Thirdly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described message processing method.

[0045] Fourthly, the present invention provides a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described message processing method.

[0046] In this embodiment of the invention, a number of messages to be processed and a first number of messages to be processed are obtained. The messages to be processed are divided according to a preset value and the first number of messages to be processed, resulting in several regions for each region. For any region, a tracking identifier is added to the messages to be processed within that region. This facilitates subsequent determination of the preset cache space containing the cached information of the messages to be processed based on the tracking identifier. For any region, the processing results of the nodes in a preset node graph for the messages to be processed within that region are obtained, and the processing results are recorded in the preset cache space corresponding to the messages to be processed. When processing is complete, the processing results in the preset cache space corresponding to the messages to be processed are imported into a preset log database, and then the processing results in the preset cache space are cleared. This allows messages to be processed in a new region to continue using the preset cache space, reducing storage space usage. This application limits the number of messages processed at one time by batch processing the messages to be processed. Furthermore, different batches of messages to be processed share the same preset cache space, greatly reducing storage space and improving system performance. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of the steps of a message processing method provided in an embodiment of the present invention;

[0049] Figure 2 yes Figure 1 The flowchart shown is a step 103 of a message processing method provided in an embodiment of the present invention;

[0050] Figure 3 yes Figure 1 The flowchart shown is a step 104 of a message processing method provided in an embodiment of the present invention;

[0051] Figure 4 yes Figure 1 The flowchart shown is a step 105 of a message processing method provided in an embodiment of the present invention;

[0052] Figure 5 This is a structural diagram of a message processing device provided in an embodiment of the present invention;

[0053] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] DPDK (Data Plane Development Kit) is an open-source library and driver collection designed to accelerate packet processing operations. DPDK is primarily used in applications such as Network Functions Virtualization (NFV), Software-Defined Networking (SDN), high-performance routing, and firewalls. A DPDK node graph typically refers to the logical structure of the packet processing flow within a DPDK application. This graph shows the flow path of packets between different processing nodes, with each node representing a logical unit or stage of packet processing. Each node is responsible for specific packet processing tasks, such as receiving, parsing, forwarding, modifying, and sending packets. In DPDK, node tracing is a debugging and performance analysis tool used to trace the flow path of packets in the node graph. Through node tracing, developers can observe the transmission of packets between nodes, identify potential performance bottlenecks, and optimize accordingly. However, existing node tracing methods, when used in high-traffic scenarios, require significant space due to the large amount of data being traced, impacting system performance. Furthermore, when customizing packet tracing, the tracing conditions must be set in the underlying code, and code modifications can only be made by the developer, resulting in limited flexibility. To address the aforementioned problems, this invention proposes a message processing method.

[0056] Reference Figure 1 , Figure 1 This is a flowchart of a message processing method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method may include:

[0057] Step 101: Obtain the messages to be processed and the first number of messages to be processed.

[0058] In this embodiment of the invention, the messages to be processed can also be referred to as data packets to be processed. In scenarios with high traffic, there will be a large number of messages to be processed. If all of them were processed in a preset node graph, the large amount of data to be tracked would require a significant amount of space, impacting system performance. Therefore, in this embodiment of the invention, a first number of messages to be processed is obtained, and then they are divided for batch processing.

[0059] Step 102: According to the preset value and the first number of messages to be processed, divide the messages to be processed to obtain several division regions for the messages to be processed.

[0060] The preset values ​​in this embodiment can be set based on the maximum data capacity of the preset node graph, or they can be set according to a standard that allows the system performance to be at its best. This invention does not impose specific limitations here. When dividing the packets to be processed according to the preset value and the first number of packets to be processed, the first number of packets to be processed must first be greater than the preset value. If it is less than the preset value, it indicates that there are fewer packets to be processed, and in this case, batch processing is not required, and it will not affect system performance. When it is determined that the first number of packets to be processed is greater than the preset value, the packets to be processed can be divided according to the preset value.

[0061] For example, if the preset value is 32 and there are 100 messages to be processed, then 100 ÷ 32 = 3...4, so we can get 4 division regions. Among them, there are 32 messages to be processed in 3 regions and 4 messages to be processed in 1 region.

[0062] Step 103: For any segmented region, add a tracking identifier to the messages to be processed within the segmented region.

[0063] Multiple partitioned regions are obtained through the above steps. When processing pending messages using the preset node graph, messages within one partitioned region are processed at a time. Each pending message needs a trace ID to facilitate locating its corresponding position in the pre-allocated cache. When setting the trace ID, since pending messages are processed in batches, different batches will not interfere with each other. Therefore, the trace IDs of pending messages in different partitioned regions can be set consistently, while the trace IDs of pending messages in the same batch (same partitioned region) can be set starting from 0 and incrementing sequentially. Alternatively, for differentiation, an ID can be set for each partitioned region, and then a trace ID and partitioned region ID can be added to each pending message to uniquely identify the corresponding pending message.

[0064] Further, in step 103, as... Figure 2 As shown:

[0065] Step 1031: For any partitioned region, determine the second number of messages to be processed within the partitioned region.

[0066] Step 1032: Obtain the target integer value from 0 to the second quantity minus one.

[0067] Step 1033: Use the target integer value as a tracking identifier and add it one by one to the unprocessed messages in the divided area.

[0068] For example, if there are pending messages A, B, C, D, E, and F in partition region a, it can be seen that there are 6 pending messages in partition region A, with target integer values ​​of 0, 1, 2, 3, 4, and 5. Adding these target integer values ​​to the pending messages one by one results in pending messages A0, B1, C2, D3, E4, and F5. If a partition region identifier 'a' is set for partition region a, then the pending messages after adding the identifier are Aa0, Ba1, Ca2, Da3, Ea4, and Fa5.

[0069] The above steps assign a unique tracking identifier to each message to be processed within the partitioned area, which facilitates finding its corresponding position in the pre-allocated cache based on the tracking identifier. At the same time, the message to be processed can also be tracked and processed based on this tracking identifier.

[0070] Step 104: Determine the preset buffer space corresponding to the message to be processed based on the tracking identifier.

[0071] To ensure that each pending message has its own cache space, the number of preset cache spaces in this embodiment of the invention is set according to a preset value. Furthermore, a certain number of preset cache spaces are set for each preset node graph, and the identity of the preset node graph is associated with each cache space. Then, the preset cache spaces are sorted, and the sorting position value of each cache space is obtained. The sorting can be based on some rule, such as the size of the cache space, the order of creation time, etc., which is not specifically limited in this invention. Afterwards, a unique position identifier is added to each cache space according to the sorting position value. The position identifier can be a unique identifier based on the sorting position value. The specific steps include:

[0072] Obtain the identity identifier of the preset node graph;

[0073] Set a preset number of preset cache spaces for the preset node graph, and the preset cache spaces carry the identity identifier of the preset node graph;

[0074] Sort the preset cache space and obtain the sort position value of the preset cache space;

[0075] Based on the sort position value, add different position identifiers to different preset cache spaces.

[0076] For example, there is a preset node graph with the identifier "graph_123". Ten cache spaces are set up for the preset node graph, and each cache space carries the identifier "graph_123" of the preset node graph. After sorting by creation time, the sort position values ​​are: Cache space 1: sort position value = 0, Cache space 2: sort position value = 1, Cache space 3: sort position value = 2...Cache space 10: sort position value = 9. Therefore, the position identifier of cache space 1 is 0, the position identifier of cache space 2 is 1, the position identifier of cache space 3 is 2...the position identifier of cache space 10 is 9.

[0077] These steps establish a preset cache space for the preset node graph and assign a unique location identifier to each preset cache space, facilitating subsequent management and use.

[0078] Furthermore, after setting a tracking identifier for each segmented region's pending message and a location identifier for the preset buffer space, values ​​are obtained from the tracking identifier and the location identifier. Then, based on the quantity value of the location identifier, the value of the tracking identifier is moduloed. The resulting modulo operation value is compared with the value obtained from the location identifier. If the comparison matches, the preset buffer space corresponding to the location identifier at the time of comparison is allocated to the pending message corresponding to the tracking identifier at the time of comparison. Step 104, as follows. Figure 3 As shown:

[0079] Step 1041: Perform a modulo operation on the tracking markers according to the preset quantity to obtain the modulo operation value.

[0080] Step 1042: Search the location identifier based on the modulo operation value.

[0081] Step 1043: If a target location identifier that matches the modulo operation value is found from the location identifier, then the target preset cache space corresponding to the target location identifier is obtained.

[0082] Step 1044: The target preset cache space is determined as the preset cache space corresponding to the message to be processed corresponding to the tracking identifier.

[0083] Here, the preset quantity refers to the number of preset cache spaces. For example, if the preset quantity is set to 32, and the tracking identifier values ​​are 01, 02, 05, and 08, the modulo operation on the tracking identifier values ​​is: 1 mod 32 = 1, 2 mod 32 = 2, 5 mod 32 = 5, and 8 mod 32 = 8. Therefore, tracking identifier 01 corresponds to preset cache space 01, tracking identifier 02 corresponds to preset cache space 02, tracking identifier 05 corresponds to preset cache space 05, and tracking identifier 08 corresponds to preset cache space 08. It should be noted that if the preset value set for the divided region is less than or equal to the preset quantity set for the preset cache space, then the corresponding position identifier can be directly determined based on the tracking identifier, and thus the corresponding preset cache space can be determined without performing a modulo operation.

[0084] The above steps allocate a corresponding preset cache space for each message to be processed, which facilitates the subsequent querying and management of the cache information of the messages to be processed.

[0085] Step 105: For any partitioned region, obtain the processing results of the nodes in the preset node graph for the messages to be processed in the partitioned region, and record the processing results in the preset cache space corresponding to the messages to be processed.

[0086] In this embodiment of the invention, after allocating a preset buffer space for each message to be processed within any partitioned region, the messages can be processed through multiple nodes in the preset node graph. However, before this, messages can be filtered using a user-defined strategy, and only messages that meet the strategy will be tracked. The specific steps include:

[0087] Retrieve the tracking policy set by the user for the pending message;

[0088] According to the tracking strategy, a target valid node is determined from the nodes in the preset node graph, wherein the target valid node is used to obtain the processing result of the message to be processed in the divided region.

[0089] It should be noted that when users set tracking policies, these policies can include various parameters, such as network communication protocol, source IP address, destination IP address, source port, destination port, target node identifiers allowed to obtain packet content, and a switch for tracking dropped packets. For example, suppose the user sets the following tracking policy: Network communication protocol: TCP, Source IP address: 192.168.1.10, Destination IP address: 10.0.0.5, Source port: 80, Destination port: 443, Target node identifiers allowed to obtain packet content: node_123, Dropped packet tracking switch: Enabled. Based on these settings, the target valid nodes are determined from the preset node graph; these are the nodes that meet the requirements of the tracking policy. Assume the following nodes are present in the preset node graph: Node 1: handles TCP protocol, Node 2: handles source IP address 192.168.1.10, Node 3: handles destination IP address 10.0.0.5, Node 4: handles source port 80, Node 5: handles destination port 443, Node 6: allows retrieval of packet content, and Node 7: handles dropped packets. Based on the user-defined tracing policy, the identified target valid nodes are: Node 1 (handles TCP protocol), Node 2 (handles source IP address 192.168.1.10), Node 3 (handles destination IP address 10.0.0.5), Node 4 (handles source port 80), Node 5 (handles destination port 443), Node 6 (allows retrieval of packet content), and Node 7 (handles dropped packets). The packets to be processed in the divided area are processed through these target valid nodes, and the cached information of the packets to be processed in these valid nodes is obtained. Other nodes do not record the cached processing results during processing.

[0090] Furthermore, in this embodiment of the invention, the setting of the tracking strategy does not require code modification; the parameter adjustment interface can be directly accessed via command line, improving the flexibility of user-defined strategies (rules). Specific steps include:

[0091] The pre-set tracking strategy includes a parameter adjustment page, which includes at least one of the following parameters to be adjusted: network communication protocol, source Internet Protocol address, destination Internet Protocol address, source port, destination port, target node identifier that allows acquisition of message content, and dropped packet tracking switch.

[0092] In response to the user's settings for the parameters to be adjusted on the parameter adjustment page, a tracking strategy for the pending messages is generated.

[0093] It should be noted that the network communication protocols on the parameter adjustment page include ARP, ICMP, TCP, DUP, etc., which play key roles in different network and transport layers. The source and destination Internet Protocol addresses can be specified in combination, as can the source and destination ports. Specific IP and port values ​​are not specifically limited in this invention. Because the original preset node graph does not capture dropped data packets, this embodiment of the invention allows for the acquisition of packet loss information within the preset node graph when a dropped data packet tracking switch is enabled.

[0094] In addition, parameters can specify whether the match is for the underlay or overlay portion. The underlay network refers to the physical network infrastructure, including physical switches, routers, fiber optic cables, and other cables. It is the foundational layer of the network, responsible for transmitting data packets between physical devices. The underlay network is a virtual network built on top of the underlay network. It creates a logical network layer by adding additional encapsulation and tunneling techniques to the underlay network. Overlay networks are typically used in virtualization environments (such as virtual machines and containers) and software-defined networking (SDN), providing flexible network configuration and management. They are usually traditional IP networks, using IP addresses and MAC addresses for packet forwarding and routing. An underlay can be understood as a public network within a specific area, while an overlay can be understood as a private network within that public network.

[0095] In addition, to minimize the performance impact on the original process, by default only node identifiers are recorded, not message information. Specifically, in step 105, as follows... Figure 4 As shown:

[0096] Step 1051: For any partitioned region, if any node in the preset node graph is determined to process the message to be processed in the partitioned region, obtain the node identifier of the node.

[0097] Step 1052: Use the node identifier as the processing result of the message to be processed.

[0098] Step 1053: Record the processing result in the preset cache space corresponding to the message to be processed.

[0099] For example, suppose there is a pre-defined node graph, in which one node is responsible for processing packets with a source IP address of 192.168.1.10. This node's node identifier is "node_123". We can first determine whether this node is a valid target node. If so, we record the node identifier "node_123" as the processing result in the pre-defined cache space corresponding to the packet to be processed.

[0100] It should be noted that if detailed message information is required, it can be configured in the tracking policy set by the user for the message to be processed. After configuration, not only the node identifier can be obtained, but also the specific information of the message to be processed at that node can be obtained.

[0101] Step 106: After determining that the processing is complete, import the processing results in the preset cache space corresponding to the message to be processed into the preset log library, and then clear the processing results in the preset cache space.

[0102] In this embodiment of the invention, the preset log library can be the log processing component zlog. zlog provides flexible logging functionality, supporting multiple log levels, log formats, and output destinations (such as files, console, etc.). zlog is designed to provide high-performance and low-latency logging, making it suitable for systems requiring efficient logging.

[0103] In this embodiment of the invention, after determining that all messages to be processed within the same partitioned area have been processed by nodes in the preset node graph, the processing results (also known as the corresponding cache information) of different messages to be processed are obtained from the preset cache space. This processing result is backed up, i.e., imported into the preset log library. Then the preset cache space is cleared so that the next batch of messages to be processed can be used when passing through the preset node graph. The preset cache space can be used periodically.

[0104] In this embodiment of the invention, a number of messages to be processed and a first number of messages to be processed are obtained. The messages to be processed are divided according to a preset value and the first number of messages to be processed, resulting in several regions for each message to be processed. For any given region, a tracking identifier is added to the messages to be processed within that region. This facilitates subsequent determination of the preset cache space containing the cached information of the messages to be processed based on the tracking identifier. For any given region, the processing results of the nodes in a preset node graph for the messages to be processed within that region are obtained, and the processing results are recorded in the preset cache space corresponding to the messages to be processed. When processing is determined to be complete, the processing results in the preset cache space corresponding to the messages to be processed are imported into a preset log database, and then the processing results in the preset cache space are cleared. This allows messages to be processed in new regions to continue using the preset cache space, reducing storage space usage. This application limits the number of messages processed at one time by batch processing the messages to be processed. Furthermore, different batches of messages to be processed share the same preset cache space, greatly reducing storage space and improving system performance.

[0105] Figure 5 This is a structural diagram of a message processing apparatus provided in an embodiment of the present invention. The apparatus may include:

[0106] The first acquisition module 201 is used to acquire the message to be processed and the first number of messages to be processed.

[0107] The segmentation module 202 is used to segment the messages to be processed according to a preset value and a first number of messages to be processed, so as to obtain several segmentation regions for the messages to be processed.

[0108] The tracking identifier adding module 203 is used to add tracking identifiers to the messages to be processed within any segmented area.

[0109] The first determining module 204 is used to determine the preset buffer space corresponding to the message to be processed based on the tracking identifier.

[0110] The recording module 205 is used to obtain the processing results of the nodes in the preset node graph for the message to be processed in any segmented region, and record the processing results in the preset cache space corresponding to the message to be processed.

[0111] The clearing module 206 is used to clear the processing results in the preset cache space after importing the processing results in the preset log library corresponding to the message to be processed into the preset cache space when the processing is determined to be completed.

[0112] Optionally, the tracking identifier adding module 203 specifically includes:

[0113] The first determination submodule is used to determine the second number of packets to be processed within any given region.

[0114] The first acquisition submodule is used to acquire the target integer value from 0 to the second quantity minus one.

[0115] The identifier addition submodule is used to add target integer values ​​as tracking identifiers to the pending messages within the defined region.

[0116] Optionally, the message processing apparatus may also include:

[0117] The second acquisition module is used to acquire the identity identifier of the preset node graph.

[0118] The cache space setting module is used to set a preset number of preset cache spaces for the preset node graph. The preset cache space carries the identity identifier of the preset node graph.

[0119] The third acquisition module is used to sort the preset cache space and obtain the sort position value of the preset cache space.

[0120] The location identifier addition module is used to add different location identifiers to different preset cache spaces based on the sort position value.

[0121] Optionally, the first determining module 204 specifically includes:

[0122] The modulo operation submodule is used to perform modulo operations on the tracking markers according to a preset number to obtain the modulo operation value.

[0123] The retrieval submodule is used to perform retrieval based on the modulo operation value within the location identifier.

[0124] The second acquisition submodule is used to acquire the target preset cache space corresponding to the target location identifier if a target location identifier that matches the modulo operation value is retrieved from the location identifier.

[0125] The second determination submodule is used to determine the target preset cache space as the preset cache space corresponding to the message to be processed corresponding to the tracking identifier.

[0126] Optionally, the message processing apparatus may also include:

[0127] The fourth acquisition module is used to acquire the tracking policy set by the user for the message to be processed.

[0128] The second determination module is used to determine the target valid node from the nodes in the preset node graph according to the tracking strategy. The target valid node is used to obtain the processing result of the message to be processed in the divided area.

[0129] Optionally, the fourth acquisition module specifically includes:

[0130] The page settings submodule is used to pre-set the tracking strategy, including a parameter adjustment page. The parameter adjustment page includes at least one of the following parameters to be adjusted: network communication protocol, source Internet Protocol address, destination Internet Protocol address, source port, destination port, target node identifier that allows the acquisition of message content, and dropped packet tracking switch.

[0131] The generation submodule is used to generate a tracking strategy for pending messages in response to user settings for parameters to be adjusted on the parameter adjustment page.

[0132] Optionally, the recording module 205 specifically includes:

[0133] The third acquisition submodule is used to acquire the node identifier of any node in any partitioned region, provided that any node in the preset node graph processes the message to be processed in the partitioned region.

[0134] The result acquisition submodule is used to take the node identifier as the processing result of the message to be processed.

[0135] The recording submodule is used to record the processing results to the preset cache space corresponding to the message to be processed.

[0136] In this embodiment of the invention, a number of pending messages and a preset quantity of pending messages are obtained. The pending messages are then divided into several regions based on a preset value and the preset quantity of pending messages. For any given region, a tracking identifier is added to the pending messages within that region. This facilitates subsequent determination of the preset cache space containing the cached information of the pending messages based on the tracking identifier. For any given region, the processing results of the nodes in a preset node graph for the pending messages within that region are obtained, and the processing results are recorded in the preset cache space corresponding to the pending messages. When processing is complete, the processing results in the preset cache space corresponding to the pending messages are imported into a preset log database, and then the processing results in the preset cache space are cleared. This allows pending messages in a new region to continue using the preset cache space, reducing storage space usage. This application limits the number of messages processed at once by batch processing pending messages. Furthermore, different batches of pending messages share the same preset cache space, significantly reducing storage space and improving system performance. Additionally, by default, only node identifiers are tracked, and only the message information of the nodes required by the user is obtained, further reducing performance overhead.

[0137] The present invention also provides an electronic device, Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. See also... Figure 6 The system includes a processor 301, a memory 302, and a computer program 3021 stored in the memory and executable on the processor. When the processor executes the program, it implements the following message processing method:

[0138] Obtain the messages to be processed and a first number of the messages to be processed;

[0139] According to a preset value and a first number of messages to be processed, the messages to be processed are divided to obtain several division regions for the messages to be processed.

[0140] For any of the defined regions, add a tracking identifier to the message to be processed within the defined region;

[0141] The preset cache space corresponding to the message to be processed is determined based on the tracking identifier;

[0142] For any of the aforementioned partitioned regions, the processing results of the nodes in the preset node graph for the message to be processed in the partitioned region are obtained, and the processing results are recorded in the preset cache space corresponding to the message to be processed.

[0143] Once the processing is complete, the processing results in the preset cache space corresponding to the message to be processed are imported into the preset log library, and then the processing results in the preset cache space are cleared.

[0144] The present invention also provides a readable storage medium, wherein when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the message processing method of the foregoing embodiments.

[0145] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0146] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. The structure required to construct such a system is readily apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0147] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0148] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0149] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose.

[0150] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0151] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0152] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0154] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0155] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

Claims

1. A message processing method, characterized in that, The method includes: Obtain the messages to be processed and a first number of the messages to be processed; According to a preset value and the first number of messages to be processed, the messages to be processed are divided to obtain several division regions for the messages to be processed. The preset value is set according to the maximum data processing capacity of the preset node graph or the standard that allows the system performance to be in the best state. For any of the defined regions, add a tracking identifier to the message to be processed within the defined region; The preset cache space corresponding to the message to be processed is determined based on the tracking identifier; For any of the aforementioned partitioned regions, the processing results of the nodes in the preset node graph for the packets to be processed in the partitioned region are obtained, and the processing results are recorded in the preset cache space corresponding to the packets to be processed. The preset node graph is the node graph of DPDK. The node graph of DPDK refers to the logical structure diagram of the packet processing flow in the DPDK application. The logical structure diagram shows the flow path of the packets between different processing nodes. A node in the logical structure diagram represents a logical unit or stage of packet processing. Once the processing is complete, the processing results in the preset cache space corresponding to the message to be processed are imported into the preset log library, and then the processing results in the preset cache space are cleared.

2. The method according to claim 1, characterized in that, For any of the defined regions, adding a tracking identifier to the unprocessed message within the defined region includes: For any of the defined regions, determine a second number of packets to be processed within the defined region; Obtain the target integer value from 0 to the second quantity minus one; The target integer value is used as a tracking identifier and added to the message to be processed in the divided region one by one.

3. The method according to claim 1, characterized in that, Before determining the preset buffer space corresponding to the message to be processed based on the tracking identifier, the method further includes: Obtain the identity identifier of the preset node graph; A preset number of preset cache spaces are set for the preset node graph, and the preset cache spaces carry the identity identifier of the preset node graph; Sort the preset cache space and obtain the sorting position value of the preset cache space; Based on the sorting position value, different position identifiers are added to different preset cache spaces.

4. The method according to claim 3, characterized in that, The step of determining the preset buffer space corresponding to the message to be processed based on the tracking identifier includes: The tracking identifier is moduloed according to the preset quantity to obtain the modulo value; The location identifier is searched based on the modulo operation value; If a target location identifier that matches the modulo operation value is found from the location identifiers, then the target preset cache space corresponding to the target location identifier is obtained; The target preset cache space is determined as the preset cache space corresponding to the message to be processed corresponding to the tracking identifier.

5. The method according to claim 1, characterized in that, Before obtaining the processing results of the nodes within the preset node graph for the message to be processed in the divided region, the method further includes: Obtain the tracking policy set by the user for the message to be processed; According to the tracking strategy, a target valid node is determined from the nodes in the preset node graph, wherein the target valid node is used to obtain the processing result of the message to be processed in the divided region.

6. The method according to claim 5, characterized in that, The step of obtaining the tracking policy set by the user for the message to be processed further includes: The tracking strategy is pre-configured with a parameter adjustment page, which includes at least one of the following parameters to be adjusted: network communication protocol, source Internet Protocol address, destination Internet Protocol address, source port, destination port, target node identifier that allows acquisition of message content, and dropped packet tracking switch. In response to the user's settings for the parameters to be adjusted on the parameter adjustment page, a tracking strategy for the message to be processed is generated.

7. The method according to claim 1, characterized in that, The step of obtaining the processing results of the nodes in the preset node graph for the unprocessed message in any of the partitioned regions, and recording the processing results in the preset cache space corresponding to the unprocessed message, further includes: For any of the aforementioned partitioned regions, if it is determined that any node in the preset node graph processes the message to be processed in the partitioned region, the node identifier of the node is obtained. The node identifier is used as the processing result of the message to be processed; The processing result is recorded in the preset cache space corresponding to the message to be processed.

8. A message processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the message to be processed and the first number of the message to be processed; The partitioning module is used to partition the messages to be processed according to a preset value and a first number of messages to be processed, to obtain several partitioned regions for the messages to be processed. The preset value is set according to the maximum data processing capacity of the preset node graph or the standard that allows the system performance to be in the best state. The tracking identifier adding module is used to add a tracking identifier to the message to be processed within any of the defined regions. The first determining module is used to determine the preset cache space corresponding to the message to be processed based on the tracking identifier; The recording module is used to obtain the processing results of the nodes in the preset node graph for the packet to be processed in any of the partitioned regions, and record the processing results in the preset cache space corresponding to the packet to be processed. The preset node graph is the node graph of DPDK. The node graph of DPDK refers to the logical structure diagram of the packet processing flow in the DPDK application. The logical structure diagram shows the flow path of the packet between different processing nodes. A node in the logical structure diagram represents a logical unit or stage of packet processing. The clearing module is used to import the processing results in the preset cache space corresponding to the message to be processed into the preset log library after determining that the processing is completed, and then clear the processing results in the preset cache space.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the message processing method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the message processing method as described in any one of claims 1-7.

Images