Message processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a message processing method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining a to-be-processed message and a first number of the to-be-processed message, dividing the to-be-processed message according to a preset value and the first number of the to-be-processed message, and obtaining a plurality of divided regions related to the to-be-processed message, for any divided region, adding a tracking identifier to the to-be-processed message in the divided region, determining a preset cache space corresponding to the to-be-processed message according to the tracking identifier, and for any divided region, obtaining a processing result of the to-be-processed message in the divided region by a node in the preset node graph, according to the method and the device, the to-be-processed message is processed, the processing result is recorded to the preset cache space corresponding to the to-be-processed message, and the processing result in the preset cache space corresponding to the to-be-processed message is cleared after the processing result in the preset cache space corresponding to the to-be-processed message is imported into the preset log library under the condition of determining that the processing is finished, so that the occupied storage space is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a message processing method, device, electronic equipment and readable storage medium. Background Art

[0002] The Node Graph framework of DPDK (Data Plane Development Kit) is inspired by the open source technology solution VPP (Vector Packet Processing). The framework adopts the architecture of vector processing packets and divides the packet processing flow into independent processing nodes (nodes). Each node processes a batch of packets in batches, and then flows sequentially to the next node to continue batch processing. The Node Trace node tracking function of DPDK can provide relevant information about packet processing. This design effectively solves the instruction cache miss problem caused by complex changes in traditional pipelines, and significantly improves the system's throughput.

[0003] However, in scenarios with heavy traffic, the Node Trace function of DPDK requires the tracing library to collect a large amount of data, which takes up a lot of storage space and causes system performance degradation. Summary of the invention

[0004] The present invention provides a message processing method, device, electronic device and readable storage medium to solve the problem that in the existing DPDK Node Trace node tracing function under the scenario of large traffic, the tracing library needs to collect a large amount of data, and these tracing data will occupy a large amount of storage space, resulting in a decrease in system performance.

[0005] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0006] In a first aspect, the present invention provides a message processing method, the method comprising:

[0007] Obtaining a message to be processed and a first number of the messages to be processed;

[0008] Dividing the messages to be processed according to a preset value and the first number of the messages to be processed to obtain a plurality of divided areas for the messages to be processed;

[0009] For any of the divided areas, adding a tracking mark to the message to be processed in the divided area;

[0010] Determine a preset cache space corresponding to the message to be processed according to the tracking identifier;

[0011] For any of the divided areas, obtaining the processing result of the nodes in the preset node graph on the to-be-processed message in the divided area, and recording the processing result into the preset cache space corresponding to the to-be-processed message;

[0012] When it is determined that the processing is finished, the processing result in the preset cache space corresponding to the message to be processed is imported into the preset log library, and then the processing result in the preset cache space is cleared.

[0013] Optionally, for any of the divided areas, adding a tracking mark to the to-be-processed message in the divided area includes:

[0014] For any of the divided areas, determining a second number of the to-be-processed messages in the divided area;

[0015] Get a target integer value between 0 and the second number minus one;

[0016] The target integer value is used as a tracking identifier and is added one by one to the to-be-processed messages in the divided area.

[0017] Optionally, before determining the preset cache space corresponding to the to-be-processed message according to the tracking identifier, the method further includes:

[0018] Get the identity of the preset node graph;

[0019] Setting a preset number of preset cache spaces for the preset node graph, wherein the preset cache spaces carry the identity of the preset node graph;

[0020] Sorting the preset cache space to obtain a sorting position value of the preset cache space;

[0021] According to the sorting position value, different position identifiers are added to different preset cache spaces.

[0022] Optionally, determining a preset cache space corresponding to the to-be-processed message according to the tracking identifier includes:

[0023] Performing a modular operation on the tracking identifier according to the preset number to obtain a modular operation value;

[0024] Retrieving the location identifier according to the modulo operation value;

[0025] If a target location identifier that is consistent with the modulo operation value is retrieved from the location identifier, a target preset cache space corresponding to the target location identifier is obtained;

[0026] The target preset cache space is determined as the preset cache space corresponding to the to-be-processed message corresponding to the tracking identifier.

[0027] Optionally, before obtaining the processing result of the nodes in the preset node graph on the to-be-processed messages in the divided areas, the method further includes:

[0028] Obtaining a tracking strategy set by a user for the message to be processed;

[0029] According to the tracking strategy, a target valid node is determined from the nodes in the preset node graph, wherein the target valid node is used to obtain the processing result of the to-be-processed message in the divided area.

[0030] Optionally, the obtaining of a tracking strategy set by a user for the message to be processed further includes:

[0031] Pre-setting a tracking strategy includes a parameter adjustment page, wherein the parameter adjustment page includes at least one parameter to be adjusted among a network communication protocol, a source Internet Protocol address, a destination Internet Protocol address, a source port, a destination port, a target node identifier for allowing acquisition of message content, and a discarded data packet tracking switch;

[0032] In response to the user setting the parameter to be adjusted on the parameter adjustment page, a tracking strategy for the message to be processed is generated.

[0033] Optionally, for any of the divided areas, obtaining a processing result of a node in a preset node graph on the message to be processed in the divided area, and recording the processing result in a preset cache space corresponding to the message to be processed, further includes:

[0034] For any of the divided areas, when it is determined that any node in the preset node graph processes the to-be-processed message of the divided area, obtaining a node identifier of the node;

[0035] Using the node identifier as a processing result of the message to be processed;

[0036] The processing result is recorded in the preset cache space corresponding to the message to be processed.

[0037] In a second aspect, the present invention provides a message processing device, the device comprising:

[0038] A first acquisition module, used to acquire a message to be processed and a first number of the messages to be processed;

[0039] A division module, used for dividing the to-be-processed message according to a preset value and the first number of the to-be-processed messages to obtain a plurality of division areas for the to-be-processed message;

[0040] A tracking mark adding module, used for adding a tracking mark to the message to be processed in any of the divided areas;

[0041] A first determining module, configured to determine a preset cache space corresponding to the message to be processed according to the tracking identifier;

[0042] A recording module, used for obtaining, for any of the divided areas, processing results of the nodes in the preset node graph on the messages to be processed in the divided areas, and recording the processing results into a preset cache space corresponding to the messages to be processed;

[0043] The clearing module is used to, when it is determined that the processing is finished, import the processing result in the preset cache space corresponding to the message to be processed into the preset log library, and then clear the processing result in the preset cache space.

[0044] In a third aspect, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned message processing method when executing the program.

[0045] In a fourth aspect, the present invention provides a readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned message processing method.

[0046] In an embodiment of the present invention, a message to be processed and a first number of messages to be processed are obtained, and the messages to be processed are divided according to a preset value and the first number of messages to be processed to obtain a number of divided areas for the messages to be processed. For any divided area, a tracking mark is added to the messages to be processed in the divided area, so as to facilitate the subsequent determination of the preset cache space where the cache information of the messages to be processed is located according to the tracking mark. For any divided area, the processing results of the nodes in the preset node graph on the messages to be processed in the divided area are obtained, and the processing results are recorded in the preset cache space corresponding to the messages to be processed. When it is determined that the processing is finished, the processing results in the preset cache space corresponding to the messages to be processed are imported into the preset log library, and the processing results in the preset cache space are cleared, so as to facilitate the subsequent use of the preset cache space by the messages to be processed in the new divided area, thereby reducing the occupancy of the storage space. The present application limits the number of messages processed at one time by batch processing the messages to be processed, and at the same time, different batches of messages to be processed share the same preset cache space, which greatly reduces the storage space and improves the system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a flowchart of a message processing method provided by an embodiment of the present invention;

[0049] Figure 2 yes Figure 1 The flowchart of step 103 in a message processing method provided by an embodiment of the present invention is shown;

[0050] Figure 3 yes Figure 1 The flowchart of step 104 in a message processing method provided by an embodiment of the present invention is shown;

[0051] Figure 4 yes Figure 1 The flowchart of step 105 in a message processing method provided by an embodiment of the present invention is shown;

[0052] Figure 5 is a structural diagram of a message processing device provided by an embodiment of the present invention;

[0053] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] DPDK (Data Plane Development Kit) is an open source library and driver collection designed to accelerate packet processing operations. DPDK is mainly used in application scenarios such as network function virtualization (NFV), software defined network (SDN), high-performance routing and firewall. The node graph of DPDK usually refers to the logical structure diagram of the packet processing flow in the DPDK application. This graph shows the flow path of the packet between different processing nodes. A node in the graph represents a logical unit or stage of packet processing. Each node is responsible for a specific packet processing task, such as receiving, parsing, forwarding, modifying, and sending packets. In DPDK, node trace is a debugging and performance analysis tool for tracing the flow path of packets in the node graph. Through node trace, developers can observe the transmission of packets between nodes, identify potential performance bottlenecks, and optimize them. However, when the existing node trace is used in scenarios with large traffic, due to the large amount of traced data, it takes up a large space and affects the system performance. At the same time, when custom tracing of packets is performed, the setting of tracing conditions needs to be performed in the underlying code, and the code can only be modified by the developer, which has poor flexibility. In order to solve the above problem, a message processing method is proposed in an embodiment of the present invention.

[0056] Reference Figure 1 , Figure 1 is a flow chart of the steps of a message processing method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0057] Step 101: Obtain a message to be processed and a first number of messages to be processed.

[0058] In the embodiment of the present invention, the message to be processed can also be referred to as a data packet to be processed. In a scenario with large traffic, there will be a large number of messages to be processed. If all of them enter the preset node graph for processing, due to the large amount of data being tracked, a large space will be required, affecting system performance. Therefore, in the embodiment of the present invention, the first number of messages to be processed will be obtained, and then divided for batch processing.

[0059] Step 102: Divide the messages to be processed according to a preset value and a first number of the messages to be processed to obtain a plurality of divided areas for the messages to be processed.

[0060] The preset value in the embodiment of the present invention can be set according to the maximum data capacity of the preset node graph, or it can be set according to the standard that allows the system performance to be in the best state. The present invention does not make specific limitations here. When dividing the messages to be processed according to the preset value and the first number of messages to be processed, it is first required that the first number of messages to be processed is greater than the preset value. If it is less than the preset value, it indicates that there are fewer messages to be processed. At this time, it will not be processed in batches, and it will not affect the system performance. When it is determined that the first number of messages to be processed is greater than the preset value, the messages to be processed can be divided according to the preset value.

[0061] For example, the preset value is set to 32, there are 100 messages to be processed, 100÷32=3...4, so 4 divided areas can be obtained, among which there are 32 messages to be processed in 3 areas and 4 messages to be processed in 1 area.

[0062] Step 103: for any divided area, a tracking mark is added to the message to be processed in the divided area.

[0063] Through the above steps, multiple partitioned areas are obtained. When processing the pending messages through the preset node graph, the pending messages in one partitioned area are processed at a time. And these pending messages need to add a tracking identifier trace id, so that it is convenient to find the corresponding position in the pre-allocated cache according to the tracking identifier later. When setting the tracking identifier, because the pending messages are processed in batches, the pending messages of different batches will not affect each other. Therefore, the tracking identifiers of the pending messages in different partitioned areas can be set to be consistent, and the tracking identifiers of the pending messages of the same batch (same partitioned area) are set from 0 and increase in sequence. Of course, in order to distinguish, it is also possible to set an identifier for the partitioned area, and then add the tracking identifier and the partitioned area identifier to the pending message, so as to uniquely determine the corresponding pending message.

[0064] Further, step 103, such as Figure 2 As shown:

[0065] Step 1031: for any divided area, determine a second number of messages to be processed in the divided area.

[0066] Step 1032, obtaining a target integer value between 0 and the second number minus one.

[0067] Step 1033: Use the target integer value as a tracking identifier and add it one-to-one to the messages to be processed in the divided area.

[0068] For example, if there are pending messages A, B, C, D, E, and F in partition area a, it can be seen that there are 6 pending messages in partition area A, and the target integer values ​​are 0, 1, 2, 3, 4, and 5. These target integer values ​​are added to the pending messages one by one, and the pending messages A0, B1, C2, D3, E4, and F5 are obtained. If the partition area identifier a is set for partition area a, the pending messages after adding the identifier are Aa0, Ba1, Ca2, Da3, Ea4, and Fa5.

[0069] Through the above steps, a unique tracking identifier is assigned to each to-be-processed message in the divided area, so that the corresponding position in the pre-allocated cache can be found according to the tracking identifier. At the same time, the to-be-processed message can also be tracked and processed according to this tracking identifier.

[0070] Step 104: determine a preset cache space corresponding to the message to be processed according to the tracking identifier.

[0071] In order to ensure that each message to be processed has its own cache space, the number of preset cache spaces in the embodiment of the present invention is set according to a preset value. Moreover, a certain number of preset cache spaces will be set for each preset node graph, and the identity identifier of the preset node graph will be associated with each cache space. Then, the preset cache spaces are sorted, and the sorting position value of each cache space is obtained. The sorting can be based on certain rules, such as the size of the cache space, the order of creation time, etc., which are not specifically limited in the present invention. Then, according to the sorting position value, a unique position identifier is added to each cache space. The position identifier can be a unique identifier based on the sorting position value. The specific steps include:

[0072] Get the identity of the preset node graph;

[0073] Setting a preset number of preset cache spaces for a preset node graph, the preset cache spaces carrying an identity identifier of the preset node graph;

[0074] Sort the preset cache space and obtain the sort position value of the preset cache space;

[0075] According to the sorting position value, different position identifiers are added to different preset cache spaces.

[0076] For example, there is a preset node graph with an identity "graph_123". 10 cache spaces are set for the preset node graph. Each cache space carries the identity "graph_123" of the preset node graph. After sorting by creation time, the sorting position values ​​are: cache space 1: sorting position value = 0, cache space 2: sorting position value = 1, cache space 3: sorting position value = 2... cache space 10: sorting position value = 9, so the position identification of cache space 1 is 0, the position identification of cache space 2 is 1, the position identification of cache space 3 is 2... the position identification of cache space 10 is 9.

[0077] Through these steps, a preset cache space is set for the preset node graph, and a unique location identifier is allocated to each preset cache space to facilitate subsequent management and use.

[0078] Further, after a tracking identifier is set for each of the to-be-processed messages in each divided area and a location identifier is set for the preset cache space, a value is obtained from the tracking identifier and the location identifier, and then the value of the tracking identifier is modulo-calculated according to the quantity value of the location identifier, and the obtained modulo operation value is compared with the value obtained from the location identifier. If the comparison is consistent, the preset cache space corresponding to the location identifier at the time of comparison is allocated to the to-be-processed message corresponding to the tracking identifier at the time of comparison. Step 104, as Figure 3 As shown:

[0079] Step 1041 , performing a modular operation on the tracking identifier according to a preset number to obtain a modular operation value.

[0080] Step 1042, searching in the location identifier according to the modulo operation value.

[0081] Step 1043: If a target location identifier that is consistent with the modulo operation value is retrieved from the location identifier, the target preset cache space corresponding to the target location identifier is obtained.

[0082] Step 1044: determine the target preset cache space as the preset cache space corresponding to the to-be-processed message corresponding to the tracking identifier.

[0083] Among them, the preset number is the number of preset cache spaces. For example, the preset number is set to 32, the values ​​of the tracking identifiers are 01, 02, 05, and 08, and the modulo operation of the values ​​of the tracking identifiers is: 1mod32=1, 2mod32=2, 5mod32=5, 8mod32=8, so the tracking identifier 01 corresponds to the preset cache space 01, the tracking identifier 02 corresponds to the preset cache space 02, the tracking identifier 05 corresponds to the preset cache space 05, and the tracking identifier 08 corresponds to the preset cache space 08. It should be noted that if the preset value set in the divided area is less than or equal to the preset number set for the preset cache space, then the corresponding location identifier can be directly determined according to the tracking identifier, and then the corresponding preset cache space can be determined without performing a modulo operation.

[0084] Through the above steps, a corresponding preset cache space is allocated to each message to be processed, which facilitates the subsequent query and management of the cache information of the message to be processed.

[0085] Step 105 , for any divided area, obtain the processing result of the nodes in the preset node graph on the to-be-processed message in the divided area, and record the processing result into the preset cache space corresponding to the to-be-processed message.

[0086] In the embodiment of the present invention, for any divided area, after allocating a preset cache space for each to-be-processed message therein, the to-be-processed message can be processed by multiple nodes in the preset node graph. However, before that, the message can also be filtered by a user-defined policy, and only the to-be-processed messages that meet the policy will be tracked. The specific steps include:

[0087] Get the tracking policy set by the user for the message to be processed;

[0088] According to the tracking strategy, a target valid node is determined from the nodes in the preset node graph, wherein the target valid node is used to obtain the processing result of the to-be-processed message in the divided area.

[0089] It should be noted that when the user sets the tracking strategy, the tracking strategy can include various parameters, such as network communication protocol, source IP address, destination IP address, source port, destination port, target node identifier that allows obtaining message content, and discarded data packet tracking switch, etc. For example, assume that the tracking strategy set by the user is as follows: network communication protocol: TCP, source IP address: 192.168.1.10, destination IP address: 10.0.0.5, source port: 80, destination port: 443, target node identifier that allows obtaining message content: node_123, discarded data packet tracking switch: on. According to the above settings, the target valid nodes are determined from the preset node graph, and the target valid nodes are those nodes that meet the requirements of the tracking strategy. Assume that there are the following nodes in the preset node graph: Node 1: Processing TCP protocol, Node 2: Processing source IP address 192.168.1.10, Node 3: Processing destination IP address 10.0.0.5, Node 4: Processing source port 80, Node 5: Processing destination port 443, Node 6: Allowing to obtain message content, Node 7: Processing discarded data packets. According to the tracking strategy set by the user, the target valid nodes are determined as follows: Node 1 (processing TCP protocol), Node 2 (processing source IP address 192.168.1.10), Node 3 (processing destination IP address 10.0.0.5), Node 4 (processing source port 80), Node 5 (processing destination port 443), Node 6 (allowing to obtain message content), Node 7 (processing discarded data packets). The to-be-processed messages in the divided area are processed through the target valid nodes, and the cache information of the to-be-processed messages in these valid nodes is obtained. Other nodes do not record the cache processing results when processing.

[0090] Furthermore, in the embodiment of the present invention, the setting of the tracking strategy does not require code modification, and the parameter adjustment interface can be directly called through the command line, which improves the flexibility of the user-defined strategy (rule). The specific steps include:

[0091] Pre-setting the tracking strategy includes a parameter adjustment page, the parameter adjustment page includes at least one parameter to be adjusted among a network communication protocol, a source Internet Protocol address, a destination Internet Protocol address, a source port, a destination port, a target node identifier that allows obtaining message content, and a discarded data packet tracking switch;

[0092] In response to the user's setting of the to-be-adjusted parameter on the parameter adjustment page, a tracking strategy for the to-be-processed message is generated.

[0093] It should be noted that the network communication protocols on the parameter adjustment page include ARP, ICMP, TCP, DUP, etc., which play a key role in different network layers and transport layers. The source Internet Protocol address and the destination Internet Protocol address can be specified in combination at the same time. Similarly, the source port and the destination port can also be specified in combination at the same time. The specific IP and port values ​​are not specifically limited in this invention. Because the original preset node graph will not obtain the situation of discarded data packets, the embodiment of the present invention can obtain the packet loss situation in the preset node graph when it is turned on by setting a discarded data packet tracking switch.

[0094] In addition, the parameter can also specify whether to match the underlay or overlay part, where the underlay network refers to the physical network infrastructure, including physical switches, routers, optical fibers, cables, etc. It is the basic layer of the network and is responsible for transmitting data packets between physical devices. The underlay network is a virtual network built on top of the underlay network. It creates a logical network layer by adding additional encapsulation and tunneling technologies to the underlay network. Overlay networks are commonly used in virtualized environments (such as virtual machines, containers) and software-defined networks (SDNs) to provide flexible network configuration and management. They are usually traditional IP networks that use IP addresses and MAC addresses for data packet forwarding and routing. Underlay can be understood as a public network in a region, and overlay can be understood as a private network under the public network.

[0095] In addition, in order to reduce the performance impact on the original process, by default, only the node identifier is recorded, and the message information is not recorded. Specifically, step 105, such as Figure 4 As shown:

[0096] Step 1051 , for any divided area, when it is determined that any node in the preset node graph processes the to-be-processed message of the divided area, a node identifier of the node is obtained.

[0097] Step 1052: Use the node identifier as the processing result of the message to be processed.

[0098] Step 1053, recording the processing result into the preset cache space corresponding to the message to be processed.

[0099] For example, suppose there is a preset node graph, in which a node is responsible for processing the message with source IP address 192.168.1.10. The node identifier of the node is "node_123". You can first determine whether the node is a target valid node. If so, the node identifier "node_123" is recorded as the processing result in the preset cache space corresponding to the message to be processed.

[0100] It should be noted that if detailed message information is needed, it can be set in the tracking strategy set by the user for the message to be processed. After setting, not only the node identifier can be obtained, but also the specific information of the message to be processed at the node can be obtained.

[0101] Step 106 , when it is determined that the processing is finished, the processing result in the preset cache space corresponding to the message to be processed is imported into the preset log library, and then the processing result in the preset cache space is cleared.

[0102] In the embodiment of the present invention, the preset log library can be a log processing component zlog, which provides a flexible logging function and supports multiple log levels, log formats and output targets (such as files, consoles, etc.). The design goal of zlog is to provide high-performance and low-latency logging, which is suitable for systems that require efficient logging.

[0103] In the embodiment of the present invention, after determining that all pending messages in the same divided area are processed by the nodes in the preset node graph, the processing results of different pending messages (also referred to as corresponding cache information) are obtained from the preset cache space, and the processing result is backed up, that is, imported into the preset log library, and then the preset cache space is cleared for use by the next batch of pending messages when passing through the preset node graph. The preset cache space can be used periodically.

[0104] In an embodiment of the present invention, a message to be processed and a first number of messages to be processed are obtained, and the messages to be processed are divided according to a preset value and the first number of messages to be processed to obtain a number of divided areas for the messages to be processed. For any divided area, a tracking mark is added to the messages to be processed in the divided area, so as to facilitate the subsequent determination of the preset cache space where the cache information of the messages to be processed is located according to the tracking mark. For any divided area, the processing results of the nodes in the preset node graph on the messages to be processed in the divided area are obtained, and the processing results are recorded in the preset cache space corresponding to the messages to be processed. When it is determined that the processing is finished, the processing results in the preset cache space corresponding to the messages to be processed are imported into the preset log library, and the processing results in the preset cache space are cleared, so as to facilitate the subsequent new divided areas to continue to use the preset cache space for the messages to be processed, thereby reducing the storage space occupied. The present application limits the number of messages processed at one time by batch processing the messages to be processed, and at the same time, different batches of messages to be processed share the same preset cache space, which greatly reduces the storage space and improves the system performance.

[0105] Figure 5 : is a structural diagram of a message processing device provided by an embodiment of the present invention, and the device may include:

[0106] The first acquisition module 201 is used to acquire a message to be processed and a first number of messages to be processed.

[0107] The division module 202 is used to divide the messages to be processed according to a preset value and a first number of the messages to be processed to obtain a plurality of divided areas for the messages to be processed.

[0108] The tracking mark adding module 203 is used to add a tracking mark to the message to be processed in any divided area.

[0109] The first determining module 204 is configured to determine a preset cache space corresponding to the message to be processed according to the tracking identifier.

[0110] The recording module 205 is used to obtain, for any divided area, the processing result of the nodes in the preset node graph on the to-be-processed message in the divided area, and record the processing result in the preset cache space corresponding to the to-be-processed message.

[0111] The clearing module 206 is used to clear the processing results in the preset cache space after importing the processing results in the preset cache space corresponding to the message to be processed into the preset log library when it is determined that the processing is completed.

[0112] Optionally, the tracking identifier adding module 203 specifically includes:

[0113] The first determining submodule is used to determine, for any divided area, a second number of messages to be processed in the divided area.

[0114] The first acquisition submodule is used to acquire a target integer value between 0 and the second number minus one.

[0115] The identifier adding submodule is used to add the target integer value as the tracking identifier to the to-be-processed messages in the divided area one by one.

[0116] Optionally, the message processing device further includes:

[0117] The second acquisition module is used to acquire the identity of the preset node graph.

[0118] The cache space setting module is used to set a preset number of preset cache spaces for a preset node graph, and the preset cache space carries the identity of the preset node graph.

[0119] The third acquisition module is used to sort the preset cache space and obtain the sorting position value of the preset cache space.

[0120] The position identifier adding module is used to add different position identifiers for different preset cache spaces according to the sorting position values.

[0121] Optionally, the first determining module 204 specifically includes:

[0122] The modular operation submodule is used to perform a modular operation on the tracking mark according to a preset number to obtain a modular operation value.

[0123] The retrieval submodule is used to search in the position identifier according to the modulo operation value.

[0124] The second acquisition submodule is used to acquire the target preset cache space corresponding to the target location identifier if a target location identifier consistent with the modulo operation value is retrieved from the location identifier.

[0125] The second determining submodule is used to determine the target preset cache space as the preset cache space corresponding to the to-be-processed message corresponding to the tracking identifier.

[0126] Optionally, the message processing device further includes:

[0127] The fourth acquisition module is used to acquire the tracking strategy set by the user for the message to be processed.

[0128] The second determination module is used to determine a target valid node from the nodes in the preset node graph according to the tracking strategy, wherein the target valid node is used to obtain the processing result of the to-be-processed message in the divided area.

[0129] Optionally, the fourth acquisition module specifically includes:

[0130] The page setting submodule is used to pre-set the tracking strategy, including a parameter adjustment page, which includes at least one parameter to be adjusted, including a network communication protocol, a source Internet Protocol address, a destination Internet Protocol address, a source port, a destination port, a target node identifier that allows acquisition of message content, and a discarded data packet tracking switch.

[0131] The generating submodule is used to generate a tracking strategy for the message to be processed in response to the user's setting of the parameter to be adjusted on the parameter adjustment page.

[0132] Optionally, the recording module 205 specifically includes:

[0133] The third acquisition submodule is used to acquire the node identifier of any divided area when it is determined that any node in the preset node graph processes the to-be-processed message of the divided area.

[0134] The result acquisition submodule is used to use the node identifier as the processing result of the message to be processed.

[0135] The recording submodule is used to record the processing results into the preset cache space corresponding to the message to be processed.

[0136] In an embodiment of the present invention, a message to be processed and a first number of messages to be processed are obtained, and the messages to be processed are divided according to a preset value and the first number of messages to be processed to obtain a number of divided areas for the messages to be processed. For any divided area, a tracking mark is added to the messages to be processed in the divided area, so as to facilitate the subsequent determination of the preset cache space where the cache information of the messages to be processed is located according to the tracking mark. For any divided area, the processing results of the nodes in the preset node graph on the messages to be processed in the divided area are obtained, and the processing results are recorded in the preset cache space corresponding to the messages to be processed. When it is determined that the processing is finished, the processing results in the preset cache space corresponding to the messages to be processed are imported into the preset log library, and the processing results in the preset cache space are cleared, so as to facilitate the subsequent use of the preset cache space by the messages to be processed in the new divided area, thereby reducing the occupancy of the storage space. The present application limits the number of messages to be processed at one time by batch processing the messages to be processed, and at the same time, different batches of messages to be processed share the same preset cache space, which greatly reduces the storage space and improves the system performance. At the same time, by default, only the node identifier is tracked, and only the message information of the node required by the user is obtained, further reducing the loss of performance.

[0137] The present invention also provides an electronic device, Figure 6 is a structural block diagram of an electronic device provided by an embodiment of the present invention, see Figure 6 , comprising a processor 301, a memory 302, and a computer program 3021 stored in the memory and executable on the processor, wherein the processor implements the following steps of the message processing method when executing the program:

[0138] Obtaining a message to be processed and a first number of the messages to be processed;

[0139] Dividing the messages to be processed according to a preset value and the first number of the messages to be processed to obtain a plurality of divided areas for the messages to be processed;

[0140] For any of the divided areas, adding a tracking mark to the message to be processed in the divided area;

[0141] Determine a preset cache space corresponding to the message to be processed according to the tracking identifier;

[0142] For any of the divided areas, obtaining the processing result of the nodes in the preset node graph on the to-be-processed message in the divided area, and recording the processing result into the preset cache space corresponding to the to-be-processed message;

[0143] When it is determined that the processing is finished, the processing result in the preset cache space corresponding to the message to be processed is imported into the preset log library, and then the processing result in the preset cache space is cleared.

[0144] The present invention also provides a readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the message processing method of the aforementioned embodiment.

[0145] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0146] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other equipment. According to the above description, it is obvious that the structure required for constructing such a system is. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific language is for disclosing the best mode of the present invention.

[0147] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0148] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all of the features of the individual embodiments disclosed above. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0149] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0150] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0151] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0154] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0155] It should be noted that the various data-related processes in the embodiments of the present application are all carried out in compliance with the corresponding data protection laws and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

Claims

1. A message processing method, characterized in that: The method comprises: Obtaining a message to be processed and a first number of the messages to be processed; Dividing the messages to be processed according to a preset value and the first number of the messages to be processed to obtain a plurality of divided areas for the messages to be processed; For any of the divided areas, adding a tracking mark to the message to be processed in the divided area; Determine a preset cache space corresponding to the message to be processed according to the tracking identifier; For any of the divided areas, obtaining the processing result of the nodes in the preset node graph on the to-be-processed message in the divided area, and recording the processing result into the preset cache space corresponding to the to-be-processed message; When it is determined that the processing is finished, the processing result in the preset cache space corresponding to the message to be processed is imported into the preset log library, and then the processing result in the preset cache space is cleared.

2. The method according to claim 1, characterized in that: The step of adding a tracking mark to the message to be processed in any of the divided areas includes: For any of the divided areas, determining a second number of the to-be-processed messages in the divided area; Get a target integer value between 0 and the second number minus one; The target integer value is used as a tracking identifier and is added one by one to the to-be-processed messages in the divided area.

3. The method according to claim 1, characterized in that: Before determining the preset cache space corresponding to the to-be-processed message according to the tracking identifier, the method further includes: Get the identity of the preset node graph; Setting a preset number of preset cache spaces for the preset node graph, wherein the preset cache spaces carry the identity of the preset node graph; Sorting the preset cache space to obtain a sorting position value of the preset cache space; According to the sorting position value, different position identifiers are added to different preset cache spaces.

4. The method according to claim 3, characterized in that: The determining, according to the tracking identifier, a preset cache space corresponding to the message to be processed includes: Performing a modular operation on the tracking identifier according to the preset number to obtain a modular operation value; Retrieving the location identifier according to the modulo operation value; If a target location identifier that is consistent with the modulo operation value is retrieved from the location identifier, a target preset cache space corresponding to the target location identifier is obtained; The target preset cache space is determined as the preset cache space corresponding to the to-be-processed message corresponding to the tracking identifier.

5. The method according to claim 1, characterized in that: Before obtaining the processing result of the nodes in the preset node graph on the to-be-processed messages in the divided areas, the method further includes: Obtaining a tracking strategy set by a user for the message to be processed; According to the tracking strategy, a target valid node is determined from the nodes in the preset node graph, wherein the target valid node is used to obtain the processing result of the to-be-processed message in the divided area.

6. The method according to claim 5, characterized in that: The step of obtaining the tracking strategy set by the user for the message to be processed further includes: Pre-setting a tracking strategy includes a parameter adjustment page, wherein the parameter adjustment page includes at least one parameter to be adjusted among a network communication protocol, a source Internet Protocol address, a destination Internet Protocol address, a source port, a destination port, a target node identifier for allowing acquisition of message content, and a discarded data packet tracking switch; In response to the user setting the parameter to be adjusted on the parameter adjustment page, a tracking strategy for the message to be processed is generated.

7. The method according to claim 1, characterized in that: The method of obtaining, for any of the divided areas, processing results of the nodes in the preset node graph on the to-be-processed messages in the divided areas, and recording the processing results into a preset cache space corresponding to the to-be-processed messages, further includes: For any of the divided areas, when it is determined that any node in the preset node graph processes the to-be-processed message of the divided area, obtaining a node identifier of the node; Using the node identifier as a processing result of the message to be processed; The processing result is recorded in the preset cache space corresponding to the message to be processed.

8. A message processing device, characterized in that: The device comprises: A first acquisition module, used to acquire a message to be processed and a first number of the messages to be processed; A division module, used for dividing the to-be-processed message according to a preset value and the first number of the to-be-processed messages to obtain a plurality of division areas for the to-be-processed message; A tracking mark adding module, used for adding a tracking mark to the message to be processed in any of the divided areas; A first determining module, configured to determine a preset cache space corresponding to the message to be processed according to the tracking identifier; A recording module, used for obtaining, for any of the divided areas, processing results of the nodes in the preset node graph on the messages to be processed in the divided areas, and recording the processing results into a preset cache space corresponding to the messages to be processed; The clearing module is used to, when it is determined that the processing is finished, import the processing result in the preset cache space corresponding to the message to be processed into the preset log library, and then clear the processing result in the preset cache space.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the message processing method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enable the processors to execute the message processing method according to any one of claims 1 to 7.

Images