A message processing method, device, and storage medium

Abstract

The present application discloses a message processing method, device and storage medium, which includes: when a message from a physical port is obtained, querying a preset mapping relationship table according to the physical port to determine the default priority of the physical port and the corresponding logical port; performing hard parsing on the message to obtain a message hard parsing result; wherein the message hard parsing result includes the preset fields and / or protocol message type of the message; determining the processing priority of the message according to the message hard parsing result and the logical port; storing the message in the cache or discarding it according to the preset discard threshold value corresponding to the processing priority and the current cache occupancy. The technical solution of the present application can support parsing multiple types of messages, thereby refining the priority of message processing and improving cache utilization.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a message processing method, device and storage medium. Background Art

[0002] In communication networks, there are scenarios where a large number of messages burst in. In such scenarios, the number of messages received far exceeds the system's maximum processing capacity. As a result, a large number of messages cannot be correctly forwarded, severely impacting network transmission efficiency and stability. This can even cause avalanches, leading to network equipment crashes and loss of user service quality. Increasing system processing power or cache capacity to cope with these inevitable bursts of messages in networks can lead to overcapacity and waste of resources. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a message processing method, device and storage medium, aiming to support the processing of core services of a communication network even when a large number of messages are received.

[0004] The present invention provides a method for processing a message, including:

[0005] When a message from a physical port is obtained, querying a preset mapping relationship table according to the physical port to determine the default priority of the physical port and the corresponding logical port;

[0006] Performing hard parsing on the message to obtain a message hard parsing result; wherein the message hard parsing result includes a preset field and / or a protocol message type of the message;

[0007] Determining a processing priority of the message according to a hard parsing result of the message and the logical port;

[0008] The message is stored in the cache or discarded according to the preset discard threshold corresponding to the processing priority and the current cache occupancy.

[0009] An embodiment of the present application also provides a device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the message processing method provided in the embodiment is realized.

[0010] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the message processing method provided in the embodiment.

[0011] The message processing method, device and storage medium provided in the embodiments of the present application support parsing of multiple types of messages, thereby refining the priority of message processing to solve the problems of system excess performance and resource waste caused by increasing the system's processing power or cache capacity to handle message burst scenarios, and ensure that the processing of core services of the communication network can still be supported when a large number of messages are received. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flowchart of a message processing method provided in an embodiment of the present application.

[0013] Figure 2 It is a structural diagram of a message processing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] To make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of this application can be combined with each other in any way.

[0015] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0016] In the subsequent description, suffixes such as "module," "component," or "unit" used to represent elements are used only to facilitate the description of the present application and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0017] like Figure 1 As shown, this embodiment provides a message processing method, which includes:

[0018] S110. When a message is obtained from a physical port, query a preset mapping table according to the physical port to determine a default priority of the physical port and a corresponding logical port;

[0019] S120. Perform hard parsing on the message to obtain a message hard parsing result; wherein the message hard parsing result includes a preset field and / or a protocol message type of the message;

[0020] S130, determining a processing priority of the message according to the message hard parsing result, the logical port, and the physical port;

[0021] S140: storing the message in a cache or discarding it according to a preset discarding threshold corresponding to the processing priority and a current cache occupancy.

[0022] Among them, the messages sent from different physical ports correspond to different default priorities. The priorities of messages sent from different physical ports can be configured in advance to generate a preset mapping relationship table including the correspondence between physical ports and default priorities. For example, for messages from a standard Ethernet port, the default priority and logical port are obtained by looking up the table of the physical port. The obtained message is hard parsed, and the hard parsing identifies the protocol type through different layers of the message header and identifies the preset fields in the message. The preset fields include labels related to the protocol type, and other user-defined fields for programmable parsing of messages. There are two main methods of message parsing. In addition to hard parsing, message parsing can also use programmable parsing. Programmable parsing is to extract the user's interested fields inside the message, form a key value, compare it with the configured template, and process it after the comparison is successful. The fields of user interest include labels related to the protocol type and other user-defined fields for programmable parsing of messages. Currently, hard parsing of packets only covers a limited number of protocol types. To support the expansion of parsing for new packet types, hard parsing and programmable parsing are combined. During the hard parsing phase, pre-set fields are identified as fields of interest set by the user during programmable parsing. The pre-set fields obtained through hard parsing and the logical port mapped to the physical port are combined as key values, which are then matched against the corresponding entry. If a match is found, the corresponding processing priority is mapped. Pre-set fields can be used for programmable parsing to expand protocol packet type parsing, improving overall flexibility. Based on the processing priority of a packet, the preset discard threshold corresponding to that processing priority can be queried. Current cache occupancy can include the cache occupancy of packets of at least one processing priority, the cache occupancy rate of packets of at least one processing priority, and the total cache occupancy rate. Comparing the current cache occupancy with the pre-set discard threshold for a packet can determine whether to cache or discard the packet. Typically, packets related to core communications network services are assigned a higher processing priority to ensure they can be cached. Other packets can be assigned a lower processing priority. When there are more high-priority packets, packets with lower processing priorities are discarded.

[0023] In one implementation, the preset fields of the message include:

[0024] The message includes at least one of a data-gathering medium access control (DMAC) field, a wireless sensor network medium access control (SMAC) field, a virtual local area network (VLAN) tag, a multi-protocol label switching (MPLS) tag, a differentiated services code point (DSCP) field of the IP message, and a custom field for TCAM (Ternary Content Addressable Memory) matching.

[0025] Among them, it supports obtaining message types based on different physical port parsing, and can support the parsing of Layer 2 messages, Layer 3 IP messages, Transmission Control Protocol (TCP) messages and User Datagram Protocol (UDP) messages; it can support VLAN tag parsing; it can support MPLS label parsing.

[0026] Supported protocol message types include: Layer 2 protocol messages such as Bridge Protocol Data Unit (BPDU) messages and Intermediate System-to-Intermediate System (IS-IS) messages; and Layer 3 protocol messages such as Internet Group Management Protocol (IGMP) messages, Internet Control Message Protocol (ICMP) messages, Multicast Listener Discovery (MLD) messages, and Address Resolution Protocol (ARP) messages.

[0027] VLAN parsing: supports parsing of packets without VLAN, with one layer of VLAN (ivlan, ovlan) and with two layers of VLAN.

[0028] DSCP parsing: supports DSCP field parsing of IPV4 and IPV6 packets.

[0029] MPLS parsing: supports up to seven layers of MPLS label parsing.

[0030] In one implementation, determining the processing priority of the message according to the hard parsing result of the message and the logical port includes:

[0031] Perform TCAM matching based on at least one of the protocol message type, the DMAC field, the SMAC field, the VLAN tag, the MPLS tag, the DSCP field of the IP message, and a custom field for TCAM matching;

[0032] If the TCAM matches and hits a preset index value, the priority mapped to the hit preset index value is used as the processing priority.

[0033] Support for TCAM matching to obtain processing priority addresses the limited protocol coverage of hard parsing alone and supports the expansion of parsing for new message types. TCAM matching combines the preset fields obtained from hard parsing and the logical port mapped to the physical port as a key value, matching it with the corresponding preset index value. If a match is found, the TCAM priority is mapped as the processing priority.

[0034] In one implementation, if the TCAM does not match a preset index value, the method further includes:

[0035] If the message is a preset protocol message, the priority of the configured preset protocol message mapping is used as the processing priority;

[0036] If the message is a non-preset protocol message, in the case where the preset field includes a VLAN tag, the priority of the VLAN tag mapping is used as the processing priority;

[0037] If the message is a non-preset protocol message, and the preset field includes an MPLS label, using the priority of the MPLS label mapping as the processing priority;

[0038] If the message is not a message of the preset protocol, and the preset field does not include a VLAN tag or an MPLS tag, the default priority of the physical port is used as the processing priority.

[0039] If the TCAM does not match a preset index value, the hard parsing result is used to determine whether the message is a preset protocol message, such as the aforementioned BPDU protocol message, IS-IS protocol message, IGMP protocol message, ICMP protocol message, MLD protocol message, and ARP protocol message, and the corresponding preset priority is queried. If the message is not a preset protocol message, the preset field identified by the hard parsing is used to query the corresponding preset priority. If the preset field is not identified, the default priority of the physical port is used as the processing priority.

[0040] In one implementation, storing the message in a cache or discarding it according to a preset discard threshold corresponding to the processing priority and a current cache occupancy includes:

[0041] If the preset discard threshold is greater than the total cache occupancy, storing the message in the cache;

[0042] If the preset discard threshold is less than or equal to the total cache occupancy rate, the message is discarded.

[0043] After determining the processing priority of a message, the corresponding preset discard threshold is determined based on the correspondence between the processing priority and the preset discard threshold, combined with the current buffer occupancy. By comparing the preset discard threshold with the total buffer occupancy, the decision is made whether to cache or discard the message. The preset discard threshold for each processing priority is assigned based on the total buffer occupancy; typically, higher-priority messages have correspondingly higher preset discard thresholds.

[0044] In one implementation, before storing the message in a cache or discarding it according to the preset discard threshold corresponding to the processing priority and the current cache occupancy, the method further includes:

[0045] A corresponding fixed discarding threshold is set for each of the processing priorities, and the fixed discarding threshold is used as the preset discarding threshold.

[0046] A corresponding fixed discard threshold is set for each processing priority, and the fixed discard threshold generally increases with the processing priority from low to high. In this implementation, a static threshold is used, that is, each processing priority corresponds to a fixed discard threshold.

[0047] In one implementation, before setting a corresponding fixed discard threshold for each processing priority and using the fixed discard threshold as the preset discard threshold, the method further includes:

[0048] Obtaining a cache occupancy rate of each message of the processing priority;

[0049] Adjust the fixed discard threshold of the message of the second preset processing priority according to the comparison result of the cache occupancy rate of the message of the first preset processing priority and at least one level of preset occupancy rate; wherein, the first preset processing priority is higher than the second preset processing priority.

[0050] Among them, by monitoring the cache occupancy rate of each processing priority message, the fixed discard threshold value of each processing priority message is adjusted in real time. The first preset processing priority is a relatively high priority, and the second preset processing priority is a relatively low priority. When it is detected that the cache occupancy rate of the first preset processing priority message is low, the fixed discard threshold value of the second preset processing priority message is increased through dynamic adjustment to avoid excessively low cache utilization when the system receives a large number of low priority messages. Multiple levels of preset occupancy rates can be set, and each level of preset occupancy rate corresponds to a different fixed threshold value adjustment target. When the cache occupancy rate of the message of the first preset processing priority is greater than the preset occupancy rate of a certain level, the corresponding fixed threshold value adjustment target is used to adjust the fixed discard threshold value of the message of the second preset processing priority.

[0051] In one implementation, before adjusting the fixed discard threshold for messages of the second preset processing priority based on a comparison result of the cache occupancy of messages of the first preset processing priority with at least one level of preset occupancy, the method further includes:

[0052] Get the total cache occupancy rate;

[0053] According to the comparison result of the total cache occupancy rate and the total occupancy rates of at least two levels of preset values, the adjustment range of the fixed discarding threshold for the second preset processing priority message is controlled.

[0054] Among them, at least two levels of preset total occupancy can be configured, and each level of preset total occupancy has a corresponding fixed discard threshold adjustment range for the second preset processing priority message. By monitoring whether the total cache occupancy exceeds a certain level of preset total occupancy, the corresponding fixed discard threshold adjustment range is selected, and the fixed discard threshold value of the second preset processing priority message is adjusted in real time within the fixed discard threshold adjustment range. When the total cache usage is high, the dynamic adjustment of the low priority is appropriately turned off to restore it to the fixed discard threshold value, thereby avoiding the problem that the low priority message occupies a large amount of cache space due to dynamic adjustment, causing squeezing of the subsequent high priority packet cache.

[0055] Figure 2 The schematic diagram of the structure of the message processing system is shown. The system includes: a port mapping module, a message hard parsing module, a message priority arbitration module and a cache management module.

[0056] The port mapping module obtains the default priority and logical port from the physical port table for the message from the standard Ethernet port;

[0057] The packet hard parsing module performs hard parsing on packets, identifies the DMAC and SMAC fields, parses VLAN tags, MPLS tags, and the DSCP field of IP packets, and obtains the protocol packet type and custom fields used for TCAM matching.

[0058] The message processing priority is obtained through the message priority arbitration module. Based on the protocol message type, VLAN and other special fields obtained by hard parsing, as well as the custom fields inside the intercepted message, TCAM matching is performed. If a hit is found, the priority mapped by the hit index value is used; if a hit is not found and it is identified as a protocol message, the configured protocol message priority is used; if it is a non-protocol VLAN message, the VLAN mapping priority is used; if it is an MPLS message, the MPLS mapping priority is used; if it is other types of messages, the default priority of the port mapping is used;

[0059] The cache management module performs cache management and packet discard operations based on the packet processing priority obtained through TCAM matching and hard parsing results. The cache management module determines the action for incoming packets based on the processing priority of the incoming packet, the discard threshold for the current processing priority, and the current cache occupancy. If the cache occupancy exceeds the discard threshold for a processing priority, the packet is discarded; if the cache occupancy does not exceed the discard threshold for a processing priority, the packet is stored in the cache.

[0060] An embodiment of the present application provides a device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for implementing connection and communication between the processor and the memory. When the program is executed by the processor, any one of the message processing methods in the embodiments of the present application is implemented.

[0061] An embodiment of the present application provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any message processing method in the embodiment of the present application.

[0062] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0063] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0064] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention shall fall within the scope of the present invention.

Claims

1. A message processing method, characterized in that: include: When a message from a physical port is obtained, querying a preset mapping table according to the physical port to determine the default priority of the physical port and the corresponding logical port; Performing hard parsing on the message to obtain a message hard parsing result; wherein the message hard parsing result includes preset fields and / or protocol message types of the message; the preset fields of the message include: a DMAC field, an SMAC field, a VLAN tag, an MPLS tag, a DSCP field of the IP message, and at least one of a custom field for TCAM matching; Determining a processing priority of the message according to a hard parsing result of the message and the logical port; storing the message in the cache or discarding it according to a preset discard threshold corresponding to the processing priority and the current cache occupancy; The determining, according to the hard parsing result of the message and the logical port, a processing priority of the message includes: Perform TCAM matching based on at least one of the protocol message type, the DMAC field, the SMAC field, the VLAN tag, the MPLS tag, the DSCP field of the IP message, and a custom field for TCAM matching; If the TCAM matches a preset index value, the priority mapped to the preset index value is used as the processing priority; If the TCAM match does not hit the preset index value, the method further includes: If the message is a preset protocol message, the priority of the configured preset protocol message mapping is used as the processing priority; If the message is a non-preset protocol message, in the case where the preset field includes a VLAN tag, the priority of the VLAN tag mapping is used as the processing priority; If the message is a non-preset protocol message, and the preset field includes an MPLS label, using the priority of the MPLS label mapping as the processing priority; If the message is not a message of the preset protocol, and the preset field does not include a VLAN tag or an MPLS tag, the default priority of the physical port is used as the processing priority.

2. The method according to claim 1, characterized in that The step of storing the message in a cache or discarding the message according to a preset discard threshold corresponding to the processing priority and a current cache occupancy includes: If the preset discard threshold is greater than the total cache occupancy, storing the message in the cache; If the preset discard threshold is less than or equal to the total cache occupancy rate, the message is discarded.

3. The method according to claim 1 or 2, characterized in that Before storing the message into the cache or discarding it according to the preset discard threshold corresponding to the processing priority and the current cache occupancy, the method further includes: A corresponding fixed discarding threshold is set for each of the processing priorities, and the fixed discarding threshold is used as the preset discarding threshold.

4. The method according to claim 3, characterized in that Before setting a corresponding fixed discard threshold for each processing priority and using the fixed discard threshold as the preset discard threshold, the method further includes: Obtaining a cache occupancy rate of each message of the processing priority; Adjust the fixed discard threshold of the message of the second preset processing priority according to the comparison result of the cache occupancy rate of the message of the first preset processing priority and at least one level of preset occupancy rate; wherein, the first preset processing priority is higher than the second preset processing priority.

5. The method according to claim 4, characterized in that Before adjusting the fixed discard threshold for messages of the second preset processing priority according to the comparison result of the cache occupancy of the messages of the first preset processing priority with at least one level of preset occupancy, the method further includes: Get the total cache occupancy rate; According to the comparison result of the total cache occupancy rate and the total occupancy rates of at least two levels of preset values, the adjustment range of the fixed discarding threshold for the second preset processing priority message is controlled.

6. A device, characterized in that The device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection and communication between the processor and the memory. When the program is executed by the processor, the message processing method according to any one of claims 1 to 5 is implemented.

7. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the message processing method according to any one of claims 1 to 5.

Images