Metadata marking method based on VXLAN protocol and packet transmission system

Abstract

The application discloses a metadata marking method and a message transmission system based on a VXLAN protocol, relates to the technical field of protocol communication, and has the technical scheme as follows: an asset mapping table corresponding one-to-one between cloud asset information and identification numbers is constructed; target asset information corresponding to a target data packet to be forwarded is acquired, and a corresponding identification number is matched from the asset mapping table as a digital identification according to the target asset information; the digital identification is written into the header of the VXLAN protocol, and a VXLAN data packet with completed metadata marking is obtained after the target data packet is encapsulated by the VXLAN protocol. The application enriches the information content that can be carried by the VXLAN tunnel protocol by adopting the mode of the asset mapping table, realizes the association between asset information and flow data in the business scene of multiple cloud platforms, and overcomes the problem that metadata of data is lost due to the VXLAN when the VXLAN is used to forward data packets.

Description

Technical Field

[0001] This invention relates to the field of protocol communication technology, and more specifically, to a metadata tagging method and message transmission system based on the VXLAN protocol. Background Technology

[0002] When simultaneously capturing and analyzing traffic from multiple cloud platforms in a cloud network environment, multiple pieces of information need to be represented within the traffic to associate assets with the traffic. For example, at the IaaS level, the cluster, VPC, and virtual machine ID of the traffic need to be labeled; in a Kubernetes cluster, the Kubernetes cluster, namespace, and POD ID of the traffic need to be labeled. However, due to limitations in some network infrastructures, some scenarios only allow the use of the VXLAN protocol to forward traffic captured within the cloud to the analysis terminal.

[0003] Currently, when analyzing observation packets forwarded based on VXLAN, VXLAN reserves only a limited number of bits, resulting in extremely limited information representation. This can easily lead to the analysis terminal being unable to associate traffic with target assets, such as virtual machines and PODs. While network restrictions can generally be modified to associate traffic with target assets due to VXLAN's inherent limitations, this severely impacts the stability of VXLAN communication for users.

[0004] Therefore, how to research and design a metadata tagging method and message transmission system based on the VXLAN protocol that can overcome the above-mentioned defects is a problem that we urgently need to solve. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a metadata tagging method and message transmission system based on the VXLAN protocol. By adopting an asset mapping table model, the information content that the VXLAN tunnel protocol can carry is enriched. In business scenarios of multiple cloud platforms, the association between asset information and traffic data is realized, overcoming the problem that VXLAN will cause data metadata loss when VXLAN is needed to forward data packets.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0007] Firstly, a metadata tagging method based on the VXLAN protocol is provided, including the following steps:

[0008] Construct an asset mapping table that establishes a one-to-one correspondence between cloud asset information and its identifier numbers;

[0009] Obtain the target asset information corresponding to the target data packet to be forwarded, and match the corresponding identifier number from the asset mapping table based on the target asset information as the digital identifier;

[0010] The numerical identifier is written into the header of the VXLAN protocol, and the target data packet is encapsulated in VXLAN to obtain a VXLAN data packet with completed metadata marking.

[0011] Furthermore, the identification number is generated using the asset's metadata as input.

[0012] Furthermore, the digital identifier includes an asset region and a unique identifier region;

[0013] The asset area represents the asset using a 24-bit number;

[0014] The unique identifier region is represented by an 8-bit number.

[0015] Furthermore, the numbers for the asset regions are generated using an auto-incrementing ID algorithm.

[0016] Furthermore, if an ID in the asset area exceeds 24 bits and overflows, the deleted IDs are reset and reused until the IDs are exhausted.

[0017] Furthermore, the method also includes:

[0018] A fixed tag is generated based on the network configuration parameters corresponding to the analysis terminal that receives the VXLAN data packets;

[0019] After writing the fixed marker into the first 8 VXLAN flag bits in the VXLAN protocol header, VXLAN encapsulation is performed.

[0020] Secondly, a message transmission system based on the VXLAN protocol is provided, including:

[0021] Cloud APIs are used to obtain cloud asset information from the corresponding cloud platform.

[0022] The controller is used to build an asset mapping table that corresponds one-to-one between cloud asset information and identification numbers, and to synchronously update the asset mapping table to the collector and analyzer.

[0023] The collector is used to collect the business traffic of at least one cloud platform to obtain target data packets and target asset information, and to match the corresponding identification number from the asset mapping table as a digital identifier based on the target asset information, and to write the digital identifier into the header of the VXLAN protocol, and to encapsulate the target data packet with VXLAN to obtain a VXLAN data packet with completed metadata marking.

[0024] The analyzer is used to unpack and analyze received VXLAN packets based on the asset mapping table.

[0025] Furthermore, the controller is deployed on a non-business node within the cloud;

[0026] The controller exposes a RESTful API interface to the outside world;

[0027] Commands are created via the API interface, and the controller sends control commands to the data acquisition unit and the analyzer via gRPC.

[0028] Furthermore, the collector is deployed on user service nodes within the cloud.

[0029] Furthermore, the analyzer is deployed on a non-business node outside the cloud.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The metadata tagging method based on the VXLAN protocol provided by this invention enriches the information content that the VXLAN tunnel protocol can carry by adopting the asset mapping table mode. In the business scenarios of multiple cloud platforms, such mapping mode can be used to identify asset information. In other business scenarios, it can also identify other richer information, realize the association between asset information and traffic data, and overcome the problem that VXLAN will cause the data metadata to be missing when VXLAN is needed to forward data packets.

[0032] 2. This invention uses an auto-incrementing ID algorithm to generate digital identifiers, which not only considers the uniqueness of the digital identifiers but also reflects the specific circumstances of the assets. Furthermore, it overcomes the problem of overflow caused by a large number of identifiers representing assets through ID reuse.

[0033] 3. In the process of network packet encapsulation, the VXLAN flag bit of the present invention is forcibly set to a fixed flag to indicate that this is a data packet sent by an acceptable terminal. The analysis terminal will determine whether the data packet should be parsed according to the fixed flag bit, which is conducive to the analysis end accurately receiving and parsing the received data packets. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0035] Figure 1 This is a flowchart from Embodiment 1 of the present invention;

[0036] Figure 2 This is a system block diagram in Embodiment 2 of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0038] Example 1: Metadata tagging method based on VXLAN protocol, such as Figure 1 As shown, it includes the following steps:

[0039] S1: Construct an asset mapping table that corresponds one-to-one between cloud asset information and identification numbers;

[0040] S2: Obtain the target asset information corresponding to the target data packet to be forwarded, and match the corresponding identifier number from the asset mapping table as the digital identifier based on the target asset information;

[0041] S3: Write the numerical identifier into the header of the VXLAN protocol and encapsulate the target data packet with VXLAN to obtain a VXLAN data packet with completed metadata marking.

[0042] This invention enriches the information content that the VXLAN tunnel protocol can carry by adopting an asset mapping table model. In business scenarios of multiple cloud platforms, such a mapping model can be used to identify asset information. In other business scenarios, it can also identify other richer information, realizing the association between asset information and traffic data. This overcomes the problem that VXLAN will cause the data metadata to be missing when VXLAN is needed to forward data packets.

[0043] In the asset mapping table, the value (asset) represents the asset's metadata. The asset type differs across different cloud platform types. For example, in OpenStack-like IaaS platforms, the asset is a virtual machine, while in Kubernetes-like containers, the asset is a Pod, and in bare metal environments, the asset is the physical machine. However, the value in the mapping table has a unified format, so it needs to be abstracted. The abstracted value format is as follows:

[0044]

[0045] Here, probeId represents the collector ID, which is globally unique. A collector is associated with a system resource; for example, in OpenStack, a collector is associated with a virtual machine or a physical machine. In Kubernetes, a collector is associated with a Node. Since a collector may collect traffic from multiple VPCs, a flag is also needed to identify the specific scope of the traffic.

[0046] labelType indicates the type of traffic range. For example, in an IaaS platform, labelType is 1, indicating that the traffic comes from a VPC; in Kubernetes, the value is 2, indicating that the traffic comes from a Pod; and the value is 3, indicating that the traffic comes from a physical machine.

[0047] The labelValue represents the range to which the traffic belongs. For example, in an IaaS platform, this value identifies the VPC ID; in Kubernetes, it identifies the Namespace ID; and in a physical machine cluster, it represents the cluster ID.

[0048] Cloud assets are constantly changing, and this invention needs to ensure a one-to-one correspondence between the generated key (identifying number) and value. When synchronizing cloud assets, the asset's metadata is used as the input parameter to generate the key. To manage multiple regions, we rely on the database's auto-incrementing ID algorithm to generate a 24-bit number representing the asset, and an 8-bit number to uniquely identify the region, combining them into a 32-bit number as the key. To prevent the 24-bit ID from overflowing, this invention records a list of available IDs, and when the ID count reaches 24 bits and is about to overflow, it resets and reuses deleted IDs until the IDs are exhausted, reaching the algorithm's limit.

[0049] This invention uses an auto-incrementing ID algorithm to generate digital identifiers, which takes into account both the uniqueness of the digital identifiers and the specific details of the assets. Furthermore, it overcomes the problem of overflow caused by a large number of identifiers representing assets through ID reuse.

[0050] When forwarding data packets based on VXLAN, the analysis terminal cannot distinguish between traffic forwarded by the collector and service VXLAN traffic, leading to inconsistent analysis results. To address this issue, this solution fixes the first 8 VXLAN flag bits in the VXLAN header as 0b 0001 1000 (i.e., 0x18). In certain network environments, this VXLAN flag bit may be blocked, requiring configuration and permission through operational measures. When the analysis terminal receives VXLAN traffic, it first parses the VXLAN flag bits. If the VXLAN flag bit is 0b 0001 1000, it indicates that this is analysis traffic from the collector, and the analysis terminal should analyze it; otherwise, it discards the traffic.

[0051] In the process of network packet encapsulation, the VXLAN flag bit of the present invention is forcibly set to a fixed flag to indicate that this is a data packet sent by an acceptable terminal. The analysis terminal will determine whether the data packet should be parsed based on the fixed flag bit, which is conducive to the analysis end accurately receiving and parsing the received data packets.

[0052] After receiving the VXLAN data packet from the collector, the analyzer parses the key. It then queries the asset mapping table synchronized from the controller to retrieve the asset information represented by that key. The data packet is then matched with the specific cloud asset to generate meaningful analysis results.

[0053] Example 2: A message transmission system based on the VXLAN protocol, such as Figure 2 As shown, it includes cloud API, controller, collector and analyzer, with cloud API and cloud platform settings corresponding one-to-one.

[0054] The system comprises: a cloud API for obtaining cloud asset information from the corresponding cloud platform; a controller for constructing an asset mapping table that corresponds one-to-one between cloud asset information and identification numbers, and synchronously updating the asset mapping table to the collector and analyzer; a collector for collecting business traffic from at least one cloud platform to obtain target data packets and target asset information, matching the corresponding identification numbers from the asset mapping table based on the target asset information as digital identifiers, writing the digital identifiers into the VXLAN protocol header, and encapsulating the target data packets in VXLAN to obtain VXLAN data packets with completed metadata marking; and an analyzer for unpacking and analyzing the received VXLAN data packets based on the asset mapping table.

[0055] In this invention, the asset mapping table is kept synchronized between the collector and the analysis terminal; the same key only has meaning if it has the same meaning. The collector is distributed, which is not conducive to maintaining a centralized table. The analysis terminal is located outside the cloud and cannot access asset information within the cloud. To facilitate table synchronization, a controller needs to be deployed in the cloud. The controller acquires cloud assets in real time and then updates the asset information to the collector and analysis terminal in real time to ensure their asset mapping tables remain synchronized.

[0056] For example, in IaaS platforms, the controller periodically pulls a list of VPCs from the cluster and stores it in memory. Each pull updates the in-memory asset mapping table, which is then broadcast to all collectors and analytics terminals via gRPC for full synchronization.

[0057] In a cluster similar to Kubernetes, the Watch mechanism provided by Kubernetes is used to incrementally obtain changes to the cluster namespace and Pods, then synchronize the changed assets to the asset mapping table, and finally broadcast them to all collectors and analysis terminals.

[0058] In this invention, the controller synchronizes cloud asset information via a cloud API. Cloud API information, including the API address, cloud type, and API authentication information, is added through the controller's API interface. Based on the type of cloud API, synchronization can be categorized into scheduled synchronization and real-time synchronization. Scheduled asset synchronization is performed periodically by the controller, with a full replacement and update after each acquisition. Real-time asset synchronization relies on the cloud API's event listening mechanism, adding or deleting cloud assets based on event types to achieve synchronization.

[0059] Furthermore, this invention sends collection commands to the collector through the API interface exposed by the controller. The collection commands carry the VPC or Namespace to be collected. The collector filters the network card according to the VPC or Namespace to capture packets. A collector is allowed to be in multiple VPCs or Namespaces at the same time, and these packets are marked using VPCs or Namespaces.

[0060] Working principle: This invention uses an asset mapping table to enrich the information content that the VXLAN tunnel protocol can carry. In business scenarios of multiple cloud platforms, this mapping mode can be used to identify asset information. In other business scenarios, it can also identify other richer information, realizing the association between asset information and traffic data. This overcomes the problem that VXLAN will cause the data metadata to be missing when VXLAN is used to forward data packets.

[0061] Furthermore, this invention uses an auto-incrementing ID algorithm to generate digital identifiers, which not only considers the uniqueness of the digital identifiers but also reflects the specific circumstances of the assets. Moreover, it overcomes the problem of overflow caused by a large number of identifiers representing assets through ID reuse.

[0062] In addition, during the network packet encapsulation process, the VXLAN flag bit of the present invention is forcibly set to a fixed flag to indicate that this is a data packet sent by an acceptable terminal. The analysis terminal will determine whether the data packet should be parsed based on the fixed flag bit, which is conducive to the analysis end accurately receiving and parsing the received data packets.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A metadata tagging method based on the VXLAN protocol, characterized in that: Includes the following steps: Construct an asset mapping table that establishes a one-to-one correspondence between cloud asset information and identification numbers, wherein the assets in the asset mapping table represent the metadata of the assets, and the identification numbers are generated using the metadata of the assets as input parameters; Obtain the target asset information corresponding to the target data packet to be forwarded, and match the corresponding identifier number from the asset mapping table based on the target asset information as the digital identifier; The numerical identifier is written into the header of the VXLAN protocol, and the target data packet is encapsulated in VXLAN to obtain a VXLAN data packet with completed metadata marking. The method further includes: generating a fixed tag based on the network configuration parameters corresponding to the analysis end receiving the VXLAN data packet; writing the fixed tag into the first 8 VXLAN flag bits in the VXLAN protocol header and then performing VXLAN encapsulation; and, in a specific network, configuring and allowing the intermediate transmission process based on the fixed tag through operation and maintenance means.

2. The metadata tagging method based on the VXLAN protocol according to claim 1, characterized in that, The digital identifier includes an asset area and a unique identifier area; The asset area represents the asset using a 24-bit number; The unique identifier region is represented by an 8-bit number.

3. The metadata tagging method based on the VXLAN protocol according to claim 2, characterized in that, The numbers in the asset area are generated using an auto-incrementing ID algorithm.

4. The metadata tagging method based on the VXLAN protocol according to claim 3, characterized in that, If an ID in an asset region overflows after exceeding 24 bits, the deleted IDs are reset and reused until the IDs are exhausted.

5. A message transmission system based on the VXLAN protocol, characterized in that, include: Cloud APIs are used to obtain cloud asset information from the corresponding cloud platform. The controller is used to construct an asset mapping table that corresponds one-to-one between cloud asset information and identification numbers, and to synchronously update the asset mapping table to the collector and analyzer. The assets in the asset mapping table represent the metadata of the assets, and the identification numbers are generated using the metadata of the assets as input parameters. The collector is used to collect the business traffic of at least one cloud platform to obtain target data packets and target asset information, and to match the corresponding identification number from the asset mapping table as a digital identifier based on the target asset information, and to write the digital identifier into the header of the VXLAN protocol, and to encapsulate the target data packet with VXLAN to obtain a VXLAN data packet with completed metadata marking. The analyzer is used to unpack and analyze received VXLAN packets based on the asset mapping table. The collector is further configured to: generate a fixed tag based on the network configuration parameters corresponding to the analyzer that receives the VXLAN data packets; encapsulate the fixed tag into the first 8 VXLAN flag bits in the VXLAN protocol header; and, in a specific network, configure and allow intermediate transmission processes based on the fixed tag through operation and maintenance methods.

6. The message transmission system based on the VXLAN protocol according to claim 5, characterized in that, The controller is deployed on a non-business node within the cloud; The controller exposes a RESTful API interface to the outside world; Commands are created via the API interface, and the controller sends control commands to the data acquisition unit and the analyzer via gRPC.

7. The message transmission system based on the VXLAN protocol according to claim 5, characterized in that, The collector is deployed on user service nodes within the cloud.

8. The message transmission system based on the VXLAN protocol according to claim 5, characterized in that, The analyzer is deployed on a non-business node outside the cloud.

Images