Method and system for merging and processing multi-source data streams in cloud environment

Abstract

The invention provides a method and a system for merging and processing multi-source data streams in a cloud environment. The method comprises the following steps of: S1, initializing an annular cache queue according to configuration parameters; the configuration parameters comprise the maximum cache time and the time granularity; the annular cache queue is used for storing data packets from a plurality of collection points; s2, calculating a corresponding initial slot position according to the initial data packet, caching the initial data packet into an annular cache queue according to the slot, and recording cache initial time at the same time; and S3, calculating the cache slot of the data packet according to the time when the new data packet arrives and the cache starting time, and caching the data packet into the corresponding slot. According to the method, the annular cache queue with a certain size is designed according to the maximum cache time and granularity, and the annular cache queue is combined with the slot to ensure that data packets from a plurality of acquisition points can be correctly recombined according to the original occurrence time sequence.

Description

Technical Field

[0001] The present invention relates to the fields of computer network technology and data processing, and in particular to a method and system for merging and processing multi-source data streams in a cloud environment. Background Art

[0002] In traditional network monitoring, data packets carrying services are copied through network devices (switches) or network mirroring devices (TAPs) and then transmitted to the network port of the network monitoring and analysis device for capture, and then analyzed and processed by the analysis device. The time of the data packet is uniformly added by the network monitoring and analysis device when capturing the data packet. In this case, the time sequence of data on a capture port is consistent, which is called a data flow, such as Figure 1 shown.

[0003] In a cloud environment (or virtualized environment), packet capture and analysis processing are often separate. Packet capture occurs in a cloud container (or virtual machine) or on a host, which can be called capture at the capture point. The captured packets are then transmitted to a remote (or even off-cloud) network monitoring and analysis device through a network protocol for unified analysis and processing, which can be called processing at the processing point. Figure 2 Obviously, in this case, the data packets occurring at each capture point are in order, and are time-ordered data streams; at the processing point, multiple data streams are merged, and the data packet times are out of order.

[0004] However, for network monitoring and analysis, it is necessary to ensure that the data is in time order. The data needs to be saved, analyzed, processed, and visualized by time.

[0005] Patent document CN101369869A provides a network monitoring method, a network monitoring device, a line failure prevention system and a computer program for the network monitoring device, aiming to solve the problems of line quality degradation and line interruption caused by rainfall, which is essentially different from the technical method adopted by the present invention.

[0006] The present invention aims to solve the time sequence challenge when merging multiple data streams in a cloud environment, and provides an efficient and accurate data packet processing solution suitable for network monitoring and data analysis applications in a modern cloud computing environment. Summary of the invention

[0007] In view of the defects in the prior art, the object of the present invention is to provide a method and system for merging and processing multi-source data streams in a cloud environment.

[0008] A method for merging and processing multi-source data streams in a cloud environment provided by the present invention includes: Step S1: Initialize a ring buffer queue according to configuration parameters; The configuration parameters include maximum cache time and time granularity; The circular buffer queue is used to store data packets from multiple collection points; Step S2: Calculate the corresponding starting slot position according to the starting data packet, cache the starting data packet into the ring cache queue according to the slot, and record the cache start time; Step S3: Calculate the cache slot of the data packet according to the time of the new data packet and the cache start time, and cache the data packet into the corresponding slot; when the slot position is larger than the cache queue size, drive the output of the cached data packet, and update the start slot position and the start data packet at the same time.

[0009] Preferably, it also includes: Step S4: When caching or outputting data packets, choose whether to sort the data packets according to demand.

[0010] Preferably, the time granularity is used to round the time of the data packet and archive the data packet into a certain time slot.

[0011] Preferably, the calculation formula of the ring buffer queue is: slot_count = max_timespan / time_granularity; Among them, max_timespan is the maximum cache time span, and time_granularity is the time granularity.

[0012] Preferably, the cache slot calculation process of the data packet includes: The time pkt_time of reading the data packet, the cache start time cache_start_time and the time granularity time_granularity; Check cache_start_time == 0. If so, it means no data packet has been received. Update cache_start_time using pkt_time and time_granularity: Divide cache_start_time by the value of time_granularity, round up, and then multiply by time_granularity; Use pkt_time and cache_start_time to calculate the time difference pkt_delta_time. The value of pkt_delta_time is: pkt_time minus cache_start_time. Use pkt_delta_time and time_granularity to calculate the slot position pkt_slot. The value of pkt_slot is: pkt_delta_time is divided by the value of time_granularity and rounded up.

[0013] Preferably, the data packets are divided into cached data packets, output data packets and discarded data packets according to the value of pkt_slot and then processed respectively: Read pkt_slot, slot_count. If pkt_slot<0, the data packet is classified as a discarded data packet for processing; If pkt_slot ≥ 0 and pkt_slot ≥ slot_count, the data packet is classified as an output data packet for processing; otherwise, the data packet is classified as a cached data packet for processing.

[0014] Preferably, the processing of caching data packets includes: Read in pkt_slot, cache_start_slot, slot_count and cache_ring; Among them, cache_start_slot is the cache starting slot; Use pkt_slot, cache_start_slot, slot_count to calculate index. Index is actually the physical storage location of the slot in the ring cache cache_ring. The value of index is: (pkt_slot + cache_start_slot) % slot_count; Among them, % is the modulo remainder operation; Add the data packet to the array of cache_ring[index]. Use index to get the storage unit of cache_ring[index] and add the data packet to the array of the storage unit.

[0015] Preferably, the processing of the output data packet includes: Read in pkt_slot, slot_count, cache_ring, cache_start_slot, pkt_time, and time_granularity; According to pkt_slot, slot_count and calculate export_count, the value of export_count is: (pkt_slot - slot_count + 1) % slot_count; Among them, % is the modulo remainder operation; Export export_count units of cache_ring starting from cache_slot_start; Update cache_start_slot according to cache_slot_star, export_count, and slot_count. The value of cache_start_slot is: (cache_slot_star + export_count) % slot_count; Update cache_start_time using pkt_time and time_granularity.

[0016] A system for merging and processing multi-source data streams in a cloud environment provided by the present invention includes: Module M1: Initialize a ring buffer queue according to the configuration parameters; The configuration parameters include maximum cache time and time granularity; The circular buffer queue is used to store data packets from multiple collection points; Module M2: Calculate the corresponding starting slot position according to the starting data packet, cache the starting data packet into the ring cache queue according to the slot, and record the cache start time; Module M3: Calculate the cache slot of the data packet according to the time of the new data packet and the cache start time, and cache the data packet into the corresponding slot; when the slot position is larger than the cache queue size, drive the output of the cached data packet and update the start slot position and the start data packet at the same time.

[0017] Preferably, it also includes: Module M4: When caching or outputting data packets, choose whether to sort the data packets according to the needs.

[0018] Preferably, the time granularity is used to round the time of the data packet and archive the data packet into a certain time slot.

[0019] Preferably, the calculation formula of the ring buffer queue is: slot_count = max_timespan / time_granularity; Among them, max_timespan is the maximum cache time span, and time_granularity is the time granularity.

[0020] Preferably, the cache slot calculation process of the data packet includes: The time pkt_time of reading the data packet, the cache start time cache_start_time and the time granularity time_granularity; Check cache_start_time == 0. If so, it means no data packet has been received. Update cache_start_time using pkt_time and time_granularity: Divide cache_start_time by the value of time_granularity, round up, and then multiply by time_granularity; Use pkt_time and cache_start_time to calculate the time difference pkt_delta_time. The value of pkt_delta_time is: pkt_time minus cache_start_time. Use pkt_delta_time and time_granularity to calculate the slot position pkt_slot. The value of pkt_slot is: pkt_delta_time is divided by the value of time_granularity and rounded up.

[0021] Preferably, the data packets are divided into cached data packets, output data packets and discarded data packets according to the value of pkt_slot and then processed respectively: Read pkt_slot, slot_count. If pkt_slot<0, the data packet is classified as a discarded data packet for processing; If pkt_slot ≥ 0 and pkt_slot ≥ slot_count, the data packet is classified as an output data packet for processing; otherwise, the data packet is classified as a cached data packet for processing.

[0022] Preferably, the processing of caching data packets includes: Read in pkt_slot, cache_start_slot, slot_count and cache_ring; Among them, cache_start_slot is the cache starting slot; Use pkt_slot, cache_start_slot, slot_count to calculate index. Index is actually the physical storage location of the slot in the ring cache cache_ring. The value of index is: (pkt_slot + cache_start_slot) % slot_count; Among them, % is the modulo remainder operation; Add the data packet to the array of cache_ring[index]. Use index to get the storage unit of cache_ring[index] and add the data packet to the array of the storage unit.

[0023] Preferably, the processing of the output data packet includes: Read in pkt_slot, slot_count, cache_ring, cache_start_slot, pkt_time, and time_granularity; According to pkt_slot, slot_count and calculate export_count, the value of export_count is: (pkt_slot - slot_count + 1) % slot_count; Among them, % is the modulo remainder operation; Export export_count units of cache_ring starting from cache_slot_start; Update cache_start_slot according to cache_slot_star, export_count, and slot_count. The value of cache_start_slot is: (cache_slot_star + export_count) % slot_count; Update cache_start_time using pkt_time and time_granularity.

[0024] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention designs a circular cache queue of a certain size according to the maximum cache time and granularity, and ensures that data packets from multiple collection points can be correctly reassembled according to the original time sequence of their occurrence by combining the circular cache queue and slot.

[0025] 2. The present invention calculates the corresponding cache slot according to the time of the first packet, caches the data into a circular queue by slot, records the cache start time at the same time, and then continuously calculates the cache slot of the newly arrived data packet according to the time of the newly arrived data packet and the cache start time, and caches the data packet into the corresponding slot; when the slot position is larger than the cache queue size, drives the output of the cached data packet, and updates the starting slot position and the starting data packet at the same time, thereby improving the overall performance of the system.

[0026] 3. The present invention can flexibly sort when caching data packets or when outputting data packets according to actual needs, and has good practicality.

[0027] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 It is a schematic diagram of mirroring network data packets from a switch to a monitoring and analysis device in the present invention.

[0029] Figure 2 A schematic diagram of capturing data from a cloud environment to a monitoring and analysis device in the present invention.

[0030] Figure 3 It is a schematic diagram of the main process of the present invention.

[0031] Figure 4 It is a schematic diagram of the process of calculating pkt_slot in the present invention.

[0032] Figure 5 The figure is a flow chart of determining cache, output and discard according to pkt_slot in the present invention.

[0033] Figure 6 It is a schematic diagram of the processing flow of caching data packets in the present invention.

[0034] Figure 7 It is a schematic diagram of the processing flow of outputting data packets in the present invention. DETAILED DESCRIPTION

[0035] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0036] In order to solve the problem of aggregating multiple data sources formed by multiple collection points in a cloud environment into a single data stream for processing, the present invention first designs a circular cache queue of a certain size according to the maximum cache time and granularity, and then calculates the corresponding cache slot according to the first packet time, caches the data into the circular queue by slot, and records the cache start time at the same time. Then, the cache slot of the newly arrived data packet is continuously calculated according to the time of the newly arrived data packet and the start time of the cache, and the data packet is cached into the corresponding slot. When the slot position is larger than the cache queue size, the output of the cached data packet is driven, and the starting slot position and the starting data packet are updated at the same time. At the same time, according to actual needs, it can be designed to sort when caching data packets, or to sort when outputting data packets.

[0037] Reference Figure 3 As shown, a method for merging and processing multi-source data streams in a cloud environment, the steps are as follows: Step 1: Load the configuration.

[0038] The present invention includes the following two configurations: max_timespan: The maximum cache time span. It is limited by the maximum processing delay that the system can accept.

[0039] time_granularity: Time granularity. It is used to round the time of data packets and archive the data packets into a certain time slot. The commonly used granularity is seconds, minutes, hours, etc.

[0040] time_granularity is the smallest unit of the ring, which affects the processing frequency and latency. The smaller the granularity, the higher the processing frequency and the smaller the latency. On the other hand, due to memory limitations, the smaller the granularity, the more memory is required.

[0041] Step 2: Initialize the data packet cache. The data packet cache of the present invention includes the following parts: slot_count: max_timespan divided by time_granularity, rounded to the nearest integer.

[0042] cache_ring: Use a circular cache, the size of the ring is slot_count. Each storage unit of cache_ring is an array, which is used to cache the data packets of the current slot.

[0043] The ring uses modulo operations, so the entire ring is used cyclically. cache_start_time can locate the starting position, so if the slot is full, the next slot will continue to be used modulo, but the starting position will be moved forward.

[0044] cache_start_slot: Set the initial value to 0.

[0045] cache_start_time cache start time: Set the initial value to 0, indicating that no data packets have been received.

[0046] Step 3: Determine whether the program has exited. This can be achieved using Boolean variables, or synchronization mechanisms such as Signal or Mutex.

[0047] Step 4: Read the time of the data packet pkt_time and calculate pkt_slot.

[0048] pkt_time: the time of the data packet.

[0049] pkt_slot: is calculated based on pkt_time, cache_start_time and time_granularity.

[0050] Reference Figure 4 As shown, the specific process is: Step 4.1: Read in pkt_time, cache_start_time, time_granularity.

[0051] Step 4.2: Check if cache_start_time == 0. This value == 0 means that no data packet has been received.

[0052] Step 4.3: Update cache_start_time using pkt_time and time_granularity. Actually, cache_start_time is initialized based on the first data packet received and regularized according to time_granularity.

[0053] The specific calculation method is: cache_start_time divided by the value of time_granularity, then rounded up, and then multiplied by time_granularity.

[0054] Step 4.4: Calculate the time difference of pkt_delta_time using pkt_time and cache_start_time. The value of pkt_delta_time is: pkt_time minus cache_start_time. This value may be negative, indicating that the new data packet is earlier than the cache time and will be processed in step 5.

[0055] Step 4.5: Calculate the pkt_slot slot position using pkt_delta_time and time_granularity. The value of pkt_slot is: pkt_delta_time divided by the value of time_granularity, rounded up. Again, this value may be negative, indicating that the new packet is earlier than the cache time and will be processed in step 5.

[0056] Step 5: Determine cache, output, or discard based on pkt_slot. The judgment process also requires the introduction of slot_count, such as Figure 5 shown.

[0057] Step 5.1: Read in pkt_slot, slot_count.

[0058] Step 5.2: Check if pkt_slot<0.

[0059] Yes: The packet time is earlier than the current cache start time, so it needs to be discarded. Refer to step 8.

[0060] No: Indicates that the data packet needs further judgment.

[0061] Step 5.3: Check if pkt_slot>= slot_count.

[0062] Yes: The packet time is greater than the maximum time range of the cache, so "output" processing is required. Refer to step 7.

[0063] No: It means that the packet time is within the cache time range. Only the "cache" process is required. Refer to step 6.

[0064] Step 6: Processing of cached data packets, such as Figure 6 shown.

[0065] Step 6.1: Read in pkt_slot, cache_start_slot, slot_count, cache_ring.

[0066] Step 6.2: Use pkt_slot, cache_start_slot, and slot_count to calculate index. Index is actually the physical storage location of the slot in the cache_ring.

[0067] The value of index is: (pkt_slot + cache_start_slot) % slot_count, where % is the modulo remainder operation.

[0068] Step 6.3: Add the data packet to the array of cache_ring[index]. Use index to obtain the storage unit (data packet array) of cache_ring[index], and add the data packet to the array of the storage unit. The addition operation here can be a simple direct append operation at the end of the array, or an insertion sort can be used to keep the time order of the array. In the case of a simple append operation, the current append process has no overhead, but the data packets need to be sorted by time when they are output, which brings processing delays; but asynchronous threads can be used to reduce the burden. In the case of insertion sort, the sorting overhead is amortized to the processing process of each packet, which increases the time overhead of each packet processing, but the delay of outputting the data packet is small. The choice of the two addition operations can be selected according to the actual scenario.

[0069] Step 7: Processing of output data packets, such as Figure 7 shown.

[0070] Step 7.1: Read in pkt_slot, slot_count, cache_ring, cache_start_slot, pkt_time, time_granularity.

[0071] Step 7.2: Calculate export_count based on pkt_slot, slot_count and.

[0072] export_count takes the following values: (pkt_slot - slot_count + 1) % slot_count, where % is the modulo remainder operation.

[0073] Step 7.3: Output the export_count units of cache_ring starting from cache_slot_start. Using the standard iterative method of the ring queue, the data of export_count storage units can be obtained starting from cache_slot_start. The data of each unit is a data packet list, so the data obtained is a set of data packet lists. If step 6.3 is an insertion sort operation, there is no need to sort at this time, just splice these data packet lists into a large list in order; if step 6.3 is a simple append operation, sorting is required at this time.

[0074] Step 7.4: Update according to cache_slot_star, export_count, slot_count.

[0075] cache_start_slot.

[0076] The value of cache_start_slot is: (cache_slot_star + export_count) % slot_count.

[0077] Step 7.5: Update cache_start_time using pkt_time and time_granularity. This step is calculated in the same way as step 4.3.

[0078] Step 8: Processing of discarded data packets: The present invention does not perform special processing on discarded data packets, and conventional operations such as counting and outputting logs can be performed.

[0079] The present invention also provides a system for merging and processing multi-source data streams in a cloud environment. The system for merging and processing multi-source data streams in a cloud environment can be implemented by executing the process steps of the method for merging and processing multi-source data streams in a cloud environment, that is, those skilled in the art can understand the method for merging and processing multi-source data streams in a cloud environment as a preferred implementation of the system for merging and processing multi-source data streams in a cloud environment.

[0080] Specifically, a system for merging and processing multi-source data streams in a cloud environment includes: Module M1: Initialize a ring buffer queue according to the configuration parameters; The configuration parameters include maximum cache time and time granularity; The circular buffer queue is used to store data packets from multiple collection points; Module M2: Calculate the corresponding starting slot position according to the starting data packet, cache the starting data packet into the ring cache queue according to the slot, and record the cache start time; Module M3: Calculate the cache slot of the data packet according to the time of the new data packet and the cache start time, and cache the data packet into the corresponding slot; when the slot position is larger than the cache queue size, drive the output of the cached data packet and update the start slot position and the start data packet at the same time.

[0081] Also includes: Module M4: When caching or outputting data packets, choose whether to sort the data packets according to requirements.

[0082] The time granularity is used to round the time of a data packet and file the data packet into a certain time slot.

[0083] The calculation formula of the ring buffer queue is: slot_count = max_timespan / time_granularity; Among them, max_timespan is the maximum cache time span, and time_granularity is the time granularity.

[0084] The cache slot calculation process for a data packet includes: The time pkt_time of reading the data packet, the cache start time cache_start_time and the time granularity time_granularity; Check cache_start_time == 0. If so, it means no data packet has been received. Update cache_start_time using pkt_time and time_granularity: Divide cache_start_time by the value of time_granularity, round up, and then multiply by time_granularity; Use pkt_time and cache_start_time to calculate the time difference pkt_delta_time. The value of pkt_delta_time is: pkt_time minus cache_start_time. Use pkt_delta_time and time_granularity to calculate the slot position pkt_slot. The value of pkt_slot is: pkt_delta_time is divided by the value of time_granularity and rounded up.

[0085] According to the value of pkt_slot, the data packets are divided into cached data packets, output data packets and discarded data packets, and then processed separately: Read pkt_slot, slot_count. If pkt_slot<0, the data packet is classified as a discarded data packet for processing; If pkt_slot ≥ 0 and pkt_slot ≥ slot_count, the data packet is classified as an output data packet for processing; otherwise, the data packet is classified as a cached data packet for processing.

[0086] The process of caching data packets includes: Read in pkt_slot, cache_start_slot, slot_count and cache_ring; Among them, cache_start_slot is the cache starting slot; Use pkt_slot, cache_start_slot, slot_count to calculate index. Index is actually the physical storage location of the slot in the ring cache cache_ring. The value of index is: (pkt_slot + cache_start_slot) % slot_count; Among them, % is the modulo remainder operation; Add the data packet to the array of cache_ring[index]. Use index to get the storage unit of cache_ring[index] and add the data packet to the array of the storage unit.

[0087] The processing of outgoing packets includes: Read in pkt_slot, slot_count, cache_ring, cache_start_slot, pkt_time, and time_granularity; According to pkt_slot, slot_count and calculate export_count, the value of export_count is: (pkt_slot - slot_count + 1) % slot_count; Among them, % is the modulo remainder operation; Export export_count units of cache_ring starting from cache_slot_start; Update cache_start_slot according to cache_slot_star, export_count, and slot_count. The value of cache_start_slot is: (cache_slot_star + export_count) % slot_count; Update cache_start_time using pkt_time and time_granularity.

[0088] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0089] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for merging and processing multi-source data streams in a cloud environment, characterized in that: include: Step S1: Initialize a ring buffer queue according to configuration parameters; The configuration parameters include maximum cache time and time granularity; The circular buffer queue is used to store data packets from multiple collection points; Step S2: Calculate the corresponding starting slot position according to the starting data packet, cache the starting data packet into the ring cache queue according to the slot, and record the cache start time; Step S3: Calculate the cache slot of the data packet according to the time of the new data packet and the cache start time, and cache the data packet into the corresponding slot; when the slot position is larger than the cache queue size, drive the output of the cached data packet, and update the start slot position and the start data packet at the same time.

2. The method for merging and processing multi-source data streams in a cloud environment according to claim 1, characterized in that: Also includes: Step S4: When caching or outputting data packets, choose whether to sort the data packets according to demand.

3. The method for merging and processing multi-source data streams in a cloud environment according to claim 1, characterized in that: The time granularity is used to round the time of a data packet and file the data packet into a certain time slot.

4. The method for merging and processing multi-source data streams in a cloud environment according to claim 1, characterized in that: The calculation formula of the ring buffer queue is: slot_count = max_timespan / time_granularity; Among them, max_timespan is the maximum cache time span, and time_granularity is the time granularity.

5. The method for merging and processing multi-source data streams in a cloud environment according to claim 4, characterized in that: The cache slot calculation process for a data packet includes: The time pkt_time of reading the data packet, the cache start time cache_start_time and the time granularity time_granularity; Check cache_start_time == 0. If so, it means no data packet has been received. Update cache_start_time using pkt_time and time_granularity: Divide cache_start_time by the value of time_granularity, round up, and then multiply by time_granularity; Use pkt_time and cache_start_time to calculate the time difference pkt_delta_time. The value of pkt_delta_time is: pkt_time minus cache_start_time. Use pkt_delta_time and time_granularity to calculate the slot position pkt_slot. The value of pkt_slot is: pkt_delta_time is divided by the value of time_granularity and rounded up.

6. The method for merging and processing multi-source data streams in a cloud environment according to claim 5, characterized in that: According to the value of pkt_slot, the data packets are divided into cached data packets, output data packets and discarded data packets, and then processed separately: Read pkt_slot, slot_count. If pkt_slot < 0, the data packet is classified as a discarded data packet for processing; If pkt_slot ≥ 0 and pkt_slot ≥ slot_count, the data packet is classified as an output data packet for processing; otherwise, the data packet is classified as a cached data packet for processing.

7. The method for merging and processing multi-source data streams in a cloud environment according to claim 6, characterized in that: The process of caching data packets includes: Read in pkt_slot, cache_start_slot, slot_count and cache_ring; Among them, cache_start_slot is the cache starting slot; Use pkt_slot, cache_start_slot, slot_count to calculate index. Index is actually the physical storage location of the slot in the ring cache cache_ring. The value of index is: (pkt_slot + cache_start_slot) % slot_count; Among them, % is the modulo remainder operation; Add the data packet to the array of cache_ring[index]. Use index to get the storage unit of cache_ring[index] and add the data packet to the array of the storage unit.

8. The method for merging and processing multi-source data streams in a cloud environment according to claim 7, characterized in that: The processing of outgoing packets includes: Read in pkt_slot, slot_count, cache_ring, cache_start_slot, pkt_time, and time_granularity; According to pkt_slot, slot_count and calculate export_count, the value of export_count is: (pkt_slot - slot_count + 1) % slot_count; Among them, % is the modulo remainder operation; Export export_count units of cache_ring starting from cache_slot_start; Update cache_start_slot according to cache_slot_star, export_count, and slot_count. The value of cache_start_slot is: (cache_slot_star + export_count) % slot_count; Update cache_start_time using pkt_time and time_granularity.

9. A system for merging and processing multi-source data streams in a cloud environment, characterized in that: include: Module M1: Initialize a ring buffer queue according to the configuration parameters; The configuration parameters include maximum cache time and time granularity; The circular buffer queue is used to store data packets from multiple collection points; Module M2: Calculate the corresponding starting slot position according to the starting data packet, cache the starting data packet into the ring cache queue according to the slot, and record the cache start time; Module M3: Calculate the cache slot of the data packet according to the time of the new data packet and the cache start time, and cache the data packet into the corresponding slot; when the slot position is larger than the cache queue size, drive the output of the cached data packet and update the start slot position and the start data packet at the same time.

10. The system for merging and processing multi-source data streams in a cloud environment according to claim 9, characterized in that: Also includes: Module M4: When caching or outputting data packets, choose whether to sort the data packets according to the needs.

Images