Processing Method, Device, Equipment and Storage Medium for Data Stream Aggregation or Sharding

Abstract

An embodiment of the present invention discloses a method, apparatus, device, and storage medium for processing data stream aggregation or fragmentation. The method includes: obtaining a data stream set containing data streams; determining the data stream classification to which each data stream in the data stream set belongs according to the same period of each data stream in the data stream set; aggregating or fragmenting each data stream according to the data stream classification and corresponding preset processing rules to process it into a new data stream set. In the embodiment of the present invention, the data stream classification is determined according to the same period of each data stream in the data stream set, and the data stream can be accurately classified; according to the classification situation, the data stream is aggregated or fragmented according to different processing rules to process it into a new data stream set with unified priority or unified priority and the data stream size meeting the scale threshold, solving the problem that the model cannot process in the scenario of large-scale data streams, preventing network jitter, and increasing the determinacy and reliability of the network.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of network technologies, and in particular, to a method, apparatus, device, and storage medium for processing data stream aggregation or fragmentation. Background Art

[0002] In the prior art, the end-to-end delay of periodic traffic flows depends on the sending time point of the specified traffic flows processed in the switch. Therefore, whether the periodic traffic flows meet the application requirements depends on whether the periodic traffic flows can be processed. The time-triggered processing was first modeled by Wilfried Steiner of TTTech with linear constraints at the top international conference in the real-time field (Real-time System Symposium, abbreviated as RTSS) in 2010. This model has been used until now and has become the benchmark model for time-triggered processing technology. It elaborates on the linear constraints that a feasible time-triggered processing should satisfy from six aspects. Specifically: (1) Conflict-free constraint for multi-period traffic flows: For any two traffic flows fi and fj, packets cannot be sent to a data link [vk, vl] at the same time, otherwise a conflict will occur on this link. (2) Path-dependency constraint: For any traffic flow fi, when it is transmitted on two adjacent links [vx, vj] and [vj, vy], it is always received by node vj first and then forwarded and sent out from vj. Let pdelay[vx, vj] represent the sending processing time of node vx, and ldelay[vx, vj] represent the propagation delay on the data link [vx, vj]. (3) Cache occupancy constraint: Similar to (2), this constraint describes that any traffic flow fi cannot occupy the cache of node vj for too long. Let the longest time for a traffic flow to occupy the cache of node vj be membound[vj, vy]. (4) Simultaneous forwarding constraint: This constraint describes that for any traffic flow fi, when it is multicast from node vj to multiple links, it can be constrained to send to multiple links simultaneously. (5) End-to-end delay constraint: This constraint describes that for any traffic flow fi, the end-to-end delay cannot exceed the worst delay specified by the application requirements. (6) Other constraints to meet application requirements: This type of constraint is related to specific application requirements. For example, traffic flow fa needs to be sent at least Δ time after traffic flow fb is sent.

[0003] However, in the prior art, when applying the above model to solve large-scale traffic, the input of large-scale traffic into this model can cause problems where the model cannot be processed, resulting in the network power increasing due to frequent switching of gating, network jitter occurring, increasing network delay and the complexity of the communication network, thereby increasing the load, reducing efficiency, and the determinacy and reliability of the network. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device and storage medium for processing data stream aggregation or sharding, which can solve the problem that the model cannot process in the scenario of large-scale data streams, prevent network jitter, reduce network latency and the complexity of network communication, and at the same time reduce network load, and increase the determinacy and reliability of the network.

[0005] According to one aspect of the present invention, an embodiment of the present invention provides a method for processing data stream aggregation or sharding, the method comprising:

[0006] Obtaining a data stream set containing data streams;

[0007] Determining data stream classification according to the situation that the periods of the data streams in the data stream set are the same;

[0008] Aggregating or sharding each data stream according to the data stream classification and corresponding preset processing rules to process into a new data stream set, wherein the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set meet a scale threshold.

[0009] According to another aspect of the present invention, an embodiment of the present invention further provides a device for processing data stream aggregation or sharding, the device comprising:

[0010] A set acquisition module, configured to obtain a data stream set containing data streams;

[0011] A classification determination module, configured to determine data stream classification according to the situation that the periods of the data streams in the data stream set are the same;

[0012] A new data stream set determination module, configured to perform aggregation or splitting according to the data stream classification and corresponding preset processing rules to process into a new data stream set, wherein the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set meet a scale threshold.

[0013] According to another aspect of the present invention, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor. When executed by the at least one processor, the computer program enables the at least one processor to execute the data stream aggregation or sharding processing method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium storing computer instructions for implementing the data stream aggregation or sharding processing method according to any embodiment of the present invention when executed by a processor.

[0018] Through the above technical solutions of the embodiments of the present invention, by obtaining a data stream set containing data streams and determining data stream classification according to the same period of each data stream in the data stream set, it is possible to accurately classify data streams with the same period and data streams with different periods, facilitating subsequent processing of data streams; according to the classification of data streams, each data stream within the corresponding data stream classification is aggregated or sharded according to different processing rules to be processed into a new data stream set. Among them, the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set meet the scale threshold, which can solve the problem that the model cannot handle in the scenario of large-scale data streams, prevent network jitter, reduce network latency and the complexity of network communication, and at the same time reduce network load, increasing the determinacy and reliability of the network.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0021] Figure 1 It is a flowchart of a data stream aggregation or sharding processing method provided by an embodiment of the present invention;

[0022] Figure 2 It is a flowchart of another data stream aggregation or sharding processing method provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic flowchart of yet another data stream aggregation or sharding processing method provided by an embodiment of the present invention;

[0024] Figure 4 It is a structural block diagram of a processing device for data stream aggregation or sharding provided by an embodiment of the present invention;

[0025] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0026] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data used may be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] The present invention can be used for technical implementations that require traffic aggregation or sharding scheduling in industrial Internet scenarios such as Time-Sensitive Network (TSN), 5G, and deterministic networks.

[0029] In one embodiment, Figure 1 It is a flowchart of a processing method for data stream aggregation or sharding provided by an embodiment of the present invention. This embodiment is applicable to the situation of aggregating or splitting periodic data streams. This method can be executed by a processing device for data stream aggregation or sharding. The processing device for data stream aggregation or sharding can be implemented in the form of hardware and / or software, and the processing device for data stream aggregation or sharding can be configured in an electronic device. As Figure 1 shown, the method includes:

[0030] S110. Obtain a data stream set containing data streams.

[0031] Among them, the data stream can also be called the service flow, which refers to the data stream related to the communication network. The data stream can be a periodically received data stream or a periodically sent related data stream.

[0032] In this embodiment, a data stream set including multiple data streams can be obtained from the communication network. Each data stream has corresponding data stream attribute information. The attribute information of the data stream can include the period of the data stream, the size of the data stream, the priority of the data stream, and the protection bandwidth between each data stream.

[0033] In one embodiment, the data stream has at least one of the following attribute information: period, size, priority, protection bandwidth.

[0034] Among them, the size of the data stream can also be called the length of the data stream. The protection bandwidth between data streams is a constant, which can be used to ensure the conflict-free constraint between multi-period data streams, and there is no overlapping phenomenon between data streams. Exemplarily, there are two existing data streams a and b, and there is a certain interval between data stream a and data stream b.

[0035] In this embodiment, the attribute information of each data stream in the data stream set includes at least one of the period of the data stream, the size of the data stream, the priority of the data stream, and the protection bandwidth between each data stream. It should be noted that the priority of the data stream can be customized by the user or set accordingly according to experimental experience. This embodiment does not limit this here. The priority of the data stream is identified by a corresponding value, and the level of the priority can be defined accordingly according to the identification specification of the value. This embodiment does not limit this here.

[0036] S120. Determine the data stream classification according to the situation that the periods of each data stream in the data stream set are the same.

[0037] Among them, the situation where the periods are the same can be the situation where the periods of each data stream in the data stream set are all the same, or the situation where the periods of each data stream in the data stream set are different. This embodiment does not limit this here. The data stream classification can be understood as the data stream classification situation obtained according to whether the periods of each data stream in the data stream set are the same. The data stream classification can be to classify the data streams with the same period in the data stream set into one category, or to classify the data streams with different periods in the data stream set into one category.

[0038] In this embodiment, the periods of the data streams in the data stream set can be compared to determine whether the periods of the data streams in the data stream set are the same according to the period comparison results, and the data streams with the same period in the data stream set can be grouped into one category, and the data streams with different periods in the data stream set can be grouped into one category; alternatively, corresponding data classification can be performed on the data streams in the data stream set by adopting corresponding algorithms, such as decision tree algorithms, classifiers, etc., to obtain the classification results of the data streams.

[0039] S130. Aggregate or fragment each data stream according to the data stream classification and the corresponding preset processing rules to process it into a new data stream set, where the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set meet the scale threshold.

[0040] Among them, the preset processing rules can be understood as the relevant processing rules for aggregating or splitting the data streams with the same period and the data streams with different periods. It should be noted that after aggregating or splitting the data streams with the same period and the data streams with different periods, the size of the data stream will change to a certain extent compared with that before processing, and the size of the data stream may increase or decrease. The new data stream set can be composed of the data streams or data stream sets obtained by aggregating or splitting the data streams with the same period and the data streams with different periods.

[0041] In this embodiment, when the data stream classification is that the periods of the data streams in the data stream set are the same, the data streams with the same priority can be aggregated according to the priority of the data streams to obtain the first new data stream set in the case of the same period; when the data stream classification is that the periods of the data streams in the data stream set are different, the data stream set with the smallest period and at least one data stream set with a non-smallest period in the second type of data streams can be determined according to the period of the data stream, and the data streams can be aggregated or fragmented according to the priorities of the data streams in the data stream set with the smallest period, the priorities, sizes and scale thresholds of the data streams in at least one data stream set corresponding to the non-smallest period to obtain the second new data stream set in the case of the second type of data streams. It should be noted that the priorities of the data streams in the first new data stream set are unified, or the priorities of the data streams in the second new data stream set are unified and the sizes of the data streams need to meet the scale threshold.

[0042] In one embodiment, the scale threshold is (1 + α) * (T0 - S0), where the value range of α is 0 ≤ α ≤ 1, which represents an adjustment factor, T0 represents the smallest period, and S0 represents the reference value size of the data stream under the smallest period.

[0043] In this embodiment, the scale threshold refers to the maximum threshold of the size of a data stream and / or a set of data streams, which can be expressed by the formula (1 + α) * (T0 - S0), where α ranges from 0 ≤ α ≤ 1 and represents an adjustment factor, T0 represents the minimum period of the data streams in the currently obtained set of data streams, and S0 represents the reference value size of the data stream under the minimum period. The reference value size can be expressed by the formula where T0 represents the minimum period, represents the size of the data stream with the same priority P at the period T0 j of the data stream, and P j represents the priority of the data stream j, γ represents the protection bandwidth between data streams, and J is a set of data streams.

[0044] The above technical solution of the embodiment of the present invention can accurately classify data streams with the same period and data streams with different periods by obtaining a set of data streams containing data streams and determining the data stream classification according to the situation that the periods of the data streams in the set of data streams are the same, which is convenient for subsequent processing of data streams; according to the classification of data streams, according to different processing rules, the data streams in the corresponding data stream classification are aggregated or fragmented to process into a new set of data streams. Among them, the priorities of the data streams in the new set of data streams are unified, or the priorities of the data streams in the new set of data streams are unified and the sizes of the data streams in the new set of data streams meet the scale threshold, which can solve the problem that the model cannot be processed in the scenario of large-scale data streams, prevent network jitter, reduce network latency and the complexity of network communication, and at the same time reduce network load, and increase the determinacy and reliability of the network.

[0045] In one embodiment, Figure 2 FIG. is a flowchart of another method for processing data stream aggregation or fragmentation provided by an embodiment of the present invention. On the basis of the above embodiments, this embodiment further refines the determination of data stream classification according to the situation that the periods of the data streams in the set of data streams are the same, and the aggregation or fragmentation of each data stream according to the data stream classification and the corresponding preset processing rules to process into a new set of data streams, as Figure 2 shown. The method for processing data stream aggregation or fragmentation in this embodiment may specifically include the following steps:

[0046] S210. Obtain a set of data streams containing data streams.

[0047] S220. Compare the periods of each data stream in the set of data streams one by one.

[0048] In this embodiment, the periods of each data stream in the set of data streams can be compared one by one through a hash comparison method or a polling comparison method, and according to the comparison results of the periods of each data stream in the set of data streams, the data streams are classified accordingly.

[0049] S230. If the periods of the data streams in the data stream set are the same, group the data streams with the same period into the first type of data streams.

[0050] Among them, the first type of data streams refers to the data stream classification obtained when the periods of the data streams in the data stream set are the same.

[0051] In this embodiment, compare the periods of the data streams in the data stream set one by one. If the periods between the data streams are equal to the same value, it can be considered that the periods of the data streams in the current data stream set are the same, and group the data streams with the same period into the first type of data streams. Exemplarily, if the periods of the data streams in the data stream set are the same, it can be expressed as T1 = T2 =... = T N , where T1, T2, T N respectively represent the period of the first data stream, the period of the second data stream,..., the period of the Nth data stream. In this embodiment, for any data stream, if the period size of the data stream does not change with time, that is, they are constants, the data stream at this time can be called a periodic data stream.

[0052] S240. If the periods of the data streams in the data stream set are different, group the data streams with different periods into the second type of data streams.

[0053] Among them, the second type of data streams refers to the data stream classification obtained when the periods of the data streams in the data stream set are different.

[0054] In this embodiment, compare the periods of the data streams in the data stream set one by one. If the periods between the data streams are different, it can be considered that the periods of the data streams in the current data stream set are different, and group the data streams with different periods into the second type of data streams. Exemplarily, if the periods are different, sort the periods from small to large, and it can be expressed as T1 ≤ T2 ≤... ≤ T y , y ≤ N, where T1, T2, T y respectively represent the period of the first data stream, the period of the second data stream,..., the period of the yth data stream, and N represents the N data streams in the data stream set. In this embodiment, if there are different periods of the data streams, it can be called a multi-period service flow set at this time.

[0055] S250. In the case where the data streams are classified as the first type of data streams, aggregate the data streams with the same priority in the first type of data streams according to the priority of the data streams to obtain the first new data stream set in the case of the first type of data streams.

[0056] Among them, the first new data stream set refers to a new data stream set obtained by aggregating data streams with the same priority when the periods of the data streams in the data stream set are the same.

[0057] In this embodiment, when the data stream is classified as a first-class data stream, that is, when the periods of the data streams in the data stream set are the same, the data streams with the same priority in the first-class data stream can be aggregated according to the priority of the data stream to obtain the first new data stream set in the case of the first-class data stream. Specifically, the data streams in the first-class data stream can be divided according to their priorities to obtain at least one priority set, and each data stream in the priority set has the same priority. Then, the data streams in the priority set are aggregated to obtain the corresponding first new data stream set in the case of the first-class data stream.

[0058] It should be noted that when the periods of the data streams in the data stream set are the same, the necessary condition for N data streams to be schedulable is: Among them, S i represents the size of the data stream, N represents the total number of data streams, γ represents the protection bandwidth between data streams, and T1 represents the period of the data stream.

[0059] In one embodiment, aggregating the data streams with the same priority in the first-class data stream according to the priority of the data stream to obtain the first new data stream set in the case of the first-class data stream includes:

[0060] Dividing the data streams according to their priorities to obtain at least one priority set; among them, the data streams in the priority set have the same priority;

[0061] Aggregating the data streams in the priority set to obtain the first aggregated data stream;

[0062] Denoting the set formed by all the first aggregated data streams as the first new data stream set.

[0063] Among them, the first aggregated data stream can be understood as an aggregated data stream obtained by aggregating data streams with the same priority.

[0064] In this embodiment, each data stream is divided according to the priority of the data stream to obtain at least one priority set. The data streams within each priority set have the same priority. Then, the data streams in the priority set are aggregated to obtain a first aggregated data stream, and the set formed by all the first aggregated data streams is denoted as the first new data stream set. It should be noted that a data stream sequence can be generated by sorting in descending order of priority or in ascending order accordingly, so as to divide the data streams with the same priority into the same priority set. Exemplarily, the data streams are sorted in descending order of priority according to the priority, and the set with priority P i is represented as Then the set F is divided into M subsets M ≤ N, where represents the size of the set and represents the set with priority P1, represents the set with priority P2, represents the set with priority P M and the data streams within each priority set have the same priority.

[0065] S260. When the data stream is classified as a second-type data stream, determine the data stream set with the minimum period and at least one data stream set with a non-minimum period in the second-type data streams according to the period of the data stream, and aggregate or split the data streams according to the priority of the data streams in the data stream set with the minimum period, the priority of the data streams in the data stream set with a non-minimum period, the size of the data stream, and the scale threshold, so as to obtain a second new data stream set in the case of the second-type data stream.

[0066] Among them, the second new data stream set refers to a new data stream set obtained by aggregating or splitting the minimum period and at least one non-minimum period according to a preset processing rule when the periods of the data streams in the data stream set are different. Among them, the non-minimum period is the period greater than the minimum period.

[0067] In this embodiment, when the data stream is classified as a second-type data stream, that is, when the periods of the data streams in the data stream set are different, the data stream set corresponding to the minimum period and at least one data stream set corresponding to a non-minimum period in the second-type data streams can be determined according to the periods of the data streams, and the data streams are aggregated or split according to the priority of the data streams in the data stream set with the minimum period, the priority of the data streams in the data stream set with a non-minimum period, the size of the data stream, and the scale threshold, so as to obtain a second new data stream set in the case of the second-type data stream.

[0068] Specifically, arrange the periods of each data stream in the second type of data stream to generate a period sorting sequence, determine the minimum period and non-minimum periods in the period sorting sequence, form a data stream set with the minimum period based on the data stream corresponding to the minimum period in the second type of data stream, and aggregate the data streams according to the priorities of the data streams in the data stream set with the minimum period to obtain corresponding aggregated data streams; form at least one data stream set with non-minimum periods based on the data streams corresponding to at least one non-minimum period in the second type of data stream, and then aggregate or split the data streams according to the priorities of the data streams, the sizes of the data streams, and the corresponding scale thresholds in the at least one data stream set with non-minimum periods to obtain corresponding aggregated data streams and split data streams. Thus, a second new data stream set is composed of the aggregated data streams obtained under the minimum period, the aggregated data streams obtained under non-minimum periods, and the split data streams.

[0069] In one embodiment, determining the data stream set with the minimum period and at least one data stream set with non-minimum periods in the second type of data stream according to the periods of the data streams includes:

[0070] Arrange the periods of each data stream in the second type of data stream to generate a period sorting sequence;

[0071] Determine the minimum period and non-minimum periods in the period sorting sequence;

[0072] Form a data stream set with the minimum period based on the data stream corresponding to the minimum period in the second type of data stream;

[0073] Form at least one data stream set with non-minimum periods based on the data streams corresponding to at least one non-minimum period in the second type of data stream.

[0074] Among them, the period sorting sequence refers to the period sequence obtained by sorting the periods of the data streams in the second type of data stream in ascending or descending order. Exemplarily, the period sorting sequence is T0 = T1 ≤ T2 ≤... ≤ T y , y ≤ N, the minimum period is represented as T0, and the non-minimum periods are T i > T0 periods.

[0075] In this embodiment, sort the periods of each data stream in the second type of data stream in ascending or descending order to obtain a corresponding period sorting sequence. According to the result of the period sorting sequence, the minimum period and non-minimum periods in the period sorting sequence can be determined. Thus, form a data stream set with the minimum period based on the data stream corresponding to the minimum period in the second type of data stream, and form at least one data stream set with non-minimum periods based on the data streams corresponding to at least one non-minimum period in the second type of data stream.

[0076] In one embodiment, aggregating or splitting data streams based on the priorities of data streams in the data stream set with the minimum period, the priorities of data streams in at least one data stream set with a non-minimum period, the sizes of the data streams, and the scale threshold to obtain a second new data stream set in the case of the second type of data streams, including:

[0077] Aggregating the data streams based on the priorities of the data streams in the data stream set with the minimum period to obtain a second aggregated data stream;

[0078] Aggregating or splitting the data streams based on the priorities of the data streams in at least one data stream set with a non-minimum period, the sizes of the data streams, and the scale threshold to obtain a third aggregated data stream, a first split data stream, and / or a second split data stream;

[0079] Composing a second new data stream set based on the second aggregated data stream, the third aggregated data stream, the first split data stream, and / or the second split data stream.

[0080] Among them, the second aggregated data stream can be understood as the aggregated data stream obtained by aggregating data streams with the same priority under the minimum period. The third aggregated data stream can be understood as the aggregated data stream obtained by aggregating the data streams in the data stream set under a non-minimum period. The first split data stream can be understood as the split data stream obtained by splitting the data stream when the size of the data stream in the priority grouping is greater than the scale threshold under a non-minimum period. The second split data stream refers to the split data stream obtained when the sum of the sizes of the data streams in the priority grouping is greater than the scale threshold.

[0081] In some embodiments, the data streams can be aggregated based on the priorities of the data streams in the data stream set with the minimum period to obtain a second aggregated data stream. Specifically, the data streams in the data stream set with the minimum period can be divided into at least one data stream subset with the minimum period according to the priorities of the data streams. The data streams in each subset have the same priority, and the data streams in each data stream subset are aggregated to obtain a second aggregated data stream.

[0082] In some embodiments, data streams can be aggregated or split based on the priorities of the data streams, the sizes of the data streams, and a scale threshold in a set of data streams with at least one non-minimum period to obtain a third aggregated data stream, a first split data stream, and / or a second split data stream. Specifically, the data streams in the set of data streams with at least one non-minimum period can be grouped according to the priorities of the data streams to form at least one priority group. If there is a data stream with a size greater than the scale threshold in each priority group, the data stream with a size greater than the scale threshold is split into at least two first new data streams as the first split data stream. If there is a data stream with a size less than or equal to the scale threshold in each priority group, it is necessary to determine whether the sum of the sizes of the data streams in the priority group is less than or equal to the scale threshold, and perform corresponding data stream aggregation and splitting processing based on the determination result.

[0083] It should be noted that from the sum of the sizes of all data streams in the minimum period and the protection bandwidth between the data streams, the reference size of the data stream in the minimum period can be obtained, which can be expressed by the formula where T0 represents the minimum period, represents the size of the data stream with the same priority P in the period T0 j of the data stream, P j represents the priority of the data stream j, γ represents the protection bandwidth between the data streams, J is the set of data streams. Thus, the scale threshold of the data stream size can be obtained, which can be expressed by the formula (1 + α) * (T0 - S0), where the value range of α is 0 ≤ α ≤ 1, representing the adjustment factor, T0 represents the minimum period, and S0 represents the reference value size of the data stream in the minimum period.

[0084] In one embodiment, aggregating the data streams according to the priorities of the data streams in the set of data streams with the minimum period to obtain a second aggregated data stream includes:

[0085] Dividing the data streams in the set of data streams with the minimum period into at least one subset of data streams with the minimum period according to the priorities of the data streams;

[0086] Aggregating the data streams in the subset of data streams with the minimum period to obtain a second aggregated data stream, where the data streams in the subset of data streams with the minimum period have the same priority.

[0087] In this embodiment, for the data streams in the data stream set with the minimum period, the data streams can be sorted in descending or ascending order according to the priorities of the data streams in the data stream set with the minimum period, so as to divide the data streams in the data stream set with the minimum period into at least one data stream subset with the minimum period according to the priorities of the data streams, and aggregate the data streams in the data stream subset with the minimum period to obtain the second aggregated data stream after aggregation. Of course, the data streams in the data stream subset have the same priority.

[0088] In one embodiment, aggregating or splitting data streams according to the priorities of the data streams, the sizes of the data streams, and a scale threshold in at least one data stream set with a non-minimum period to obtain a third aggregated data stream, a first split data stream, and / or a second split data stream, includes:

[0089] Group the data streams in at least one data stream set with a non-minimum period according to the priorities of the data streams to form at least one priority group;

[0090] For each priority group, in the case where there is a data stream in the priority group whose size is greater than the scale threshold, split the data stream whose size is greater than the scale threshold into at least two first new data streams, and use the at least two first new data streams as the first split data stream, where the size of the first new data stream is less than or equal to the scale threshold;

[0091] Judge whether the sum of the sizes of the data streams in the priority group is less than or equal to the scale threshold. If so, aggregate the data streams to obtain the third aggregated data stream; if not, split the data streams into at least one second new data stream subset as the second split data stream, where the sizes of the third aggregated data stream and the second new data stream subset are both less than or equal to the scale threshold.

[0092] In this embodiment, data streams in at least one data stream set with non-minimum periods are grouped according to the priorities of the data streams to form at least one priority group. For each priority group, it is determined whether there is a data stream in the priority group whose size is greater than the scale threshold. If so, the data stream whose size is greater than the scale threshold is split into two or more first new data streams, and the two or more first new data streams are used as the first split data streams. If not, it is continued to determine whether the sum of the sizes of the data streams in the priority group is less than or equal to the scale threshold. If so, the data streams in the priority group are aggregated to obtain a third aggregated data stream. If not, the data streams in the priority group are split into one or more second new data stream subsets as the second split data streams. It should be noted that the sizes of the two or more first new data streams in the first split data streams should be less than or equal to the scale threshold, the size of the third aggregated data stream should be less than or equal to the scale threshold, and the sizes of the one or more second new data stream subsets in the second split data streams are all less than or equal to the scale threshold. It can be understood that under the condition that the size of the data stream is less than or equal to the scale threshold, multiple data stream subsets can be aggregated into an aggregated data stream.

[0093] In this embodiment, the data streams in each priority group are split into one or more second new data stream subsets, and each second new data stream subset may contain one or more data streams.

[0094] Through the above technical solutions of the embodiments of the present invention, by comparing the periods of the data streams in the data stream set one by one, if the periods of the data streams in the data stream set are the same, the data streams with the same period are classified into the first type of data streams. If the periods of the data streams in the data stream set are different, the data streams with different periods are classified into the second type of data streams, which can further accurately classify the data streams with the same period and the data streams with different periods, facilitating the subsequent processing of the data streams. When the data streams are classified into the first type of data streams, the data streams with the same priority in the first type of data streams are aggregated according to the priorities of the data streams to obtain a first new data stream set. When the data streams are classified into the second type of data streams, the data stream set with the minimum period and at least one data stream set with non-minimum periods in the second type of data streams are determined according to the periods of the data streams, and the data streams are aggregated or split according to the priorities of the data streams in the data stream set with the minimum period, the priorities of the data streams in the data stream set with non-minimum periods, the sizes of the data streams, and the scale threshold to obtain a second new data stream set, which can further solve the problem that the model cannot handle in the scenario of large-scale data streams, prevent network jitter, reduce network latency and the complexity of network communication, and at the same time reduce network load, increasing the determinacy and reliability of the network.

[0095] In one embodiment, for better understanding of the processing method of data stream aggregation or sharding, Figure 3 FIG. 281 is a schematic flowchart of another processing method for data stream aggregation or sharding provided by an embodiment of the present invention. In the embodiment of the present invention, before solving the upper control list, the multi-cycle service flow set F is preprocessed to compress the scale of the solution, so as to solve the problem that the model cannot be solved in some large-scale scenarios. By aggregating or splitting data streams during large-scale traffic scheduling planning in a time-sensitive network, it is ensured that the generated gating list in the planning process can carry large-scale traffic. In this embodiment, the service flow is the data stream in the above embodiment. A single data stream in this embodiment represents any data stream.

[0096] As Figure 3 shown, specifically as follows:

[0097] S310. Obtain the original data stream set.

[0098] In this embodiment, let the set F represent the set of N data streams in the time-sensitive network, that is, F = {f1, f2,..., f N}, each data stream f i has corresponding attribute information, and the attribute information includes <T i , S i , P i 〉, where T i represents the period of the data stream f i , S i represents the size of the data stream f i , P i represents the priority of the data stream f i , and the protection bandwidth between data streams is a constant, denoted by γ.

[0099] S320. Determine whether the periods of the data streams in the data stream set are the same. If so, execute S330; if not, execute S340.

[0100] S330. Arrange the data streams in descending order according to the priority of the data streams to obtain at least one priority set.

[0101] In the embodiment, if the periods of all data streams are the same, that is, T1 = T2 =... = T N , a necessary condition for the schedulability of N data streams is: Then, according to the priority P i sort the data streams from high to low priority, and the set with the priority of P i is represented as Then the set F is divided into M subsets where is represented as the set The size is the number of elements.

[0102] S331: Aggregate the data streams in the priority set to obtain a first aggregated data stream under the same period, and denote the set formed by all the first aggregated data streams as the first new data stream set. Among them, the data streams in the priority set have the same priority.

[0103] In this embodiment, The data streams in can be aggregated into the data stream The corresponding attribute information is expressed as At this time, the first new data stream set is expressed as f k denotes the k-th data stream in, S k denotes the data stream f k The size.

[0104] S340: Arrange the periods of the data streams in ascending order to generate a period sorting sequence, and judge the data stream sets corresponding to the minimum period and non-minimum period in the period sorting sequence. If it is the data stream set of the minimum period, execute S350; if it is the data stream set of the non-minimum period, execute S360.

[0105] In this embodiment, if there are different periods of the data streams in the data stream set, first sort them from small to large according to the data stream periods. Let T0 = T1 ≤ T2 ≤... ≤ T y , y ≤ N, where the minimum period is expressed as T0, and the set with the period of T i is expressed as The data streams are divided into the set According to its priority P j Sort the data streams from high to low priority. The set with the priority of P j constitutes the data stream set with the same priority within the same period, which is expressed as That is A subset of.

[0106] S350: Process the data stream set of the minimum period.

[0107] In this embodiment, the minimum period is set as T0, and the data stream set of the minimum period T0 is expressed as

[0108] S351: Arrange the data streams in the data stream set of the minimum period in descending order according to their priorities, and divide them into at least one data stream subset of the minimum period.

[0109] Among them, each data stream in the data stream subset has the same priority.

[0110] S352. Aggregate the data streams in the data stream subset to obtain a first aggregated data stream with a period of the minimum period T0, and then execute S370.

[0111] In this embodiment, if is a subset of the set of minimum periods T0 that is, all the data streams within can be aggregated into whose corresponding attribute information is expressed as The new data stream set under the minimum period T0 is expressed as

[0112] It should be noted that from the sum of the sizes of all data streams under the minimum period T0 and the protection bandwidth between data streams, the reference size of the data stream under the minimum period can be obtained, which can be expressed by the formula as where T0 represents the minimum period, represents the size of the data stream with the same priority P j at the period of T0, P j represents the priority of data stream j, γ represents the protection bandwidth between data streams, and J is the data stream set. Thus, the scale threshold of the data stream size can be obtained, which can be expressed by the formula as (1 + α) * (T0 - S0), where the value range of α is 0 ≤ α ≤ 1, representing the adjustment factor, T0 represents the minimum period, and S0 represents the reference value size of the data stream under the minimum period.

[0113] S360. Process each data stream set with a non - minimum period.

[0114] In this embodiment, the period greater than the minimum period T0 is represented as T i , where i = 2, 3,..., y.

[0115] S361. Sort the data streams in each data stream set with a non - minimum period in descending order according to the priority of the data streams to form at least one priority group for data stream grouping.

[0116] S362. Check whether the size of the single - stream data stream in each priority group is greater than the scale threshold. If so, execute S363; if not, execute S364.

[0117] In this embodiment, the scale threshold is obtained based on the period of the minimum period and the reference value size of the data stream under the minimum period, which can be expressed by the formula as (1 + α) * T0 - S0), where the value range of α is 0 ≤ α ≤ 1, representing the adjustment factor, T0 represents the minimum period, and S0 represents the reference value size of the data stream under the minimum period.

[0118] In this embodiment, it is determined whether there is a single stream currently with a size S k satisfies S k ≥(1 + α)*(T0 - S0). If so, the size of the data stream with a single stream is greater than the scale threshold. If not, the size of the data stream with a single stream is not greater than the scale threshold.

[0119] S363. The data stream with a single stream is split into multiple data streams, and the multiple data streams are used as the first split data streams, and then S370 is executed.

[0120] Among them, the size of each split data stream needs to satisfy that the size of the data stream is less than or equal to the scale threshold.

[0121] In this embodiment, if there is a single stream with a size S k satisfies S k ≥(1 + α)*(T0 - S0), then f k is split into multiple data streams to obtain multiple new data streams. At this time, Among them, S k0 , S k1 ,..., S kj are respectively expressed as the sizes of the data streams corresponding to f k0 , f k1 ,..., f kj correspondingly.

[0122] S364. Determine whether the sum of the sizes of the data streams in the priority grouping is less than or equal to the scale threshold. If so, execute S365; if not, execute S366.

[0123] In this embodiment, if there is no single stream with a size S k satisfies S k ≥(1 + α)*(T0 - S0), then it is necessary to determine whether the sum of the sizes of all the data streams within Among them, the value range of α is 0 ≤ α ≤ 1, which is expressed as an adjustment factor. Here, α can be taken as 1, T0 represents the minimum period, and S0 represents the reference value size of the data stream under the minimum period.

[0124] S365. Aggregate the data streams to obtain a third aggregated data stream, where the size of the third aggregated data stream is less than or equal to the scale threshold, and then execute S370.

[0125] In this embodiment, the non - minimum period can also be referred to as a period greater than the minimum period. All the data streams within the non - minimum period, that is All data streams within are aggregated into and used as a new data stream, and its corresponding attribute information is

[0126] S366. Split each data stream into at least one new data stream subset as the second split data stream, where the size of each of the at least one new data stream subset is less than or equal to the scale threshold, and then execute S370.

[0127] In this embodiment, if all data streams within the non-minimum period, that is, the sum of all data streams within satisfies then it will be further split into subsets Each subset element can be an aggregated data stream or a set composed of at least two data streams. Among them, for the aggregated data stream the size attribute needs to satisfy: Among them, are respectively expressed as the sizes of the data streams of.

[0128] S370. Compose a second new data stream set based on the second aggregated data stream, the third aggregated data stream, the first split data stream, and the second split data stream.

[0129] Among them, the new data stream set obtained when the period is greater than the minimum period includes the new data stream set obtained by splitting when the size of a single stream is greater than (1 + α) * (T0 - S0), the new data stream set obtained by aggregation when the sum of all data streams in the data stream set of the non-minimum period is greater than (1 + α) * (T0 - S0), and the new data stream set obtained by splitting when the sum of all data streams in the data stream set of the non-minimum period is less than or equal to (1 + α) * (T0 - S0).

[0130] In this embodiment, all new data stream sets constitute the second new data stream set, which is expressed by the formula as

[0131] In one embodiment, for better understanding of the processing method of data stream aggregation or sharding, this embodiment uses a formula to elaborate in detail on the processing method of data stream aggregation or sharding based on the process in the above embodiment. The specific steps are as follows:

[0132] a1. Obtain the original data stream set F.

[0133] a2. Determine whether each data stream in the data stream set F is of the same period. If so, execute a3; if not, execute a6.

[0134] a3. Arrange each data stream in descending order according to the priority of the data stream to obtain at least one priority set, and the priority is P i The set is represented as

[0135] a4. Aggregate the data streams in the priority set The data streams in can be aggregated into a data stream

[0136] a5. Denote the set formed by the first aggregated data stream as the first new data stream set, and it is represented as The corresponding attribute information is represented as

[0137] a6. Arrange the periods of the data streams in ascending order: T0 = T1 ≤ T2 ≤... ≤ T y , y ≤ N. Judge the data stream sets corresponding to the minimum period and non - minimum periods respectively. If it is the data stream set with the minimum period, execute a7; if it is the data stream set with non - minimum periods, execute a8.

[0138] a7. Process the data stream set with the minimum period T0

[0139] a71. Arrange the data streams in the data stream set with the minimum period in descending order according to the priority of each data stream, and divide them into at least one data stream subset with the minimum period, and the priority is P j The data stream set

[0140] a72. Aggregate the data streams in the data stream subset to obtain the first aggregated data stream at the minimum period T0 Among them, the corresponding attribute information is represented as Then execute a9.

[0141] a8. Process the data stream set with the period T i > T0, where i = 2, 3,..., y

[0142] a81. Sort the data streams in each data stream set with non - minimum periods in descending order according to the priority of the data stream. For the period T i The priority is P j The data stream set is represented as

[0143] a82. For each priority group, judge the size S of the single stream k Whether it satisfies S k ≥ (1 + α) * (T0 - S0). If so, execute a83; if not, execute a84.

[0144] a83, f k Split into multiple data streams and use the multiple data streams as the first split data stream. Among them, Then execute a9.

[0145] In this embodiment, S k0 , S k1 ,..., S kj Are respectively represented as f k0 , f k1 ,..., f kj The sizes of the corresponding data streams.

[0146] a84. Determine Whether the sum of the sizes of all data streams within satisfies If so, execute S465; if not, execute S466. a85. All data streams within Are aggregated into Whose corresponding attribute information is Then execute a9.

[0147] a86. Will be further split into subsets Each subset aggregates data streams Among them, the aggregated data stream The size attribute of needs to satisfy: Then execute a9.

[0148] a9.

[0149] In one embodiment, Figure 4 Is the structural block diagram of a data stream aggregation or sharding processing device provided by an embodiment of the present invention. This device is applicable to the situation of aggregating or splitting periodic data streams. This device can be implemented by hardware / software. It can be configured in an electronic device to implement a data stream aggregation or sharding processing method in an embodiment of the present invention. As Figure 4 Shown, this device includes: a set acquisition module 410, a classification determination module 420, and a new data stream set determination module 430.

[0150] Among them, the set acquisition module 410 is used to acquire a data stream set containing data streams;

[0151] The classification determination module 420 is used to determine data stream classification according to the situation that the periods of the data streams in the data stream set are the same;

[0152] A new data stream set determination module 430, configured to aggregate or split according to the data stream classification and corresponding preset processing rules to process into a new data stream set, where the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set satisfy a scale threshold.

[0153] In an embodiment of the present invention, the classification determination module can accurately classify data streams with the same period and data streams with different periods by determining the data stream classification according to the situation that the periods of each data stream in the data stream set are the same, which is convenient for subsequent processing of the data stream; the new data stream set determination module aggregates or slices each data stream in the corresponding data stream classification according to different processing rules according to the classification situation of the data stream to process into a new data stream set, where the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set satisfy a scale threshold, which can solve the problem that the model cannot handle in the scenario of large-scale data streams, prevent network jitter from occurring, reduce network latency and the complexity of network communication, and at the same time reduce network load and increase the determinacy and reliability of the network.

[0154] In an embodiment, the data stream has at least one of the following attribute information: period, size, priority, and protection bandwidth.

[0155] In an embodiment, the classification determination module 420 includes:

[0156] Compare the periods of each data stream in the data stream set one by one;

[0157] If the periods of each data stream in the data stream set are the same, classify the data streams with the same period as the first type of data stream;

[0158] If the periods of each data stream in the data stream set are different, classify the data streams with different periods as the second type of data stream.

[0159] In an embodiment, the new data stream set determination module 430 includes:

[0160] A first new data stream set unit, configured to, when the data stream is classified as the first type of data stream, aggregate the data streams with the same priority in the first type of data stream according to the priority of the data stream to obtain a first new data stream set in the case of the first type of data stream;

[0161] A second new data stream set unit, configured to, when the data stream is classified as the second type of data stream, determine a data stream set with the minimum period and at least one data stream set with a non-minimum period in the second type of data stream according to the period of the data stream, and aggregate or split the data stream according to the priority of the data stream in the data stream set with the minimum period, the priority of the data stream in the data stream set with the non-minimum period, the size of the data stream, and a scale threshold, so as to obtain a second new data stream set in the case of the second type of data stream.

[0162] In one embodiment, the first new data stream set unit includes:

[0163] A set determination subunit, configured to divide each of the data streams according to the priority of the data stream to obtain at least one priority set; wherein, the data streams within the priority set have the same priority;

[0164] A first aggregation subunit, configured to aggregate the data streams in the priority set to obtain a first aggregated data stream;

[0165] A first new set subunit, configured to denote the set formed by all the first aggregated data streams as the first new data stream set

[0166] In one embodiment, the second new data stream set unit includes:

[0167] An arrangement subunit, configured to arrange the periods of the data streams in the second type of data stream to generate a period sorting sequence;

[0168] A determination subunit, configured to determine the minimum period and the non-minimum period in the period sorting sequence;

[0169] A minimum period subunit, configured to form a data stream set with the minimum period according to the data streams corresponding to the minimum period in the second type of data stream;

[0170] A non-minimum period subunit, configured to form at least one data stream set with a non-minimum period according to the data streams corresponding to the at least one non-minimum period in the second type of data stream.

[0171] In one embodiment, the second new data stream set unit further includes:

[0172] A second aggregation subunit, configured to aggregate the data streams according to the priority of the data streams in the data stream set with the minimum period to obtain a second aggregated data stream;

[0173] An aggregation and splitting subunit, configured to aggregate or split data streams according to the priorities of the data streams, the sizes of the data streams, and a scale threshold in the set of data streams with at least one non-minimum period, so as to obtain a third aggregated data stream, a first split data stream, and / or a second split data stream;

[0174] A second new data stream set subunit, configured to form a second new data stream set according to the second aggregated data stream, the third aggregated data stream, the first split data stream, and / or the second split data stream.

[0175] In one embodiment, the second aggregation subunit is further configured to:

[0176] Divide the data streams in the set of data streams with the minimum period into at least one subset of data streams with the minimum period according to the priorities of the data streams;

[0177] Aggregate the data streams in the subset of data streams with the minimum period to obtain a second aggregated data stream, where the data streams in the subset of data streams with the minimum period have the same priority.

[0178] In one embodiment, the aggregation and splitting subunit is further configured to:

[0179] Group the data streams in the set of data streams with at least one non-minimum period according to the priorities of the data streams to form at least one priority group;

[0180] For each priority group, in the case where there is a data stream in the priority group whose size is greater than the scale threshold, split the data stream whose size is greater than the scale threshold into at least two first new data streams, and use the at least two first new data streams as the first split data stream, where the size of the first new data stream is less than or equal to the scale threshold;

[0181] Judge whether the sum of the sizes of the data streams in the priority group is less than or equal to the scale threshold. If so, aggregate the data streams to obtain the third aggregated data stream; if not, split the data streams into at least one second new data stream subset as the second split data stream, where the sizes of the third aggregated data stream and the second new data stream subset are both less than or equal to the scale threshold.

[0182] In one embodiment, the scale threshold is (1 + α) * (T0 - S0), where the value range of α is 0 ≤ α ≤ 1, which is an adjustment factor, T0 represents the minimum period, and S0 represents the reference value size of the data stream under the minimum period.

[0183] The processing device for data stream aggregation or sharding provided by the embodiments of the present invention can execute the processing method for data stream aggregation or sharding provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0184] In one embodiment, Figure 5 FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0185] As Figure 5 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0186] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0187] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the processing methods for data stream aggregation or sharding.

[0188] In some embodiments, the processing method for data stream aggregation or sharding may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the processing method for data stream aggregation or sharding described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the processing method for data stream aggregation or sharding by any other suitable means (e.g., by means of firmware).

[0189] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0190] The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data stream aggregation or sharding processing devices, such that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0191] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0192] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0193] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0194] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0195] It should be understood that various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0196] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for processing data stream aggregation or fragmentation, characterized in that Including: Obtain a data stream set containing data streams; Determine data stream classification according to the situation that the periods of the data streams in the data stream set are the same; Aggregate or fragment each of the data streams according to the data stream classification and corresponding preset processing rules to process them into a new data stream set, where the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set meet the scale threshold; Among them, the determining the data stream classification according to the situation that the periods of the data streams in the data stream set are the same includes: Compare the periods of each of the data streams in the data stream set one by one; If the periods of the data streams in the data stream set are the same, classify the data streams with the same period into the first type of data streams; If the periods of the data streams in the data stream set are different, classify the data streams with different periods into the second type of data streams; Among them, the aggregating or fragmenting each of the data streams according to the data stream classification and corresponding preset processing rules to process them into a new data stream set includes: In the case where the data stream classification is the first type of data streams, aggregate the data streams with the same priority in the first type of data streams according to the priority of the data streams to obtain a first new data stream set in the case of the first type of data streams; In the case where the data stream classification is the second type of data streams, determine the data stream set with the minimum period and at least one data stream set with a non-minimum period in the second type of data streams according to the period of the data streams, and aggregate or split the data streams according to the priority of the data streams in the data stream set with the minimum period, the priority of the data streams in the data stream set with a non-minimum period, the size of the data streams, and the scale threshold to obtain a second new data stream set in the case of the second type of data streams.

2. The method according to claim 1, wherein The data stream has at least one of the following attribute information: period, size, priority, protection bandwidth.

3. The method according to claim 1, wherein The aggregating the data streams with the same priority in the first type of data streams according to the priority of the data streams to obtain a first new data stream set in the case of the first type of data streams includes: Divide each of the data streams according to the priority of the data streams to obtain at least one priority set; where the data streams in the priority set have the same priority; Aggregate the data streams in the priority set to obtain a first aggregated data stream; Denote the set formed by all the first aggregated data streams as the first new data stream set.

4. The method according to claim 1, wherein The determining the data stream set with the minimum period and at least one data stream set with a non-minimum period in the second type of data streams according to the period of the data streams includes: Arrange the periods of each of the data streams in the second type of data streams to generate a period sorting sequence; Determine the minimum period and the non-minimum period in the period sorting sequence; Constitute the data stream set with the minimum period according to the data streams corresponding to the minimum period in the second type of data streams; A data stream set of at least one non-minimum period is formed according to the data stream corresponding to the at least one non-minimum period in the second type of data stream.

5. The method according to claim 1, wherein Aggregating or splitting the data stream according to the priority of the data stream in the data stream set of the minimum period, the priority of the data stream in the data stream set of at least one non-minimum period, the size of the data stream, and the scale threshold to obtain a second new data stream set in the case of the second type of data stream, including: Aggregating the data stream according to the priority of the data stream in the data stream set of the minimum period to obtain a second aggregated data stream; Aggregating or splitting the data stream according to the priority of the data stream, the size of the data stream, and the scale threshold in the data stream set of at least one non-minimum period to obtain a third aggregated data stream, a first split data stream, and / or a second split data stream; Composing a second new data stream set according to the second aggregated data stream, the third aggregated data stream, the first split data stream, and / or the second split data stream.

6. The method according to claim 5, characterized in that, The aggregating the data stream according to the priority of the data stream in the data stream set of the minimum period to obtain a second aggregated data stream includes: Dividing the data stream in the data stream set of the minimum period into at least one data stream subset of the minimum period according to the priority of the data stream; Aggregating the data streams in the data stream subset of the minimum period to obtain a second aggregated data stream, where the data streams in the data stream subset of the minimum period have the same priority.

7. The method according to claim 5, characterized in that The aggregating or splitting the data stream according to the priority of the data stream, the size of the data stream, and the scale threshold in the data stream set of at least one non-minimum period to obtain a third aggregated data stream, a first split data stream, and / or a second split data stream includes: Grouping the data streams in the data stream set of at least one non-minimum period according to the priority of the data stream to form at least one priority group; For each priority group, in the case where there is a data stream with a size greater than the scale threshold in the priority group, splitting the data stream with a size greater than the scale threshold into at least two first new data streams, and using the at least two first new data streams as the first split data stream, where the size of the first new data stream is less than or equal to the scale threshold; Judging whether the sum of the sizes of the data streams in the priority group is less than or equal to the scale threshold. If so, aggregating the data streams to obtain the third aggregated data stream; if not, splitting the data streams into at least one second new data stream subset as the second split data stream, where the sizes of the third aggregated data stream and the second new data stream subset are both less than or equal to the scale threshold.

8. The method according to claim 1, characterized in that The scale threshold is (1 + α)*(T0 - S0), where the value range of α is 0 ≤ α ≤ 1, which is an adjustment factor, T0 represents the minimum period, and S0 represents the reference value size of the data stream under the minimum period.

9. A processing device for data stream aggregation or fragmentation, characterized in that, Including: A set acquisition module for acquiring a data stream set containing data streams; A classification determination module for determining a data stream classification according to the situation that the periods of the data streams in the data stream set are the same; A new data stream set determination module for aggregating or splitting according to the data stream classification and corresponding preset processing rules to process into a new data stream set, where the priorities of the data streams in the new data stream set are unified, or the priorities of the data streams in the new data stream set are unified and the sizes of the data streams in the new data stream set meet a scale threshold; Among them, the classification determination module includes: Compare the periods of each data stream in the data stream set one by one; If the periods of the data streams in the data stream set are the same, the data streams with the same period are classified as the first type of data streams; If the periods of the data streams in the data stream set are different, the data streams with different periods are classified as the second type of data streams; Among them, the new data stream set determination module includes: A first new data stream set unit for aggregating the data streams with the same priority in the first type of data streams according to the priority of the data streams to obtain a first new data stream set in the case of the first type of data streams when the data stream classification is the first type of data streams; A second new data stream set unit for determining a data stream set with the smallest period and at least one data stream set with a non-smallest period in the second type of data streams according to the period of the data streams in the case that the data stream classification is the second type of data streams, and aggregating or splitting the data streams according to the priority of the data streams in the data stream set with the smallest period, the priority of the data streams in the data stream set with a non-smallest period, the size of the data streams and the scale threshold to obtain a second new data stream set in the case of the second type of data streams.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; A memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the data stream aggregation or sharding processing method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program implements the data stream aggregation or sharding processing method as described in any one of claims 1-8 when executed by a processor.

Images