Data transmission method and electronic device

Abstract

The application discloses a data transmission method and an electronic device, relates to the technical field of data communication, and predicts a bandwidth of a next period and a corresponding second data transmission rate based on a current running state, determines a weight of the second data transmission rate in final target transmission rate decision in combination with a probability of abnormal traffic in a data transmission system, so that the finally determined target transmission rate is more in line with an actual state of the data transmission system, stability of data transmission is ensured, and flexibility of transmission rate control in a data transmission process is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data communication, and particularly relates to a data transmission method and an electronic device. BACKGROUND

[0002] In a modern data center, in order to ensure the stable operation of a server, a baseboard management controller (BMC) monitors the hardware state of the server, generates corresponding log data and immediately forwards the log data to an operation and maintenance personnel, so that the operation and maintenance personnel can know the running state of the server in a timely manner.

[0003] At present, the data transmission rate is usually controlled below a set value to avoid network fluctuations or data loss caused by too fast data transmission.

[0004] However, due to the limited network bandwidth of the data center, especially under high load, network congestion and other conditions are prone to occur, which cannot meet the demand of the set transmission rate setting value, resulting in data transmission failure or data loss. Therefore, a flexible data transmission method is urgently needed to ensure the stable transmission of log data. SUMMARY

[0005] The present application provides a data transmission method and an electronic device to at least solve the problem of poor data transmission stability in the related art.

[0006] The present application provides a data transmission method, comprising:

[0007] obtaining running state data of a data transmission system in a current preset time period, the data transmission system comprising a data sending end, a data receiving end, and a communication link between the data sending end and the data receiving end, and the running state data comprising a first data transmission rate of the communication link in the current preset time period;

[0008] determining, based on the running state data, an available bandwidth prediction value of the communication link in a next preset time period and a probability of abnormal traffic occurring in the data transmission system;

[0009] determining a data transmission rate in the next preset time period according to the available bandwidth prediction value, obtaining a second data transmission rate, the second data transmission rate being positively correlated with the available bandwidth prediction value;

[0010] determining a weight of the second data transmission rate according to the probability of abnormal traffic occurring, the weight of the second data transmission rate being negatively correlated with the probability of abnormal traffic occurring in the data transmission system;

[0011] performing weighted summation on the first data transmission rate and the second data transmission rate to obtain a target transmission rate in the next preset time period;

[0012] controlling the data transmission system based on the target transmission rate in the next preset time period.

[0013] The application further provides an electronic device, comprising a memory for storing a computer program and a processor for executing the computer program to implement the steps of any of the data transmission methods.

[0014] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of any of the data transmission methods.

[0015] The application further provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the steps of any of the data transmission methods.

[0016] According to the application, the bandwidth in the next time period and the corresponding second data transmission rate are predicted based on the current running state, and the weight of the second data transmission rate in the final target transmission rate decision is determined in combination with the probability of abnormal traffic in the data transmission system, so that the finally determined target transmission rate is more in line with the actual state of the data transmission system, the stability of data transmission is ensured, and the flexibility of transmission rate control in the data transmission process is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The data transmission method flowchart provided for the embodiments of the application;

[0019] Figure 2 The training method flowchart of the bandwidth prediction model provided for the embodiments of the application;

[0020] Figure 3 The structural schematic diagram of the data transmission device provided for the embodiments of the application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the application will be described clearly and completely in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0022] It should be noted that in the description of the present application, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or inherent to such process, method, article or apparatus. The terms "first", "second" and the like in the present application are used to distinguish similar objects, not to describe a specific order or sequence.

[0023] In order to enable those skilled in the art to better understand the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0024] Figure 1 The data transmission method flowchart provided by the embodiment of the present application is shown in FIG. 1. As shown in the figure, the method comprises the following steps: Figure 1

[0025] S101, obtaining running state data of a data transmission system in a current preset time period.

[0026] The data transmission system comprises a data sending end, a data receiving end, and a communication link between the data sending end and the data receiving end. The running state data comprises a first data transmission rate of the communication link in the current preset time period.

[0027] Optionally, the data sending end is a BMC, the data receiving end is a log storage server, and the communication link is a data center network. Correspondingly, the data transmission refers to transmission of log data other than normal business traffic.

[0028] The running state data is a multi-dimensional data set reflecting the real-time running condition of the system. In addition to the first data transmission rate, it also includes the central processing unit utilization rate, memory bandwidth occupation, storage IO rate of the data sending end, and bandwidth utilization rate, round-trip delay, packet loss rate of the communication link.

[0029] The first data transmission rate is the actual data transmission rate of the communication link in the previous preset time period, which can be obtained by sFlow sampling technology.

[0030] S102, determining an available bandwidth prediction value of the communication link in a next preset time period and a probability of occurrence of abnormal traffic in the data transmission system based on the running state data.

[0031] The next preset time period refers to the next time window for bandwidth prediction. Optionally, the next preset time period is 30 seconds.

[0032] ​The available bandwidth prediction value refers to the residual bandwidth of the communication link available for transmitting data (for example, log data) in the next preset time period.

[0033] The probability of abnormal traffic occurrence refers to the possibility of occurrence of malicious traffic or sudden abnormal traffic in the data transmission system in the next preset time period, representing the stability of data transmission in the next preset time period.

[0034] S103, determining a data transmission rate in the next preset time period according to the available bandwidth prediction value, to obtain a second data transmission rate.

[0035] The second data transmission rate is positively correlated with the available bandwidth prediction value.

[0036] The second data transmission rate is a theoretical optimal transmission rate calculated based on the available bandwidth prediction value, that is, the maximum data transmission rate that the communication link can support in the next preset time period without abnormal conditions.

[0037] The greater the available bandwidth prediction value, the smaller the data transmission pressure of the communication link in the next preset time period, and the better the communication quality, which can support a larger second data transmission rate; the smaller the available bandwidth prediction value, the greater the data transmission pressure of the communication link in the next preset time period, and the communication quality may be affected, and the second data transmission rate needs to be reduced accordingly.

[0038] S104, determining the weight of the second data transmission rate according to the probability of abnormal traffic occurrence.

[0039] The weight of the second data transmission rate is negatively correlated with the probability of abnormal traffic occurrence in the data transmission system.

[0040] The greater the probability of abnormal traffic occurrence, the higher the probability of abnormal conditions such as network attacks in the data transmission system, and the lower the credibility of the second data transmission rate obtained according to the available bandwidth prediction value, so the weight of the second data transmission rate needs to be reduced to reduce the influence of the second data transmission rate on the final decision of the data transmission rate.

[0041] The smaller the probability of abnormal traffic occurrence, the lower the probability of abnormal conditions in the data transmission system, and the higher the credibility of the second data transmission rate obtained according to the available bandwidth prediction value, so the weight of the second data transmission rate can be increased to expand the influence of the second data transmission rate on the final decision of the data transmission rate.

[0042] S105, weighted sum of the first data transmission rate and the second data transmission rate, to obtain a target transmission rate in the next preset time period.

[0043] Specifically, the sum of the weight of the first data transmission rate and the weight of the second data transmission rate is a fixed value. If the weight of the second data transmission rate is determined to be small according to the probability of abnormal traffic, the weight of the first data transmission rate is relatively large, and the first data transmission rate corresponding to the current preset time period is mainly used for decision-making when abnormal situations may occur. If the weight of the second data transmission rate is determined to be large according to the probability of abnormal traffic, the weight of the first data transmission rate is relatively small, and the second data transmission rate corresponding to the available bandwidth prediction value of the next preset time period is mainly used for decision-making under normal circumstances.

[0044] S106, controlling the data transmission system based on the target transmission rate in the next preset time period.

[0045] Optionally, the target transmission rate of the data transmission system is controlled by using a token bucket. For example, the token generation rate of the token bucket is adjusted to make the data transmission rate of the data transmission system reach the target transmission rate.

[0046] The embodiments of the present application predict the bandwidth and the corresponding second data transmission rate in the next time period through the current running state, and determine the weight of the second data transmission rate in the final decision in combination with the abnormal traffic in the data transmission system. The weight of the second data transmission rate is reduced when abnormal situations may occur, so that the target transmission rate determined by the final weighted calculation is more consistent with the actual state of the data transmission system, ensuring the stability of data transmission and improving the flexibility of transmission rate control in the data transmission process.

[0047] In some embodiments, based on the running state data, the available bandwidth prediction value of the communication link in the next preset time period and the probability of abnormal traffic in the data transmission system are determined, including: inputting the running state data into a pre-trained bandwidth prediction model to obtain a bandwidth occupation prediction value output by the bandwidth prediction model; when the bandwidth occupation prediction value is less than or equal to a preset bandwidth threshold, calculating the difference between the preset bandwidth threshold and the bandwidth occupation prediction value to obtain the available bandwidth prediction value; when the bandwidth occupation prediction value is greater than the preset bandwidth threshold, determining the available bandwidth prediction value as a preset minimum bandwidth value, and the probability of abnormal traffic in the data transmission system as a preset maximum probability value.

[0048] The structure and training method of the bandwidth prediction model are introduced below.

[0049] Figure 2 The training method flow chart of the bandwidth prediction model provided by the embodiments of the present application is shown in FIG. 2. Figure 2 As shown in the figure, the method includes the following steps:

[0050] S201, data acquisition and processing.

[0051] The feature data in the normal state of the data collection and transmission system, such as load data, network data and storage data, includes but is not limited to: central processor utilization, interrupt frequency, memory occupancy, memory bandwidth, communication link bandwidth, bandwidth utilization, round-trip delay, packet loss rate, disk input / output per second (IOPS), disk usage, etc.

[0052] Further data cleaning is performed to process the noise, missing values and outliers in the collected data.

[0053] For missing values, the valid value at the previous time is used to fill in the current missing value. If the data at the time points of consecutive time points is indeed, the data in the time period is directly discarded.

[0054] For instantaneous jitter data, smoothing processing is performed according to the data of adjacent time periods. For example, the average value of the data is calculated in a 5-second time window to replace the jitter data in the time window. Alternatively, the median in the time window is taken to replace the jitter data in the time window.

[0055] Unreasonable data is identified by statistical methods. If the difference between a data and the mean value of the data is greater than the preset threshold value, or greater than the actual threshold value of the hardware, the data is marked as invalid data.

[0056] S202, constructing feature engineering.

[0057] Based on the sliding time window, the mean, maximum, minimum, variance and trend slope of the feature data of each feature are calculated to construct the time series features of the data transmission system, and normalization processing is performed.

[0058] S203, data set division.

[0059] The cleaned data is divided into a training set, a validation set and a test set to evaluate the model generalization ability.

[0060] S204, model construction and training.

[0061] The input layer is used to receive input data, and each input data includes historical time series feature data of a first time length and misjudgment time series feature data of a second time length.

[0062] The long short-term memory network (Long Short-Term Memory) layer includes a first layer LSTM, a Dropout layer and a second layer LSTM.

[0063] The first layer LSTM includes 64 neurons and returns an output sequence of full time steps to capture short-term dependencies. A dropout layer randomly masks neuron outputs with a certain probability to prevent model overfitting. The second layer LSTM includes 32 neurons and returns only the output of the last time step.

[0064] The output layer maps the 32-dimensional vector of 32 neuron outputs to a final result, which represents the bandwidth occupancy prediction value.

[0065] The absolute deviation of the predicted value of the optimized bandwidth occupancy from the true value in the feature data is calculated, and the derivative of the absolute deviation with respect to the model parameters is calculated. When the predicted value is less than the true value, the model parameters are adjusted to increase the predicted value by backpropagation; when the predicted value is greater than the true value, the model parameters are adjusted to decrease the predicted value by backpropagation.

[0066] S205, model evaluation and optimization.

[0067] The above parameter adjustment process is repeated, and when the loss function value corresponding to the validation set does not decrease in the continuous multiple rounds of training process, the training is ended.

[0068] The trained bandwidth prediction model is tested using the test set, and if the accuracy exceeds the accuracy threshold, the final trained loan training model is obtained.

[0069] S206, model deployment.

[0070] After obtaining the bandwidth occupancy prediction value output by the bandwidth prediction model, the bandwidth occupancy prediction value is compared with the preset bandwidth threshold.

[0071] The preset bandwidth threshold refers to the upper limit of the safe bandwidth occupancy of the communication link in the data transmission system. Optionally, the preset bandwidth threshold is a preset proportion of the maximum bandwidth of the communication link.

[0072] When the bandwidth occupancy prediction value is less than or equal to the preset bandwidth threshold, it represents that there is remaining bandwidth that can be allocated in the next preset time period, and the difference between the preset bandwidth threshold and the bandwidth occupancy prediction value is the available bandwidth prediction value in the next preset time period.

[0073] When the bandwidth occupancy prediction value is greater than the preset bandwidth threshold, it represents that there is no remaining bandwidth available for allocation in the next preset time period, and the second transmission rate is determined according to the preset minimum bandwidth value.

[0074] The preset minimum bandwidth value is the minimum transmission bandwidth that needs to be guaranteed even if the link bandwidth exceeds the threshold, meeting the minimum requirement of data transmission.

[0075] The preset maximum probability value is an upper limit of the probability of abnormal traffic in the data transmission system. When the probability of abnormal traffic in the data transmission system reaches the preset maximum probability value, it is determined that there is an abnormal situation in the data transmission system, and the weight of the second transmission rate is reduced to the lowest weight (for example, 0), and the target transmission rate is determined based on only the first transmission rate.

[0076] Optionally, if it is determined that the probability of abnormal traffic in the data transmission system is greater than the preset probability threshold, the target transmission rate is determined to be a preset minimum rate. For example, when the probability of abnormal traffic in the data transmission system reaches the preset maximum probability value.

[0077] Further, in the next preset time period, the data transmission system is controlled to transmit at least one data with the highest priority among the plurality of to-be-transmitted data based on the preset minimum rate.

[0078] The embodiments of the present application predict the bandwidth occupation value in the next time period based on the current running state data, and then compare the available bandwidth prediction value in the next time period with the preset bandwidth threshold, determine the corresponding control strategy according to different situations, and set a special strategy when there is no residual allocatable bandwidth in the next time period, further improving the flexibility of transmission rate control during data transmission.

[0079] In other embodiments, the way to determine the probability of abnormal traffic in the data transmission system is as follows: for any data traffic in the data transmission system, the characteristic data of the data traffic is obtained from the running state data, the characteristic data including a plurality of characteristic fields and a characteristic value corresponding to each characteristic field; the characteristic data is input into a pre-trained isolation forest model to obtain the path length of each characteristic value in the isolation forest model; the average path length corresponding to the data traffic is calculated according to the path length of each characteristic value in the isolation forest model; if the average path length is less than a preset length threshold, the data traffic is determined to be abnormal traffic; the proportion of abnormal traffic in the total number of data traffic in the current preset time period is calculated to obtain the probability of abnormal traffic in the data transmission system.

[0080] Optionally, if it is determined that the probability of abnormal traffic in the data transmission system is greater than the preset probability threshold, the target transmission rate is determined to be a preset minimum rate. Correspondingly, in the next preset time period, the data transmission system is controlled to transmit at least one data with the highest priority among the plurality of to-be-transmitted data based on the preset minimum rate.

[0081] The characteristic data of the data traffic includes multi-dimensional characteristic values, such as traffic timing characteristics, traffic structure characteristics, network attribute characteristics, etc., and the characteristic fields include but are not limited to the number of requests per second, the interval between abnormal code occurrences, the integrity of the log field, the proportion of small byte data packets, the entropy value of the source IP address, etc. The characteristic value is the specific numerical value corresponding to each characteristic field.

[0082] The pre-trained isolation forest model is composed of multiple isolated trees, and the training data of the pre-trained isolation forest model is the traffic feature data during the normal operation of the data transmission system, multiple features are randomly selected, a random threshold is set for each feature each time, and the traffic feature data is segmented multiple times to construct multiple isolated trees.

[0083] The path length refers to the number of segmentations required for a single feature value to be isolated from the root node to the leaf node of an isolated tree. Since the feature values of abnormal traffic are significantly different from the feature values of normal traffic, only a few segmentations are often required to isolate them, corresponding to a smaller path length.

[0084] The average path length corresponding to multiple feature values in the feature data reflects the difference between the feature data and the known feature value data of the normal traffic. If the average path length is less than the preset length threshold, it means that the features of the data traffic are significantly different from the features of the normal traffic carried in the isolation forest model, and the data traffic is determined to be abnormal traffic. If the average path length is greater than or equal to the preset length threshold, it means that the features of the data traffic are similar to the features of the normal traffic carried in the isolation forest model, and the data traffic is determined to be normal traffic.

[0085] Optionally, a field weight is set for each feature field according to the historical abnormal operation data of the data transmission system. The greater the difference between the feature values in the historical abnormal operation data and the historical normal operation data, the greater the corresponding field weight. After the feature data is input into the pre-trained isolation forest model and the path length of each feature value in the isolation forest model is obtained, the path lengths corresponding to the feature values are weighted and summed according to the field weight to calculate the average path length, so that the average path length is more in line with the operation rule of the current data transmission system, thereby improving the accuracy of abnormal traffic determination.

[0086] The embodiment of the present application detects abnormal traffic quickly through the isolation forest model, determines the probability of abnormal traffic in the data transmission system according to the proportion of abnormal traffic in the total data traffic in the current preset time period, thereby reflecting the possibility of abnormal situation of the data transmission system. The higher the probability of abnormal traffic, the lower the weight of the second transmission rate, ensuring that the decision of the target transmission rate focuses more on the first transmission rate, avoiding the influence of the prediction error of the available bandwidth prediction value caused by abnormal traffic on the decision of the target transmission rate, and improving the accuracy of the target transmission rate.

[0087] In some embodiments, the data transmission rate in the next preset time period is determined according to the available bandwidth prediction value, and a second data transmission rate is obtained, including: obtaining the average size of the plurality of data transmitted in the current preset time period in the running state data based on the attribute information of each piece of data; calculating the ratio of the available bandwidth prediction value to the average size to obtain the second data transmission rate.

[0088] The attribute information refers to the size and other related information of each piece of data transmitted in the current preset time period. Optionally, each piece of data is each piece of log data.

[0089] The average value of the size of each piece of data is calculated to obtain the average size of the plurality of data. The second data transmission rate is the ratio of the number of data theoretically transmittable in the next preset time period to the data length of the next preset time period, wherein the number of data is determined based on the size of each piece of data. When the available bandwidth prediction value is constant, the smaller the average size of the plurality of data, the more the number of data theoretically transmittable in the next preset time period, and the greater the second data transmission rate; the greater the average size of the plurality of data, the less the number of data theoretically transmittable in the next preset time period, and the smaller the second data transmission rate.

[0090] The present application calculates the second data transmission rate of the next preset time period based on the average size of the plurality of data transmitted in the current preset time period and the available bandwidth prediction value, which considers the characteristics of the data transmission behavior in the data transmission system, so that the second data transmission rate not only conforms to the running state of the data transmission system, but also conforms to the data transmission habit in the data transmission system, further improving the accuracy and flexibility of determining the data transmission rate.

[0091] In some embodiments, the data transmission system is controlled based on the target transmission rate in the next preset time period, including: obtaining the real-time resource value of at least one resource in the data transmission system in the next preset time period; determining the transmission rate adjustment direction and the transmission rate adjustment amount according to the difference between the preset optimal resource value and the real-time resource value corresponding to each of the at least one resource, and the absolute value of the transmission rate adjustment amount is positively correlated with the difference; controlling the data transmission system based on the target transmission rate and the transmission rate adjustment amount.

[0092] The real-time resource value includes at least one of the central processor utilization rate, the memory bandwidth, and the communication link bandwidth.

[0093] The real-time resource value is obtained according to the preset acquisition frequency to obtain the real-time resource fluctuation in the data transmission system. The preset optimal resource value refers to the efficient running standard value preset for the data transmission system, and when the real-time resource value of the data transmission system is the preset optimal resource value, the data transmission performance is optimal.

[0094] Optionally, the direction and amount of transmission rate adjustment are determined based on the difference between the preset optimal resource value and the real-time resource value corresponding to at least one resource, including: if the real-time resource value of at least one resource is less than the preset optimal resource value, then the direction of transmission rate adjustment is determined to be the direction of decreasing transmission rate; or, if the real-time resource value of at least one resource is greater than the preset optimal resource value, then the direction of adjustment is determined to be the direction of increasing transmission rate.

[0095] If the difference between the preset optimal resource value and the real-time resource value is negative, it means that the real-time resource value exceeds the preset optimal resource value, and the current data transmission system is in an overloaded state, so the target transmission rate needs to be reduced; if the difference between the preset optimal resource value and the real-time resource value is positive, it means that the real-time resource value has not reached the preset optimal resource value, and the current data transmission system has surplus resources, so the target transmission rate can be increased.

[0096] Furthermore, the greater the difference between the preset optimal resource value and the real-time resource value, the greater the gap between the current state of the data transmission system and the ideal state, requiring a larger transmission rate adjustment amount to adjust as quickly as possible; the smaller the difference between the preset optimal resource value and the real-time resource value, the smaller the gap between the current state of the data transmission system and the ideal state, allowing for fine-tuning with a smaller data transmission rate adjustment amount.

[0097] Specifically, when the real-time resource value includes multiple items, the method for calculating the difference between the preset optimal resource value and the real-time resource value is as follows: for each resource in at least one item, calculate the difference between the preset optimal resource value and the real-time resource value corresponding to the resource; obtain the preset resource weight for each resource in at least one item; based on the preset resource weight, perform a weighted summation on the difference corresponding to each resource to obtain the difference between the real-time resource value of at least one resource and the preset optimal resource data corresponding to each of the at least one resource.

[0098] Optionally, for percentage-based real-time resource values, such as CPU utilization, the difference between the preset optimal resource value and the real-time resource value can be directly calculated. For non-percentage-based real-time resource values, such as memory bandwidth and communication link bandwidth, the data needs to be normalized to percentage values ​​before calculation to ensure that the calculation dimension of the difference of multiple resources is consistent.

[0099] The preset resource weight represents the degree of influence of a resource on the data transmission system. The higher the preset resource weight, the greater the influence of the resource on the data transmission system, and the greater the impact of the difference of the resource on the final difference value. The lower the preset resource weight, the smaller the influence of the resource on the data transmission system, and the smaller the impact of the difference of the resource on the final difference value.

[0100] On this basis, for the difference value between the preset optimal resource value and the real-time resource value, the PID (Proportional-Integral-Differential) algorithm can also be used to process the instantaneous difference value at the current moment, the cumulative difference value from the start moment of the preset time period to the current moment, and the change rate of the difference value, to obtain the final difference value at the current moment by combining the PID coefficient for calculation.

[0101] The embodiments of the present application can fine-tune the transmission rate based on the resource occupation, and consider the influence weight of the difference value on the transmission rate adjustment value based on the weight of each resource, so that the data transmission based on the target transmission rate can be dynamically adjusted according to the real-time state of the current data transmission system, the error between the available bandwidth prediction value and the actual bandwidth value is compensated, and the flexibility and adaptability of data transmission are further improved.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software and the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation.

[0103] Figure 3 The data transmission device provided by the embodiments of the present application is shown in the structural schematic diagram. The data transmission device provided by the embodiments of the present application can execute the processing flow provided by the data transmission method embodiments, such as Figure 3As shown, the data transmission device 30 comprises: a first acquisition module 31, a first determination module 32, a second determination module 33, a third determination module 34, a weighted summation module 35, and a control module 36; the first determination module 32 is configured to acquire running state data of a data transmission system in a current preset time period, the data transmission system comprising a data sending end, a data receiving end, and a communication link between the data sending end and the data receiving end, and the running state data comprising a first data transmission rate of the communication link in the current preset time period; the first determination module 32 is configured to determine, based on the running state data, a predicted value of available bandwidth of the communication link in a next preset time period and a probability of occurrence of abnormal traffic in the data transmission system; the second determination module 33 is configured to determine a data transmission rate in the next preset time period according to the predicted value of available bandwidth, to obtain a second data transmission rate, the second data transmission rate being positively correlated with the predicted value of available bandwidth; the third determination module 34 is configured to determine a weight of the second data transmission rate according to the probability of occurrence of abnormal traffic, the weight of the second data transmission rate being negatively correlated with the probability of occurrence of abnormal traffic in the data transmission system; the weighted summation module 35 is configured to perform weighted summation on the first data transmission rate and the second data transmission rate, to obtain a target transmission rate in the next preset time period; and the control module 36 is configured to control the data transmission system based on the target transmission rate in the next preset time period.

[0104] Optionally, the first determination module 32 comprises a first input unit, a first calculation unit, and a first determination unit; the first input unit is configured to input the running state data into a pre-trained bandwidth prediction model, to obtain a bandwidth occupation prediction value output by the bandwidth prediction model; the first calculation unit is configured to, when the bandwidth occupation prediction value is less than or equal to a preset bandwidth threshold, calculate a difference between the preset bandwidth threshold and the bandwidth occupation prediction value, to obtain the predicted value of available bandwidth; and the first determination unit is configured to, when the bandwidth occupation prediction value is greater than the preset bandwidth threshold, determine the predicted value of available bandwidth as a preset minimum bandwidth value and the probability of occurrence of abnormal traffic in the data transmission system as a preset maximum probability value.

[0105] Optionally, the first calculation unit is further configured to, for any data traffic in the data transmission system, acquire feature data of the data traffic from the running state data, the feature data comprising a plurality of feature fields and a feature value corresponding to each feature field; input the feature data into a pre-trained isolation forest model, to obtain a path length of each feature value in the isolation forest model; calculate an average path length corresponding to the data traffic according to the path length of each feature value in the isolation forest model; if the average path length is less than a preset length threshold, determine that the data traffic is abnormal traffic; and calculate a proportion of the abnormal traffic in a total number of data traffics in the current preset time period, to obtain the probability of occurrence of abnormal traffic in the data transmission system.

[0106] Optionally, the second determining module 33 is configured to obtain an average size of the plurality of data based on attribute information of the plurality of data transmitted in the current preset time period in the running state data; and obtain a second data transmission rate by calculating a ratio of the available bandwidth prediction value to the average size.

[0107] Optionally, the control module 36 comprises an acquisition unit, a second determining unit and a control unit. The acquisition unit is configured to acquire real-time resource values of at least one resource in the data transmission system in the next preset time period. The second determining unit is configured to determine a transmission rate adjustment direction and a transmission rate adjustment amount based on a difference value between the preset optimal resource value and the real-time resource value of each of the at least one resource, and the absolute value of the transmission rate adjustment amount is positively correlated with the difference value. The control unit is configured to control the data transmission system based on the target transmission rate and the transmission rate adjustment amount.

[0108] Optionally, the control module 36 further comprises a second calculation unit configured to calculate, for each of the at least one resource, a difference value obtained by subtracting the real-time resource value from the preset optimal resource value of the resource; acquire a preset resource weight of each of the at least one resource; and obtain a difference value between the real-time resource value of the at least one resource and the preset optimal resource data of each of the at least one resource by weighted summing the difference value of each of the at least one resource based on the preset resource weight.

[0109] Optionally, the second determining unit is specifically configured to determine that the transmission rate adjustment direction is a direction in which the transmission rate decreases if the real-time resource value of the at least one resource is less than the preset optimal resource data; or determine that the adjustment direction is a direction in which the transmission rate increases if the real-time resource value of the at least one resource is greater than the preset optimal resource data.

[0110] Optionally, the real-time resource value comprises at least one of a central processing unit utilization rate, a memory bandwidth and a communication link bandwidth.

[0111] Optionally, the control module 36 is further configured to determine the target transmission rate as a preset minimum rate if it is determined that the probability of abnormal traffic occurring in the data transmission system is greater than a preset probability threshold; and control the data transmission system to transmit at least one data with the highest priority among the plurality of to-be-transmitted data based on the preset minimum rate in the next preset time period.

[0112] The features of the embodiments of the data transmission device can be referred to the related descriptions of the embodiments of the data transmission method, which will not be repeated here.

[0113] Embodiments of the present application also provide an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above data transmission method embodiments.

[0114] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is arranged to execute the steps in any of the above data transmission method embodiments when running.

[0115] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0116] The embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps in any of the above data transmission method embodiments.

[0117] The embodiment of the present application further provides another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above data transmission method embodiments.

[0118] The skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0119] The above provides a detailed description of a data transmission method and an electronic device. The principles and implementation modes of the present application are described by applying specific examples. The above description of the examples is only applicable to help understand the method and its core idea. It should be noted that for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data transmission method, characterized in that, include: The system acquires operational status data of a data transmission system within a current preset time period. The data transmission system includes a data sending end, a data receiving end, and a communication link between the data sending end and the data receiving end. The operational status data includes a first data transmission rate of the communication link within the current preset time period. Based on the operational status data, the predicted available bandwidth of the communication link in the next preset time period is determined, as well as the probability of abnormal traffic occurring in the data transmission system; The data transmission rate in the next preset time period is determined based on the available bandwidth prediction value to obtain a second data transmission rate, wherein the second data transmission rate is positively correlated with the available bandwidth prediction value; The weight of the second data transmission rate is determined based on the probability of abnormal traffic occurring, and the weight of the second data transmission rate is negatively correlated with the probability of abnormal traffic occurring in the data transmission system. The first data transmission rate and the second data transmission rate are weighted and summed to obtain the target transmission rate for the next preset time period. The data transmission system is controlled based on the target transmission rate during the next preset time period; Wherein, controlling the data transmission system based on the target transmission rate within the next preset time period includes: Within the next preset time period, obtain the real-time resource value of at least one resource in the data transmission system; The transmission rate adjustment direction and the transmission rate adjustment amount are determined based on the difference between the preset optimal resource value corresponding to each of the at least one resource and the real-time resource value, wherein the absolute value of the transmission rate adjustment amount is positively correlated with the difference value. The data transmission system is controlled based on the target transmission rate and the transmission rate adjustment amount.

2. The data transmission method according to claim 1, characterized in that, The step of determining the predicted available bandwidth of the communication link in the next preset time period based on the operational status data, and the probability of abnormal traffic occurring in the data transmission system, includes: The operating status data is input into a pre-trained bandwidth prediction model to obtain the bandwidth occupancy prediction value output by the bandwidth prediction model. When the predicted bandwidth usage value is less than or equal to a preset bandwidth threshold, the difference between the preset bandwidth threshold and the predicted bandwidth usage value is calculated to obtain the predicted available bandwidth value. When the predicted bandwidth usage value is greater than the preset bandwidth threshold, the predicted available bandwidth value is determined to be the preset minimum bandwidth value, and the probability of abnormal traffic occurring in the data transmission system is the preset maximum probability value.

3. The method according to claim 2, characterized in that, When the predicted bandwidth occupancy value is less than or equal to a preset bandwidth threshold, after calculating the difference between the preset bandwidth threshold and the predicted bandwidth occupancy value to obtain the predicted available bandwidth value, the method further includes: For any data traffic in the data transmission system, the feature data of the data traffic is obtained from the running status data. The feature data includes multiple feature fields and a feature value corresponding to each feature field. The feature data is input into a pre-trained isolated forest model to obtain the path length of each feature value in the isolated forest model; Calculate the average path length corresponding to the data traffic based on the path length of each of the aforementioned feature values ​​in the isolated forest model; If the average path length is less than a preset length threshold, the data traffic is determined to be abnormal traffic. The probability of abnormal traffic occurring in the data transmission system is obtained by calculating the proportion of the abnormal traffic to the total data traffic within the current preset time period.

4. The data transmission method according to claim 1, characterized in that, The step of determining the data transmission rate within the next preset time period based on the available bandwidth prediction value to obtain the second data transmission rate includes: Based on the attribute information of multiple data transmitted within the current preset time period in the operation status data, the average size of the multiple data is obtained; The second data transmission rate is obtained by calculating the ratio of the predicted available bandwidth to the average size.

5. The data transmission method according to claim 1, characterized in that, After obtaining the real-time resource value of at least one resource in the data transmission system, the method further includes: For each of the at least one resource, calculate the difference between the preset optimal resource value corresponding to the resource and the real-time resource value; Obtain the preset resource weight for each of the at least one resource; Based on the preset resource weights, the differences corresponding to each resource are weighted and summed to obtain the difference between the real-time resource value of the at least one resource and the preset optimal resource data corresponding to the at least one resource.

6. The data transmission method according to claim 5, characterized in that, The step of determining the transmission rate adjustment direction and the transmission rate adjustment amount based on the difference between the preset optimal resource value corresponding to each of the at least one resource and the real-time resource value includes: If the real-time resource value of at least one of the resources is less than the preset optimal resource data, then the direction of the transmission rate adjustment is determined to be the direction of decreasing transmission rate; or, If the real-time resource value of at least one resource is greater than the preset optimal resource data, then the adjustment direction is determined to be the direction of increasing the transmission rate.

7. The data transmission method according to any one of claims 1 or 5-6, characterized in that, The real-time resource values ​​include at least one of central processing unit utilization, memory bandwidth, and communication link bandwidth.

8. The data transmission method according to claim 1, characterized in that, After determining the predicted available bandwidth of the communication link in the next preset time period based on the operational status data, and the probability of abnormal traffic occurring in the data transmission system, the method further includes: If it is determined that the probability of abnormal traffic occurring in the data transmission system is greater than a preset probability threshold, then the target transmission rate is determined to be a preset minimum rate. Within the next preset time period, the data transmission system is controlled to transmit at least one of the highest priority data among multiple data to be transmitted, based on the preset minimum rate.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data transmission method as described in any one of claims 1 to 8 when executing the computer program.

Images