An artificial intelligence-based data intelligent optimization system and method

Abstract

The application relates to the technical field of data communication and artificial intelligence, and particularly discloses a data intelligent optimization system and method based on artificial intelligence, which comprises a feature curve acquisition module, a period determination module, a node grouping module, a priority calculation module, a resource allocation module and a data scheduling module. According to the application, the traffic feature curve of each data stream is acquired first, and the traffic fluctuation period is determined accordingly; then, the nodes are grouped by using K-means clustering combined with a genetic algorithm optimization; then, the transmission priority weight is calculated by comprehensively considering the hardware performance parameters and the traffic stability; the bandwidth resources are allocated according to the weight, the basic demand of the low-priority nodes is ensured, and the dynamic adjustment is adapted to the network load change; finally, a multi-objective optimization model is constructed by using a deep reinforcement learning algorithm DQN, the data stream is dynamically scheduled according to the bandwidth allocation result, the delay, the packet loss rate and the bandwidth utilization rate are taken as the optimization targets, the efficient data transmission is realized, and the network transmission efficiency and performance are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data communication and artificial intelligence, in particular to a data intelligent optimization system and method based on artificial intelligence. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, data transmission optimization has gradually become a key link to improve system performance. Especially in the application scenarios of Internet of Things, cloud computing and big data, an efficient data transmission scheme can significantly reduce delay, reduce bandwidth occupation and improve overall system efficiency.

[0003] However, the existing data transmission optimization method still has certain deficiencies in the aspects of intelligence degree, adaptability and resource allocation, and it is difficult to fully meet the dynamic needs in complex scenarios. Including the following aspects:

[0004] The traditional traffic collection method adopts fixed interval sampling, and does not effectively process noise data, resulting in that the traffic characteristic curve is seriously disturbed by instantaneous fluctuation, and the real change law of data flow cannot be accurately reflected; network traffic usually has periodic fluctuation characteristics (such as daily cycle law of daily peak and night valley, or hourly cycle law of business peak period), but the existing technology mostly relies on artificial experience to set the period, or uses time domain analysis method to roughly estimate the period, and cannot accurately extract the inherent periodicity of traffic fluctuation. This leads to that the resource allocation strategy lags behind the traffic change, and bandwidth shortage easily occurs in peak period, while resource idling exists in trough period, reducing the overall bandwidth utilization.

[0005] The traditional node grouping is mostly based on physical location or static configuration, without considering the similarity of node traffic fluctuation law, resulting in that nodes with similar traffic characteristics cannot realize collaborative resource scheduling; at the same time, when calculating node transmission priority, only the hardware performance parameters (such as processor frequency and interface rate) are relied on, ignoring the influence of traffic stability on transmission reliability; or static weight allocation bandwidth is adopted, which cannot dynamically respond to network load change. For example, if the high hardware performance node has violent traffic fluctuation, forced allocation of high bandwidth may lead to resource waste; and if the low priority node cannot obtain basic bandwidth guarantee for a long time, it will cause business transmission interruption, and it is difficult to realize the resource allocation goal of "efficiency first and fairness". SUMMARY

[0006] The purpose of the present application is to provide a data intelligent optimization system and method based on artificial intelligence to solve the problems raised in the background.

[0007] The purpose of the present application can be realized by the following technical scheme:

[0008] The first aspect of the present application provides a data intelligent optimization system based on artificial intelligence, which comprises:

[0009] characteristic curve acquisition module, configured to acquire traffic characteristic curves of data flows in network transmission nodes, the traffic characteristic curves being used to represent traffic change trends of the data flows in the network transmission nodes at different time points;

[0010] cycle determination module, configured to determine traffic fluctuation cycles of the network transmission nodes according to the traffic characteristic curves of the network transmission nodes, the traffic fluctuation cycles being used to represent regular time periods of traffic changes of the data flows in the network transmission nodes;

[0011] node grouping module, configured to group the network transmission nodes according to the traffic fluctuation cycles of the network transmission nodes, to obtain a plurality of transmission node groups;

[0012] priority calculation module, configured to calculate transmission priority weights of the network transmission nodes in each transmission node group in combination with hardware performance parameters of the network transmission nodes in the transmission node group;

[0013] resource allocation module, configured to allocate bandwidth resources of the network transmission nodes according to the transmission priority weights of the network transmission nodes and the number of the network transmission nodes in the transmission node group;

[0014] data scheduling module, configured to dynamically schedule and transmit the data flows through the bandwidth resource allocation results of the network transmission nodes.

[0015] The second aspect of the application provides a data intelligent optimization method based on artificial intelligence, which comprises the following steps:

[0016] Step 1: acquiring traffic characteristic curves of data flows in network transmission nodes, the traffic characteristic curves being used to represent traffic change trends of the data flows in the network transmission nodes at different time points;

[0017] Step 2: determining traffic fluctuation cycles of the network transmission nodes according to the traffic characteristic curves of the network transmission nodes, the traffic fluctuation cycles being used to represent regular time periods of traffic changes of the data flows in the network transmission nodes;

[0018] Step 3: grouping the network transmission nodes according to the traffic fluctuation cycles of the network transmission nodes, to obtain a plurality of transmission node groups;

[0019] Step 4: calculating transmission priority weights of the network transmission nodes in each transmission node group in combination with hardware performance parameters of the network transmission nodes in the transmission node group;

[0020] Step 5: allocating bandwidth resources of the network transmission nodes according to the transmission priority weights of the network transmission nodes and the number of the network transmission nodes in the transmission node group;

[0021] Step six, through the bandwidth resource allocation results of each network transmission node, the data flow is dynamically scheduled and transmitted.

[0022] The beneficial effects of the present application are:

[0023] The present application accurately captures the traffic trend of data flow through the sliding window sampling and smoothing processing technology of the characteristic curve acquisition module, providing a high-quality data basis for subsequent analysis; combined with the fast Fourier transform (FFT) frequency domain analysis method of the cycle determination module, it can deeply mine the periodicity of traffic fluctuation, overcome the limitations of traditional methods relying on experience judgment or single time domain analysis, and realize intelligent perception of the data transmission process through fine extraction and rule modeling of traffic characteristics, providing a scientific basis for subsequent optimization decision.

[0024] The present application accurately divides network transmission nodes with similar traffic fluctuation periods into the same group by using the K-means clustering algorithm combined with the genetic algorithm to optimize the initial center point. This grouping method based on traffic rules ensures that the nodes in the group have similar transmission load characteristics, avoiding the blindness of traditional "one-size-fits-all" resource allocation; after grouping, different resource strategies are developed for different node groups, making resource scheduling more targeted and significantly improving the utilization efficiency of overall network resources; at the same time, considering the node hardware performance parameters (processor frequency, memory capacity, network interface rate, etc.) and the stability of traffic fluctuation period, the transmission priority weight is generated through normalization processing and weighted scoring mechanism, which dynamically reflects the transmission capacity and stability demand of the node, so that the resource allocation module can tilt to high-priority nodes when allocating bandwidth, ensuring that critical business nodes can obtain sufficient resource support during traffic peak period, while taking into account the basic transmission needs of low-priority nodes, achieving the goal of "efficiency first, fairness second" resource allocation.

[0025] The application adopts a proportional fair allocation algorithm and introduces a real-time load monitoring and dynamic adjustment mechanism, so that when network load fluctuates, the system can quickly recalculate node priority weights and adjust the bandwidth allocation scheme, effectively dealing with dynamic scenarios such as burst traffic and node failure. Compared with the existing static allocation method, the dynamic adjustment mechanism of the application enables the system to maintain efficient operation in a complex network environment, effectively improves bandwidth utilization to a certain extent, and significantly reduces the transmission congestion problem caused by insufficient resources. A multi-objective optimization model is constructed based on deep reinforcement learning, with delay, packet loss rate and bandwidth utilization as the core optimization objectives. By simulating different network scenarios to train the scheduling strategy and dynamically adjusting the transmission path and order of data flow according to the real-time network state, the transmission delay and packet loss rate are reduced while the bandwidth resource utilization is maximized. Compared with the existing single-objective optimization method, the application realizes the comprehensive improvement of multi-dimensional performance indicators, meeting the demand of modern data transmission for 'high efficiency, stability and low consumption'. BRIEF DESCRIPTION OF DRAWINGS

[0026] The application will be further described below with reference to the drawings.

[0027] Figure 1 is a schematic diagram of the connection of modules of the system of the application.

[0028] Figure 2 is a flowchart of the generation and processing of the traffic characteristic curve of the application.

[0029] Figure 3 is a flowchart of the method implementation steps of the application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0031] Please refer to Figure 1 Fig. 1 shows an artificial intelligence-based data intelligent optimization system, which includes a characteristic curve acquisition module, a period determination module, a node grouping module, a priority calculation module, a resource allocation module and a data scheduling module. The characteristic curve acquisition module is connected to the period determination module, the period determination module is connected to the node grouping module, the node grouping module is connected to the priority calculation module, the priority calculation module is connected to the resource allocation module, and the resource allocation module is connected to the data scheduling module.

[0032] The feature curve acquisition module is configured to acquire a traffic feature curve of each data flow in the network transmission node, and the traffic feature curve is used to represent a traffic change trend of the data flow at different time points in the network transmission node.

[0033] Specifically, acquiring the traffic feature curve of each data flow in the network transmission node comprises:

[0034] 101: initialization: determining a fixed time interval of the sliding window algorithm, i.e., a sampling period; the time interval needs to be set according to the actual traffic change frequency of the network transmission node and the data acquisition requirement. For example, for a high-speed network and a frequently changing traffic scenario, the sampling period can be set to a short time, such as 100 milliseconds; and for a low-speed network or a relatively slow traffic change scenario, the sampling period can be appropriately extended, such as 1 second.

[0035] Setting a fixed length of the buffer area; the length of the buffer area determines the number of sampled data each time, which jointly affects the accuracy and real-time performance of the traffic feature curve. A too long buffer area length will increase the calculation complexity and memory occupation, but can more accurately reflect the traffic change trend; a too short buffer area length may lose part of the traffic information. For example, a suitable buffer area length can be estimated according to the average traffic size of the network transmission node and the sampling period, assuming that the average traffic is 100 data packets per second and the sampling period is 100 milliseconds, the buffer area length can be initially set to 10 data packets.

[0036] Configuring parameters of the Gaussian filter, including a filter window size and a standard deviation; the filter window size determines the number of data points participating in the smoothing process, and the standard deviation controls the degree of smoothing. Generally, the larger the window and the larger the standard deviation, the more obvious the smoothing effect, but it may be over-smoothed to lose some traffic details. For example, the filter window size can be initially set to 5 data points and the standard deviation can be set to 1 according to experience, and subsequent adjustments can be made according to the actual effect.

[0037] 102: traffic data sampling: starting a timer according to the set fixed time interval, and the timer triggers a data sampling operation once every sampling period; when the timer is triggered, the traffic data of the current time period is collected from the network transmission node, and the collected data can include traffic-related indicators such as the number of data packets and the number of bytes, for example, by reading the network device counter to obtain the number of data packets and the number of bytes passing through the data flow in the sampling period; the collected traffic data is stored in the buffer area with a set fixed length, and when the buffer area is not full, the data is continuously collected and stored; when the buffer area is full, the traffic value calculation and smoothing process are entered.

[0038] 103: Flow value calculation and smoothing processing: When the buffer is full, the average value of all data in the buffer is calculated. For example, if the buffer stores the number of data packets, the number of data packets of all time periods in the buffer is added up and divided by the length of the buffer to obtain the average number of data packets at the current time point as the flow value at the time point;

[0039] The calculated flow value is smoothed using a Gaussian filter. The specific steps are as follows: taking the currently calculated flow value as the center, a window containing a certain number of adjacent flow values is determined. The window size is determined by the previously configured Gaussian filter parameters. For example, if the filter window size is 5, the window contains the current flow value and the previous and next 2 flow values. The weight of each flow value in the window is calculated according to the Gaussian distribution formula: wherein represents the weight of the i-th data point, represents the standard deviation, represents the center position of the window (corresponding to the position of the current flow value), after calculating the weight of each data point in the window, normalization processing is performed to make the sum of all weights equal to 1; each flow value in the window is multiplied by its corresponding weight, and then all results are added to obtain the smoothed flow value, the formula is: wherein represents the smoothed flow value, represents the i-th flow value in the window, represents the corresponding weight, and n represents the data point flow in the window.

[0040] 104: Flow feature curve generation: after calculating and smoothing the flow value each time, the current timestamp is recorded. The timestamp is used to identify the time point corresponding to the flow value, so that the trend of flow change over time can be accurately reflected subsequently; the recorded timestamp and the corresponding smoothed flow value form a data point, for example (t, y), wherein t is the timestamp and y is the smoothed flow value. With the passage of time, the above sampling, calculation, smoothing processing and recording data point process is repeated, and the data points at each time point are connected in time sequence to form a flow feature curve that can reflect the trend of data flow change over time.

[0041] It should be noted that, as Figure 2 shown, the process starts from data acquisition, and after sliding window sampling and smoothing processing, the flow feature curve is finally output. The flow feature curve takes time as the horizontal axis and flow size as the vertical axis, forming a trend chart that can accurately characterize the trend of data flow change. This method effectively eliminates data fluctuations caused by short-term noise interference, ensuring the accuracy of the flow feature curve.

[0042] A period determination module is configured to determine a traffic fluctuation period of each network transmission node according to each traffic characteristic curve in each network transmission node, and the traffic fluctuation period is used to represent a regular time period of data flow traffic change in the network transmission node.

[0043] Specifically, the determination of the traffic fluctuation period of each network transmission node includes:

[0044] 201: Discrete processing of the traffic characteristic curve: time alignment operation is performed on the obtained traffic characteristic curve. Since the traffic characteristic curves of different data flows may have slight deviations in time, in order to perform subsequent unified analysis, the time origins of the traffic characteristic curves are unified to a common time reference.

[0045] According to the set discrete interval, the traffic characteristic curve after time alignment is sampled at equal intervals. Starting from the unified time origin, the traffic values on the traffic characteristic curve are read in turn according to the discrete interval, and they are converted into a series of equal-interval-sampled numerical sequences, and the lengths of the traffic characteristic curves of different data flows are unified. For example, the discrete interval is 1 second, starting from time t=0, a traffic value is read every 1 second , and a numerical sequence is obtained; it should be noted that since the lengths of the traffic characteristic curves of different data flows may be different, in order to facilitate subsequent FFT calculation, the lengths of the numerical sequences need to be unified. The longest length in all numerical sequences can be selected as the unified length, and for shorter numerical sequences, zeros are added at the end to make the length reach the unified length. For example, there are three numerical sequences with lengths of 100, 120 and 150, and the unified length is set to 150. The lengths of the sequences of 100 and 120 are supplemented with 50 and 30 zeros at the end, respectively.

[0046] 202: Fast Fourier transform calculation: a selected fast Fourier transform (FFT) algorithm is used to calculate the preprocessed numerical sequence, and the numerical sequence is converted from the time domain to the frequency domain; it should be noted that the FFT algorithm can be a radix-2 FFT algorithm or a mixed-radix algorithm, and the result of the FFT calculation is a complex number sequence, where each complex number represents the amplitude and phase information of a frequency component. For example, for a numerical sequence with a length of N, after FFT calculation, a complex number sequence with a length of N is obtained , where k=0, 1, …, N−1.

[0047] The frequency spectrum information is extracted from the FFT calculation result, and the amplitude spectrum corresponding to each frequency component is calculated, and the calculation formula of the amplitude spectrum is , where respectively represent the real part and the imaginary part of the complex number , and after obtaining the amplitude spectrum, the amplitude spectrum is drawn into a frequency spectrum graph.

[0048] 203: Peak frequency extraction and period calculation: find the peak frequency in the spectrum, set an amplitude threshold, in the spectrum, traverse all frequency components, mark the frequency components with amplitude exceeding the set amplitude threshold as candidate peak frequencies, compare each candidate peak frequency with the set frequency range interval, if a candidate peak frequency is within the set frequency range interval, the candidate peak frequency is retained, otherwise the candidate peak frequency is not retained, thereby screening the final peak frequency;

[0049] If there is a peak frequency, the corresponding period length is calculated according to the formula , and the period length is taken as the flow fluctuation period, where f represents the peak frequency;

[0050] If there are multiple peak frequencies, the weighted average period is calculated according to the formula , and the weighted average period is taken as the flow fluctuation period, where represents the zth peak frequency, represents the amplitude corresponding to the zth peak frequency;

[0051] Thus, the flow fluctuation period of each network transmission node is determined.

[0052] A node grouping module is configured to group the network transmission nodes according to the flow fluctuation periods of the network transmission nodes, to obtain a plurality of transmission node groups.

[0053] Specifically, grouping the network transmission nodes includes:

[0054] 301: Initialize K-means clustering parameters: determine the number of clusters k of K-means clustering by elbow rule, and initialize k cluster center points , randomly select k and the flow fluctuation periods of the network transmission nodes as initial center points; it should be noted that the elbow rule is a common method for determining the optimal number of clusters k in K-means clustering, by plotting the cluster within sum of squares (SSE) variation curve corresponding to different k values, finding the "elbow point" after the curve sharply decreases and tends to be flat as the optimal k value.

[0055] 302: Genetic algorithm optimization of initial center points:

[0056] 302-1: Encoding: encode the k cluster center points; for example, real number encoding can be used, and each center point can be represented as a real number vector;

[0057] 302-2: Initial population generation: randomly generate a certain number of initial populations, and each individual represents a group of possible k cluster center points;

[0058] 302-3: Fitness function design: design a fitness function to evaluate the pros and cons of each individual, and take the "sum of squared errors within groups (SSE)" of K-means clustering as the fitness evaluation index. The smaller the SSE, the better the clustering effect and the higher the fitness. The calculation formula is: ;

[0059] 302-4: Selection operation: adopt roulette wheel selection method to select individuals with higher fitness from the current population as parents for generating the next generation population;

[0060] 302-5: Crossover operation: crossover operation is performed on the selected parent individuals to generate new individuals. For example, single-point crossover method is adopted, and a crossover point is randomly selected to exchange the partial genes of the two parent individuals;

[0061] 302-6: Mutation operation: mutation operation is performed on the newly generated individuals to increase the diversity of the population. For example, a gene site is randomly selected and slightly disturbed;

[0062] 302-7: Iterative optimization: selection, crossover and mutation operations are performed again until the preset number of iterations is reached, and the genetic algorithm is terminated. The individual with the highest fitness in the current population is selected as the optimal initial center point .

[0063] 303: K-means clustering grouping execution:

[0064] 303-1: Assign nodes to clusters: for each network transmission node, calculate the distance between its traffic fluctuation period and the optimized k cluster center points by using the Euclidean distance formula. According to the principle of minimum distance, each node is assigned to the group closest to the cluster center point, forming k initial transmission node groups;

[0065] 303-2: Update cluster center points: for each cluster, recalculate the average value of the traffic fluctuation periods of all nodes in the cluster as the new cluster center point;

[0066] 303-3: Iterative clustering: repeat the execution steps of 301-1 and 301-2 until the cluster center points reach the preset number of iterations, complete the grouping of network transmission nodes, and obtain multiple transmission node groups .

[0067] It should be noted that the K-means clustering algorithm is used, with the traffic fluctuation period as the clustering feature, to divide network transmission nodes with similar traffic fluctuation periods into the same transmission node group; a genetic algorithm is introduced to optimize the initial centroid of K-means clustering, and the optimal initial centroid is selected iteratively to improve the accuracy and stability of grouping.

[0068] The priority calculation module is used to calculate the transmission priority weight of each network transmission node for each transmission node group, taking into account the hardware performance parameters of each network transmission node in the transmission node group.

[0069] Specifically, calculating the transmission priority weights of each network transmission node includes:

[0070] Collect hardware performance parameters of network transmission nodes, including processor clock frequency f, memory capacity m, and network interface speed r;

[0071] The collected hardware performance parameters are normalized to their maximum and minimum values, unifying the parameter values ​​to the [0,1] interval; the normalization formula is as follows: , of which Original parameter values, These are the minimum and maximum values ​​of the parameter, respectively; for example, for the processor clock speed f, assume its minimum value is... The maximum value is The normalized processor clock speed Similarly, normalizing the memory capacity m and network interface speed r yields... ;

[0072] The overall performance score is calculated by weighted summation. The formula for calculating the overall performance score is as follows: ,in, These represent the percentage weights corresponding to the preset comprehensive performance scores;

[0073] The standard deviation of the flow fluctuation period is calculated using the standard deviation calculation formula. Through formula The stability coefficient of the flow fluctuation period was calculated. It should be noted that the smaller the standard deviation of the traffic fluctuation period, the larger the stability coefficient of the traffic fluctuation period, indicating that the traffic fluctuation period is more stable.

[0074] Combined with comprehensive performance score With stability coefficient Adjust to obtain transmission priority weights ,in These represent the percentage weights of the overall performance score and stability coefficient corresponding to the preset transmission priority weights, respectively.

[0075] The resource allocation module is used to allocate bandwidth resources to each network transmission node based on the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group.

[0076] Specifically, the allocation of bandwidth resources to each network transmission node includes:

[0077] 501: Calculate the bandwidth allocation ratio: Use a proportional fair allocation algorithm to calculate the bandwidth allocation ratio for a single network transmission node. This ratio represents the transmission priority weight of that node. The sum of the priority weights of all nodes in the group The ratio, i.e. , where b represents the number of network transmission nodes in the transmission node group;

[0078] 502: Calculate the initial bandwidth allocation value: based on the total bandwidth resources and allocation ratio Calculate the initial bandwidth allocation value ,Right now ;

[0079] Iterate through all network transmission nodes, comparing the initial bandwidth allocation value with the set minimum bandwidth. If the initial bandwidth allocation value of a network transmission node is less than the set minimum bandwidth, then the network transmission node is determined to be a low-priority node. This process is then used to identify all low-priority nodes and calculate the bandwidth gap for each low-priority node. Here, k1 represents the number of each low-priority node, k1=1,2,...,l1; it should be noted that this gap represents the additional bandwidth that each node needs to allocate to meet basic transmission requirements; therefore, the total bandwidth gap is obtained by summing the bandwidth gaps of all low-priority nodes. ;

[0080] Based on the low-priority nodes, the remaining nodes are marked as adjustable nodes. The unused bandwidth potential of all adjustable nodes is summed to obtain the total adjustable bandwidth pool. It should be noted that the unused bandwidth potential of all adjustable nodes refers to the bandwidth allocation value minus the minimum bandwidth required for basic transmission.

[0081] Based on total bandwidth gap and adjustable bandwidth total pool According to the formula The adjustment ratio was calculated. ;

[0082] For each adjustable node k2, k2 represents the node number, k2=1, 2, ..., l2; its bandwidth allocation value is reduced according to the adjustment ratio, i.e., the new bandwidth allocation value. wherein is the original initial bandwidth allocation value of the adjustable node, is the minimum bandwidth that the adjustable node meets the basic transmission requirement;

[0083] It should be noted that through the above steps, while ensuring that the low-priority node meets the basic transmission requirement, the total bandwidth resource is ensured not to be over-allocated.

[0084] 503: dynamically adjusting bandwidth allocation: monitoring network load changes in real time, setting a preset threshold of load fluctuation When the load fluctuation exceeds the preset threshold, that is, the difference between the current network load and the network load at the last monitoring is greater than , the transmission priority weight is recalculated;

[0085] According to the recalculated transmission priority weight, the bandwidth allocation ratio and the initial bandwidth allocation value are recalculated according to the execution steps of 501-502, and the bandwidth allocation resource of each network transmission node is dynamically adjusted.

[0086] a data scheduling module, configured to dynamically schedule and transmit data flow through the bandwidth resource allocation results of each network transmission node.

[0087] Specifically, the dynamic scheduling and transmission of data flow comprises:

[0088] 601: constructing a multi-objective optimization model: using a deep reinforcement learning algorithm DQN to construct a multi-objective optimization model, with delay, packet loss rate and bandwidth utilization as optimization objectives;

[0089] 602: defining a state space: defining a state space as network real-time indicators, including node bandwidth usage, data flow transmission state, network delay, and combining these state indicators into a state vector;

[0090] 603: defining an action space: defining an action space as a data flow scheduling strategy, including transmission path selection and transmission order adjustment;

[0091] It should be noted that for transmission path selection, a set of all possible transmission paths can be generated in advance, and all feasible paths from the source node to the destination node can be generated by analyzing the network topology structure and using the path search algorithm in graph theory; at the same time, a unique action number is assigned to each transmission path, and the transmission path selection action is represented as a discrete action space; for transmission order adjustment, different transmission order rules are defined according to the priority, size, urgency and other factors of data flow, and an action number is assigned to each transmission order rule, and the transmission order adjustment action is also included in the action space.

[0092] 604: Design reward function: quantify the scheduling effect through the reward function, the reward value is positively correlated with the delay reduction, the packet loss rate reduction, and the bandwidth utilization improvement; let R be the reward value, is the delay reduction, is the packet loss rate reduction, is the bandwidth utilization improvement, then the reward function is expressed as: , wherein are the weight coefficients of delay, packet loss rate and bandwidth utilization respectively, and .

[0093] 605: Model training: train the model by simulating different network environment scenarios, in each scenario, the model perceives the indicators in the state space according to the current network state to form the current state vector ;

[0094] According to the current state vector , the model uses the DQN algorithm to select an action , the DQN algorithm obtains the Q value of each possible action by querying the Q network, and selects the action with the maximum Q value as the execution action; it should be noted that the action can be transmission path selection or transmission order adjustment;

[0095] Execute the selected action , schedule and transmit the data flow; it should be noted that if the selected action is transmission path selection, the data flow will be transmitted according to the selected path; if the selected action is transmission order adjustment, the data flow will be sorted and transmitted according to the new transmission order;

[0096] After executing the action, the network state will be transferred, the model perceives the next state vector , and calculates the reward value ,

[0097] The current state, action, reward and next state form an experience tuple , and it is stored in the experience replay buffer; in the training process, a batch of experience tuples are randomly sampled from the experience replay buffer to update the parameters of the Q network; by continuously iterating this process, the model gradually learns the optimal action strategy in different states, thereby achieving efficient scheduling of data flow.

[0098] 606: Dynamic scheduling and transmission: After sufficient training, the model has the ability to dynamically schedule data streams according to the network state; in the actual network environment, the model perceives the network state in real time, selects the optimal action according to the current state vector, and dynamically schedules and transmits the data stream; at the same time, the model continuously monitors the changes of the network state, and when the network state changes significantly (such as sudden increase or decrease of network load, link failure, etc.), re-performs state perception, action selection and execution, etc. Steps, realize the dynamic and efficient transmission of data streams.

[0099] It should be noted that the data scheduling module can realize dynamic scheduling and transmission of data streams according to the actual state of the network and the bandwidth resource allocation result, and improve the transmission efficiency and performance of the network.

[0100] Please refer to Figure 3 The method comprises the following steps:

[0101] Step one, obtain the traffic characteristic curve of each data stream in the network transmission node, the traffic characteristic curve is used to represent the traffic change trend of the data stream in the network transmission node at different time points;

[0102] Step two, determine the traffic fluctuation period of each network transmission node according to each traffic characteristic curve in each network transmission node, the traffic fluctuation period is used to represent the regularity time period of the data stream traffic change in the network transmission node;

[0103] Step three, group the network transmission nodes according to the traffic fluctuation period of each network transmission node, and obtain a plurality of transmission node groups;

[0104] Step four, for each transmission node group, combine the hardware performance parameters of each network transmission node in the transmission node group to calculate the transmission priority weight of each network transmission node;

[0105] Step five, according to the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group, allocate the bandwidth resources of each network transmission node;

[0106] Step six, dynamically schedule and transmit the data stream through the bandwidth resource allocation result of each network transmission node.

[0107] The above content is only an example and description of the structure of the present application, and those skilled in the art can make various modifications or supplements or use similar ways to replace the described specific embodiments, as long as they do not deviate from the structure of the invention or exceed the scope defined by the present claims, which shall belong to the protection scope of the present application.

Claims

1. A data intelligence optimization system based on artificial intelligence, characterized in that, include: The characteristic curve acquisition module is used to acquire the traffic characteristic curves of each data stream in the network transmission node. The traffic characteristic curves are used to characterize the traffic change trend of the data stream in the network transmission node at different time points. The period determination module is used to determine the traffic fluctuation period of each network transmission node based on the traffic characteristic curves of each network transmission node. The traffic fluctuation period is used to characterize the regular time period of data flow changes in the network transmission node. The node grouping module is used to group network transmission nodes according to the traffic fluctuation cycle of each network transmission node, resulting in multiple transmission node groups. The priority calculation module is used to calculate the transmission priority weight of each network transmission node for each transmission node group, taking into account the hardware performance parameters of each network transmission node in the transmission node group. The resource allocation module is used to allocate bandwidth resources to each network transmission node based on the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. The data scheduling module is used to dynamically schedule and transmit data streams based on the bandwidth resource allocation results of each network transmission node. The acquisition of traffic characteristic curves for each data stream in the network transmission node includes: 101: Initialization: Determine the fixed time interval of the sliding window algorithm, i.e., the sampling period; set the fixed length of the buffer; configure the parameters of the Gaussian filter, including the filter window size and standard deviation; 102: Traffic Data Sampling: A timer is started according to a set fixed time interval. Every time the timer reaches a sampling period, a data sampling operation is triggered. When the timer is triggered, traffic data for the current time period is collected from the network transmission node and stored in a buffer of a set fixed length. When the buffer is not full, data collection and storage continue. When the buffer is full, the next step of traffic value calculation and smoothing processing is performed. 103: Traffic value calculation and smoothing: When the buffer is full, calculate the average value of all data in the buffer, and use a Gaussian filter to smooth the calculated traffic value; 104: Flow characteristic curve generation: After each calculation and smoothing process to obtain the flow value, the current timestamp is recorded. The recorded timestamp and the corresponding smoothed flow value are combined to form a data point. As time goes by, the above process of sampling, calculation, smoothing and recording data points is repeated continuously. The data points at each time point are connected in chronological order to form a flow characteristic curve that can reflect the trend of data flow flow over time. The determination of the traffic fluctuation period of each network transmission node includes: 201: Flow characteristic curve discretization processing: Perform time alignment operation on the acquired flow characteristic curve. According to the set discretization interval, sample the time-aligned flow characteristic curve at equal intervals. Starting from a unified time starting point, read the flow value on the flow characteristic curve in sequence according to the discretization interval, and convert it into a series of equally spaced sampled numerical sequences. At the same time, unify the length of the flow characteristic curve of different data streams. 202: Fast Fourier Transform Calculation: The selected Fast Fourier Transform (FFT) algorithm is used to calculate the preprocessed numerical sequence, transforming the numerical sequence from the time domain to the frequency domain; the spectral information is extracted from the FFT calculation results, the amplitude spectrum corresponding to each frequency component is calculated, and the amplitude spectrum is then plotted as a spectrum diagram. 203: Peak Frequency Extraction and Period Calculation: Find the peak frequency in the spectrum graph, set an amplitude threshold, traverse all frequency components in the spectrum graph, mark the frequency components whose amplitude exceeds the set amplitude threshold as candidate peak frequencies, compare each candidate peak frequency with the set frequency range interval, if a candidate peak frequency is within the set frequency range interval, then the candidate peak frequency will be retained, otherwise the candidate peak frequency will not be retained, thus obtaining the final peak frequency; If a peak frequency exists, according to the formula The corresponding period length is calculated. And this is taken as the flow fluctuation period, where f represents the peak frequency; If multiple peak frequencies exist, according to the formula The weighted average period was calculated. And take it as the cycle of traffic fluctuation, in which Represented as the z-th peak frequency, This is represented as the amplitude corresponding to the z-th peak frequency; This allows us to determine the traffic fluctuation period of each network transmission node; The grouping of network transmission nodes includes: 301: Initialize K-means clustering parameters: Determine the number of K-means clusters k using the elbow rule, and initialize k cluster centers. The initial center point is selected by randomly choosing k and the traffic fluctuation period of the network transmission node. 302: Genetic algorithm optimizes the initial center point, including: 302-1: Encoding: Encode the k cluster centers; 302-2: Initial population generation: Randomly generate a certain number of initial populations, where each individual represents a set of k possible cluster centers; 302-3: Fitness Function Design: Design a fitness function to evaluate the quality of each individual; 302-4: Selection operation: The roulette wheel selection method is used to select individuals with high fitness from the current population as parents to generate the next generation of the population; 302-5: Crossover operation: Performs a crossover operation on the selected parent individual to generate a new individual; 302-6: Mutation operation: Perform mutation operation on newly generated individuals to increase the diversity of the population; 302-7: Iterative Optimization: Repeat the selection, crossover, and mutation operations until the preset number of iterations is reached. Then, terminate the genetic algorithm and select the individual with the highest fitness in the current population as the optimal initial center. ; 303: K-means clustering grouping execution: 303-1: Assigning Nodes to Clusters: For each network transmission node, calculate its traffic fluctuation period and the optimized k cluster centers using the Euclidean distance formula. Based on the distance, each node is assigned to the group containing the nearest cluster center according to the minimum distance principle, forming k initial transmission node groups; 303-2: Update cluster centroids: For each cluster, recalculate the average of the traffic fluctuation periods of all nodes in the cluster as the new cluster centroids; 303-3: Iterative Clustering: Repeat the execution steps of 301-1 and 301-2 until the cluster centers reach the preset number of iterations, completing the grouping of network transmission nodes and obtaining multiple transmission node groups. .

2. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The calculation of the transmission priority weights for each network transmission node includes: Collect hardware performance parameters of network transmission nodes, including processor clock speed, memory capacity, and network interface speed; The hardware performance parameters are normalized to their maximum and minimum values, unifying the parameter values ​​to the [0,1] range; The overall performance score is calculated by weighted summation, and the stability coefficient of the flow fluctuation period is calculated at the same time. The stability coefficient is the reciprocal of the standard deviation of the flow fluctuation period. The transmission priority weight is obtained by combining the comprehensive performance score and the stability coefficient.

3. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The allocation of bandwidth resources to each network transmission node includes: 501: Calculate the bandwidth allocation ratio: Use the proportional fair allocation algorithm to calculate the bandwidth allocation ratio of a single network transmission node; 502: Calculate the initial bandwidth allocation value: Calculate the initial bandwidth allocation value based on the total bandwidth resources and allocation ratio; traverse all network transmission nodes, compare the initial bandwidth allocation value with the set minimum bandwidth, and if the initial bandwidth allocation value of a network transmission node is less than the set minimum bandwidth, then the network transmission node is determined to be a low-priority node. In this way, the low-priority nodes are counted and the bandwidth gap of each low-priority node is calculated. Then, the bandwidth gaps of all low-priority nodes are accumulated to obtain the total bandwidth gap. Based on each low-priority node, the remaining nodes are marked as adjustable nodes, and the unused bandwidth potential of all adjustable nodes is added together to obtain the total adjustable bandwidth pool. The adjustment ratio is calculated using the formula based on the total bandwidth gap and the total pool of adjustable bandwidth. For each adjustable node, its bandwidth allocation value is reduced according to the adjustment ratio to obtain a new bandwidth allocation value; 503: Dynamically adjust bandwidth allocation: Monitor network load changes in real time, set a preset threshold for load fluctuations, and recalculate transmission priority weights when load fluctuations exceed the preset threshold. Based on the recalculated transmission priority weights, the bandwidth allocation ratio and initial bandwidth allocation value are recalculated according to the execution steps 501-502, and the bandwidth allocation resources of each network transmission node are dynamically adjusted.

4. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The dynamic scheduling and transmission of the data stream includes: 601: Constructing a multi-objective optimization model: A multi-objective optimization model is constructed using the deep reinforcement learning algorithm DQN, with latency, packet loss rate, and bandwidth utilization as optimization objectives; 602: Define the state space: Define the state space as real-time network metrics, including bandwidth usage of each node, data stream transmission status, and network latency, and combine these metrics into a state vector. 603: Define Action Space: Define the action space as a data stream scheduling strategy, including transmission path selection and transmission order adjustment; 604: Design a reward function: Quantify the scheduling effect through a reward function. The reward value is positively correlated with the amount of latency reduction, packet loss rate reduction, and bandwidth utilization improvement. 605: Model Training: The model is trained by simulating different network environment scenarios. In each scenario, the model perceives various indicators in the state space based on the current network state and forms the current state vector. ; Based on the current state vector The model uses the DQN algorithm to select an action. The DQN algorithm obtains the Q-value of each possible action by querying the Q-network and selects the action with the largest Q-value as the action to be executed. Perform the selected action This involves scheduling and transmitting data streams. After an action is performed, the network state will be sent to transition, and the model will perceive the next state vector. And calculate the reward value based on the scheduling effect. , Combine the current state, action, reward, and next state into an experience tuple. The data is stored in the experience replay buffer. During training, a batch of experience tuples is randomly sampled from the experience replay buffer to update the parameters of the Q network. Through continuous iteration of this process, the model gradually learns the strategy of taking the optimal action in different states. 606: Dynamic Scheduling and Transmission: After sufficient training, the model perceives the network state in real time, selects the optimal action based on the current state vector, and dynamically schedules and transmits the data stream.

5. A data intelligence optimization method based on artificial intelligence, based on the data intelligence optimization system based on artificial intelligence as described in any one of claims 1-4, characterized in that, Includes the following steps: Step 1: Obtain the traffic characteristic curves of each data stream in the network transmission node; Step 2: Determine the traffic fluctuation period of each network transmission node based on the traffic characteristic curves of each network transmission node. Step 3: Based on the traffic fluctuation cycle of each network transmission node, group the network transmission nodes to obtain multiple transmission node groups; Step 4: For each transmission node group, calculate the transmission priority weight of each network transmission node based on the hardware performance parameters of each network transmission node in the transmission node group. Step 5: Allocate bandwidth resources to each network transmission node according to the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. Step 6: Dynamically schedule and transmit the data stream based on the bandwidth resource allocation results of each network transmission node.

Images