Data intelligent optimization system and method based on artificial intelligence

Abstract

The invention relates to the technical field of data communication and artificial intelligence, and particularly discloses an intelligent data optimization system and method based on artificial intelligence, and the system comprises a characteristic curve acquisition module, a period determination module, a node grouping module, a priority calculation module, a resource distribution module and a data scheduling module. The method comprises the following steps: firstly, acquiring a flow characteristic curve of each data flow, and determining a flow fluctuation period; then, K-means clustering is combined with genetic algorithm optimization, and nodes are grouped; then, integrating hardware performance parameters and flow stability, and calculating a transmission priority weight; bandwidth resources are allocated according to the weights, basic requirements of low-priority nodes are ensured, and dynamic adjustment is carried out to adapt to network load changes; and finally, constructing a multi-objective optimization model through a deep reinforcement learning algorithm DQN, dynamically scheduling a data flow according to a bandwidth allocation result, and realizing efficient data transmission by taking delay, a packet loss rate and a bandwidth utilization rate as optimization objectives, thereby effectively improving the network transmission efficiency and performance.

Description

Technical Field

[0001] This invention relates to the fields of data communication and artificial intelligence technology, specifically to a data intelligence optimization system and method based on artificial intelligence. Background Technology

[0002] With the rapid development of artificial intelligence technology, data transmission optimization has gradually become a key aspect of improving system performance. Especially in application scenarios such as the Internet of Things, cloud computing, and big data, efficient data transmission solutions can significantly reduce latency, reduce bandwidth consumption, and improve overall system efficiency.

[0003] However, existing data transmission optimization methods still have certain shortcomings in terms of intelligence, adaptability, and resource allocation, making it difficult to fully meet the dynamic needs of complex scenarios. These shortcomings include the following aspects: Traditional traffic acquisition methods often employ fixed-interval sampling, failing to effectively process noisy data. This results in traffic characteristic curves being severely affected by instantaneous fluctuations, failing to accurately reflect the true changing patterns of the data flow. Network traffic typically exhibits periodic fluctuations (such as daily cycles of weekday peaks and nighttime troughs, or hourly cycles during peak business periods). However, current technologies often rely on manual experience to set the period or use time-domain analysis methods to roughly estimate the period, failing to accurately extract the inherent periodicity of traffic fluctuations. This leads to resource allocation strategies lagging behind traffic changes, resulting in bandwidth shortages during peak periods and idle resources during off-peak periods, thus reducing overall bandwidth utilization.

[0004] Traditional node grouping is mostly based on physical location or static configuration, failing to consider the similarity of node traffic fluctuation patterns. This results in nodes with similar traffic characteristics being unable to achieve coordinated resource scheduling. Furthermore, when calculating node transmission priorities, it either relies solely on hardware performance parameters (such as processor clock speed and interface speed), ignoring the impact of traffic stability on transmission reliability; or it uses static weighted bandwidth allocation, failing to dynamically respond to changes in network load. For example, forcibly allocating high bandwidth to high-performance nodes with drastic traffic fluctuations may lead to resource waste; while low-priority nodes, if deprived of basic bandwidth guarantees for extended periods, will cause service transmission interruptions, making it difficult to achieve the resource allocation goal of "efficiency first, while also considering fairness." Summary of the Invention

[0005] The purpose of this invention is to provide a data intelligence optimization system and method based on artificial intelligence to solve the problems mentioned in the background.

[0006] The objective of this invention can be achieved through the following technical solutions: The first aspect of this invention provides a data intelligence optimization system based on artificial intelligence, the system comprising: The characteristic curve acquisition module is used to acquire the traffic characteristic curves of each data stream in the network transmission node. The traffic characteristic curves are used to characterize the traffic change trend of the data stream in the network transmission node at different time points. The period determination module is used to determine the traffic fluctuation period of each network transmission node based on the traffic characteristic curves of each network transmission node. The traffic fluctuation period is used to characterize the regular time period of data flow changes in the network transmission node. The node grouping module is used to group network transmission nodes according to the traffic fluctuation cycle of each network transmission node, resulting in multiple transmission node groups. The priority calculation module is used to calculate the transmission priority weight of each network transmission node for each transmission node group, taking into account the hardware performance parameters of each network transmission node in the transmission node group. The resource allocation module is used to allocate bandwidth resources to each network transmission node based on the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. The data scheduling module is used to dynamically schedule and transmit data streams based on the bandwidth resource allocation results of each network transmission node.

[0007] A second aspect of this invention provides a data intelligence optimization method based on artificial intelligence, the method comprising the following steps: Step 1: Obtain the traffic characteristic curves of each data stream in the network transmission node. The traffic characteristic curves are used to characterize the traffic change trend of the data stream in the network transmission node at different time points. Step 2: Based on the traffic characteristic curves of each network transmission node, determine the traffic fluctuation period of each network transmission node. The traffic fluctuation period is used to characterize the regular time period of data flow changes in the network transmission node. Step 3: Based on the traffic fluctuation cycle of each network transmission node, group the network transmission nodes to obtain multiple transmission node groups; Step 4: For each transmission node group, calculate the transmission priority weight of each network transmission node based on the hardware performance parameters of each network transmission node in the transmission node group. Step 5: Allocate bandwidth resources to each network transmission node according to the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. Step 6: Dynamically schedule and transmit the data stream based on the bandwidth resource allocation results of each network transmission node.

[0008] The beneficial effects of this invention are: This invention uses the sliding window sampling and smoothing technology of the feature curve acquisition module to accurately capture the flow change trend of the data stream, providing a high-quality data foundation for subsequent analysis. Combined with the Fast Fourier Transform (FFT) frequency domain analysis method of the period determination module, it can deeply explore the periodic patterns of flow fluctuations, overcoming the limitations of traditional methods that rely on experience judgment or single time domain analysis. Through the refined extraction and pattern modeling of flow characteristics, it realizes intelligent perception of the data transmission process, providing a scientific basis for subsequent optimization decisions.

[0009] This invention employs a K-means clustering algorithm combined with a genetic algorithm to optimize initial centroids, accurately grouping network transmission nodes with similar traffic fluctuation cycles into the same group. This traffic-based grouping method ensures that nodes within a group have similar transmission load characteristics, avoiding the blindness of traditional "one-size-fits-all" resource allocation. After grouping, differentiated resource strategies are formulated for different node groups, making resource scheduling more targeted and significantly improving the overall network resource utilization efficiency. Simultaneously, considering node hardware performance parameters (processor clock speed, memory capacity, network interface speed, etc.) and traffic fluctuation cycle stability, a transmission priority weight is generated through normalization processing and a weighted scoring mechanism. This weight dynamically reflects the transmission capacity and stability requirements of nodes, enabling the resource allocation module to tilt towards high-priority nodes when allocating bandwidth. This ensures that critical business nodes receive sufficient resource support during peak traffic periods while also considering the basic transmission needs of low-priority nodes, achieving the resource allocation goal of "efficiency first, while also considering fairness."

[0010] This invention employs a proportional fair allocation algorithm and introduces a real-time load monitoring and dynamic adjustment mechanism. When network load fluctuates, the system can quickly recalculate node priority weights and adjust the bandwidth allocation scheme, effectively addressing dynamic scenarios such as sudden traffic surges and node failures. Compared to existing static allocation methods, the dynamic adjustment mechanism of this invention enables the system to maintain efficient operation in complex network environments, effectively improving bandwidth utilization to a certain extent and significantly reducing transmission congestion caused by insufficient resources. Based on deep reinforcement learning, a multi-objective optimization model is constructed, with latency, packet loss rate, and bandwidth utilization as the core optimization objectives. By simulating different network scenarios to train scheduling strategies, the transmission path and order of data streams are dynamically adjusted according to real-time network conditions, maximizing bandwidth resource utilization while reducing transmission latency and packet loss rate. Compared to existing single-objective optimization methods, this invention achieves a comprehensive improvement in multi-dimensional performance indicators, meeting the modern data transmission requirements of "high efficiency, stability, and low power consumption." Attached Figure Description

[0011] The invention will now be further described with reference to the accompanying drawings.

[0012] Figure 1This is a schematic diagram of the connections between the modules of the system of the present invention.

[0013] Figure 2 This is a schematic diagram of the process for generating and processing the flow characteristic curve of the present invention.

[0014] Figure 3 This is a flowchart of the implementation steps of the method of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 1 As shown, an artificial intelligence-based data intelligence optimization system includes: a feature curve acquisition module, a period determination module, a node grouping module, a priority calculation module, a resource allocation module, and a data scheduling module. The feature curve acquisition module is connected to the period determination module, the period determination module is connected to the node grouping module, the node grouping module is connected to the priority calculation module, the priority calculation module is connected to the resource allocation module, and the resource allocation module is connected to the data scheduling module.

[0017] The characteristic curve acquisition module is used to acquire the traffic characteristic curves of each data stream in the network transmission node. The traffic characteristic curves are used to characterize the traffic change trend of the data stream in the network transmission node at different time points. Specifically, obtaining the traffic characteristic curves of each data stream in the network transmission node includes: 101: Initialization: Determine the fixed time interval for the sliding window algorithm, i.e., the sampling period. This time interval needs to be set according to the actual traffic change frequency of the network transmission nodes and the data collection requirements. For example, for high-speed networks and scenarios with frequent traffic changes, the sampling period can be set to a shorter time, such as 100 milliseconds; while for low-speed networks or scenarios with relatively slow traffic changes, the sampling period can be appropriately extended, such as 1 second.

[0018] Set a fixed length for the buffer. The buffer length determines the number of data samples taken each time, and it, along with the sampling period, affects the accuracy and real-time performance of the traffic characteristic curve. A longer buffer increases computational complexity and memory usage, but it reflects traffic trends more accurately; a shorter buffer may lose some traffic information. For example, a suitable buffer length can be estimated based on the average traffic volume of network nodes and the sampling period. Assuming an average traffic volume of 100 packets per second and a sampling period of 100 milliseconds, the buffer length can be initially set to 10 packets per second.

[0019] Configure the parameters of the Gaussian filter, including the filter window size and standard deviation. The filter window size determines the number of data points involved in the smoothing process, while the standard deviation controls the degree of smoothing. Generally, a larger window and a larger standard deviation result in a more pronounced smoothing effect, but may lead to over-smoothing and loss of some flow details. For example, you can initially set the filter window size to 5 data points and the standard deviation to 1 based on experience, and then adjust it according to the actual effect.

[0020] 102: Traffic Data Sampling: A timer is started at a fixed time interval. Each time the timer reaches a sampling period, a data sampling operation is triggered. When the timer is triggered, traffic data for the current time period is collected from the network transmission node. The collected data may include traffic-related indicators such as the number of data packets and the number of bytes. For example, the number of data packets and the number of bytes passing through the data stream within the sampling period can be obtained by reading the counter of the network device. The collected traffic data is stored in a buffer of a set fixed length. When the buffer is not full, data collection and storage continue. When the buffer is full, the next step of traffic value calculation and smoothing processing begins.

[0021] 103: Traffic Calculation and Smoothing: When the buffer is full, calculate the average value of all data in the buffer. For example, if the buffer stores the number of data packets, add up the number of data packets for all time periods in the buffer, divide by the buffer length, and obtain the average number of data packets at the current time point, which is used as the traffic value at that time point; A Gaussian filter is used to smooth the calculated flow rate values. The specific steps are as follows: A window containing a certain number of adjacent flow rate values ​​is defined, centered on the current calculated flow rate value. The window size is determined by the previously configured Gaussian filter parameters. For example, if the filter window size is 5, the window contains the current flow rate value and the two flow rate values ​​before and after it. The weight of each flow rate value within the window is calculated according to the Gaussian distribution formula, which is: ,in This represents the weight of the i-th data point. Indicates standard deviation, This represents the center position of the window (corresponding to the current flow value). After calculating the weight of each data point within the window, normalization is performed so that the sum of all weights is 1. Each flow value within the window is multiplied by its corresponding weight, and then all results are summed to obtain the smoothed flow value. The formula is: ,in This represents the smoothed flow rate value. This represents the i-th flow value within the window. This represents the corresponding weight, and n represents the flow of data points within the window.

[0022] 104: Flow Characteristic Curve Generation: After each calculation and smoothing of the flow value, record the current timestamp. The timestamp identifies the time point corresponding to the flow value, so that the flow trend over time can be accurately reflected later. Combine the recorded timestamp and the corresponding smoothed flow value to form a data point, for example (t, y), where t is the timestamp and y is the smoothed flow value. As time progresses, repeat the above process of sampling, calculation, smoothing, and recording data points, connecting the data points at each time point in chronological order to form a flow characteristic curve that reflects the trend of data flow over time.

[0023] It should be noted that, as Figure 2 As shown, the process begins with data acquisition, followed by sliding window sampling and smoothing, ultimately outputting a flow characteristic curve. This curve, with time on the horizontal axis and flow rate on the vertical axis, forms a trend graph that accurately represents the changing trend of the data flow. This method effectively eliminates data fluctuations caused by short-term noise interference, ensuring the accuracy of the flow characteristic curve.

[0024] The period determination module is used to determine the traffic fluctuation period of each network transmission node based on the traffic characteristic curves of each network transmission node. The traffic fluctuation period is used to characterize the regular time period of data flow changes in the network transmission node. Specifically, determining the traffic fluctuation period of each network transmission node includes: 201: Discretization of Flow Characteristic Curves: Time alignment is performed on the acquired flow characteristic curves. Since the flow characteristic curves of different data streams may have slight time deviations, it is necessary to unify their time starting points to a common time base for subsequent unified analysis.

[0025] Based on the set discretization interval, the time-aligned flow characteristic curve is sampled at equal intervals. Starting from a unified time point, flow values ​​on the flow characteristic curve are read sequentially according to the discretization interval, converting them into a series of equally spaced sampled numerical sequences. Simultaneously, the length of the flow characteristic curves for different data streams is standardized. For example, with a discretization interval of 1 second, starting from time t=0, one flow value is read every second. To obtain the numerical sequence It should be noted that since the lengths of the flow characteristic curves of different data streams may vary, it is necessary to standardize the length of the numerical sequences to facilitate subsequent FFT calculations. The longest length among all numerical sequences can be chosen as the standard length. For shorter numerical sequences, zeros are padded to the end to achieve a uniform length. For example, if there are three numerical sequences with lengths of 100, 120, and 150, the standard length is set to 150, and the sequences with lengths of 100 and 120 are padded with 50 and 30 zeros respectively.

[0026] 202: Fast Fourier Transform Calculation: The preprocessed numerical sequence is calculated using the selected Fast Fourier Transform (FFT) algorithm, transforming the numerical sequence from the time domain to the frequency domain. It should be noted that the FFT algorithm can be either a radix-2 FFT or a mixed-radix algorithm. The result of the FFT calculation is a complex sequence, where each complex number represents the amplitude and phase information of a frequency component. For example, for a numerical sequence of length N, the FFT calculation yields a complex sequence of length N. , where k=0,1,...,N−1.

[0027] Extract spectral information from the FFT calculation results, and calculate the amplitude spectrum corresponding to each frequency component. The formula for calculating the amplitude spectrum is as follows: ,in Represented as complex numbers The real and imaginary parts are used to obtain the amplitude spectrum, which is then plotted as a spectrum diagram. 203: Peak Frequency Extraction and Period Calculation: Find the peak frequency in the spectrum graph, set an amplitude threshold, traverse all frequency components in the spectrum graph, mark the frequency components whose amplitude exceeds the set amplitude threshold as candidate peak frequencies, compare each candidate peak frequency with the set frequency range interval, if a candidate peak frequency is within the set frequency range interval, then the candidate peak frequency will be retained, otherwise the candidate peak frequency will not be retained, thus obtaining the final peak frequency; If a peak frequency exists, according to the formula The corresponding cycle length is calculated and used as the flow fluctuation cycle, where f represents the peak frequency. If multiple peak frequencies exist, according to the formula The weighted average period was calculated. And take it as the cycle of traffic fluctuation, in which Represented as the z-th peak frequency, This is represented as the amplitude corresponding to the z-th peak frequency; This allows us to determine the traffic fluctuation period of each network transmission node.

[0028] The node grouping module is used to group network transmission nodes according to the traffic fluctuation cycle of each network transmission node, resulting in multiple transmission node groups. Specifically, grouping network transmission nodes includes: 301: Initialize K-means clustering parameters: Determine the number of K-means clusters k using the elbow rule, and initialize k cluster centers. The initial centroids are randomly selected (k) and the network transmission node traffic fluctuation period. It should be noted that the elbow method is a common method for determining the optimal number of clusters k in K-means clustering. By plotting the sum of squares (SSE) within clusters for different k values, the optimal k value is identified as the "elbow" where the curve drops sharply and then flattens out. 302: Genetic algorithm optimizes initial center point: 302-1: Encoding: Encode the k cluster centers; for example, a real number encoding method can be used, representing each center as a real number vector; 302-2: Initial population generation: Randomly generate a certain number of initial populations, where each individual represents a set of k possible cluster centers; 302-3: Fitness Function Design: A fitness function is designed to evaluate the performance of each individual. The sum of squared errors within groups (SSE) of K-means clustering is used as the fitness evaluation index. A smaller SSE indicates better clustering and higher fitness. The calculation formula is as follows: ; 302-4: Selection operation: The roulette wheel selection method is used to select individuals with high fitness from the current population as parents to generate the next generation of the population; 302-5: Crossover operation: Perform a crossover operation on the selected parent individuals to generate new individuals; for example, a single-point crossover method is used, randomly selecting a crossover point and exchanging some genes between two parent individuals; 302-6: Mutation Operation: Mutation operations are performed on newly generated individuals to increase population diversity. For example, a gene locus is randomly selected and slightly perturbed. 302-7: Iterative Optimization: Repeat the selection, crossover, and mutation operations until the preset number of iterations is reached. Then, terminate the genetic algorithm and select the individual with the highest fitness in the current population as the optimal initial center. .

[0029] 303: K-means clustering grouping execution: 303-1: Assigning Nodes to Clusters: For each network transmission node, calculate its traffic fluctuation period and the optimized k cluster centers using the Euclidean distance formula. Based on the distance, each node is assigned to the group containing the nearest cluster center according to the minimum distance principle, forming k initial transmission node groups; 303-2: Update cluster centroids: For each cluster, recalculate the average of the traffic fluctuation periods of all nodes in the cluster as the new cluster centroids; 303-3: Iterative Clustering: Repeat the execution steps of 301-1 and 301-2 until the cluster centers reach the preset number of iterations, completing the grouping of network transmission nodes and obtaining multiple transmission node groups. .

[0030] It should be noted that the K-means clustering algorithm is used, with the traffic fluctuation period as the clustering feature, to divide network transmission nodes with similar traffic fluctuation periods into the same transmission node group; a genetic algorithm is introduced to optimize the initial centroid of K-means clustering, and the optimal initial centroid is selected iteratively to improve the accuracy and stability of grouping.

[0031] The priority calculation module is used to calculate the transmission priority weight of each network transmission node for each transmission node group, taking into account the hardware performance parameters of each network transmission node in the transmission node group. Specifically, calculating the transmission priority weights of each network transmission node includes: Collect hardware performance parameters of network transmission nodes, including processor clock frequency f, memory capacity m, and network interface speed r; The collected hardware performance parameters are normalized to their maximum and minimum values, unifying the parameter values ​​to the [0,1] interval; the normalization formula is as follows: , of which Original parameter values, These are the minimum and maximum values ​​of the parameter, respectively; for example, for the processor clock speed f, assume its minimum value is... The maximum value is The normalized processor clock speed Similarly, normalizing the memory capacity m and network interface speed r yields... ; The overall performance score is calculated by weighted summation. The formula for calculating the overall performance score is as follows: ,in, These represent the percentage weights corresponding to the preset comprehensive performance scores; The standard deviation of the flow fluctuation period is calculated using the standard deviation calculation formula. Through formula The stability coefficient of the flow fluctuation period was calculated. It should be noted that the smaller the standard deviation of the traffic fluctuation period, the larger the stability coefficient of the traffic fluctuation period, indicating that the traffic fluctuation period is more stable. Combined with comprehensive performance score With stability coefficient Adjust to obtain transmission priority weights ,in These represent the percentage weights of the overall performance score and stability coefficient corresponding to the preset transmission priority weights, respectively.

[0032] The resource allocation module is used to allocate bandwidth resources to each network transmission node based on the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. Specifically, the allocation of bandwidth resources to each network transmission node includes: 501: Calculate the bandwidth allocation ratio: Use a proportional fair allocation algorithm to calculate the bandwidth allocation ratio for a single network transmission node. This ratio represents the transmission priority weight of that node. The sum of the priority weights of all nodes in the group The ratio, i.e. , where b represents the number of network transmission nodes in the transmission node group; 502: Calculate the initial bandwidth allocation value: based on the total bandwidth resources and allocation ratio Calculate the initial bandwidth allocation value ,Right now ; Iterate through all network transmission nodes, comparing the initial bandwidth allocation value with the set minimum bandwidth. If the initial bandwidth allocation value of a network transmission node is less than the set minimum bandwidth, then the network transmission node is determined to be a low-priority node. This process is then used to identify all low-priority nodes and calculate the bandwidth gap for each low-priority node. Here, k1 represents the number of each low-priority node, k1=1,2,...,l1; it should be noted that this gap represents the additional bandwidth that each node needs to allocate to meet basic transmission requirements; therefore, the total bandwidth gap is obtained by summing the bandwidth gaps of all low-priority nodes. ; Based on the low-priority nodes, the remaining nodes are marked as adjustable nodes. The unused bandwidth potential of all adjustable nodes is summed to obtain the total adjustable bandwidth pool. It should be noted that the unused bandwidth potential of all adjustable nodes refers to the bandwidth allocation value minus the minimum bandwidth required for basic transmission. Based on total bandwidth gap and adjustable bandwidth total pool According to the formula The adjustment ratio was calculated. ; For each adjustable node k2, k2 represents the node number, k2=1, 2, ..., l2; its bandwidth allocation value is reduced according to the adjustment ratio, i.e., the new bandwidth allocation value. ,in It allows adjustment of the node's original initial bandwidth allocation value. It is the minimum bandwidth that can be adjusted to meet basic transmission requirements; It should be noted that, through the above steps, while ensuring that low-priority nodes meet basic transmission requirements, the total bandwidth resources are not over-allocated.

[0033] 503: Dynamically adjust bandwidth allocation: Monitor network load changes in real time and set preset thresholds for load fluctuations. When load fluctuations exceed a preset threshold, i.e., the difference between the current network load and the network load at the time of the last monitoring is greater than a certain threshold. Recalculate the transmission priority weights; Based on the recalculated transmission priority weights, the bandwidth allocation ratio and initial bandwidth allocation value are recalculated according to the execution steps 501-502, and the bandwidth allocation resources of each network transmission node are dynamically adjusted.

[0034] The data scheduling module is used to dynamically schedule and transmit data streams based on the bandwidth resource allocation results of each network transmission node.

[0035] Specifically, the dynamic scheduling and transmission of the data stream includes: 601: Constructing a multi-objective optimization model: A multi-objective optimization model is constructed using the deep reinforcement learning algorithm DQN, with latency, packet loss rate, and bandwidth utilization as optimization objectives; 602: Define the state space: Define the state space as real-time network metrics, including bandwidth usage of each node, data stream transmission status, and network latency, and combine these metrics into a state vector. 603: Define Action Space: Define the action space as a data stream scheduling strategy, including transmission path selection and transmission order adjustment; It should be noted that for transmission path selection, all possible transmission paths can be pre-generated. By analyzing the network topology, path search algorithms in graph theory can be used to generate all feasible paths from the source node to the destination node. At the same time, a unique action number is assigned to each transmission path, and the transmission path selection action is represented as a discrete action space. For transmission order adjustment, different transmission order rules are defined based on factors such as the priority, size, and urgency of the data flow. At the same time, an action number is assigned to each transmission order rule, and the transmission order adjustment action is also included in the action space.

[0036] 604: Design a reward function: Quantify the scheduling effect through a reward function. The reward value is positively correlated with the reduction in latency, the reduction in packet loss rate, and the improvement in bandwidth utilization. Let R be the reward value. To delay the reduction amount, To reduce packet loss rate, If the bandwidth utilization improvement is expressed as: ,in These are the weighting coefficients for latency, packet loss rate, and bandwidth utilization, respectively. .

[0037] 605: Model Training: The model is trained by simulating different network environment scenarios. In each scenario, the model perceives various indicators in the state space based on the current network state and forms the current state vector. ; Based on the current state vector The model uses the DQN algorithm to select an action. The DQN algorithm obtains the Q-value of each possible action by querying the Q-network and selects the action with the largest Q-value as the action to be executed. It should be noted that this action can be a transmission path selection or a transmission order adjustment. Perform the selected action It schedules and transmits data streams. It should be noted that if the selected action is transmission path selection, the data stream will be transmitted according to the selected path; if the selected action is transmission order adjustment, the data stream will be sorted and transmitted according to the new transmission order. After an action is performed, the network state will be sent to transition, and the model will perceive the next state vector. And calculate the reward value based on the scheduling effect. , Combine the current state, action, reward, and next state into an experience tuple. The data is stored in the experience replay buffer. During training, a batch of experience tuples is randomly sampled from the experience replay buffer to update the parameters of the Q network. Through continuous iteration of this process, the model gradually learns the strategy of taking the optimal action in different states, thereby achieving efficient scheduling of the data flow.

[0038] 606: Dynamic Scheduling and Transmission: After sufficient training, the model has the ability to dynamically schedule data streams based on network conditions. In a real network environment, the model perceives the network condition in real time, selects the optimal action based on the current state vector, and dynamically schedules and transmits the data stream. At the same time, the model continuously monitors changes in the network condition. When the network condition changes significantly (such as a sudden increase or decrease in network load, or a link failure), the model re-performs the steps of state perception, action selection, and execution to achieve dynamic and efficient transmission of the data stream.

[0039] It should be noted that the data scheduling module can dynamically schedule and transmit data streams based on the actual network status and bandwidth resource allocation results, thereby improving network transmission efficiency and performance.

[0040] Please see Figure 3 As shown, an artificial intelligence-based data intelligence optimization method includes the following steps: Step 1: Obtain the traffic characteristic curves of each data stream in the network transmission node. The traffic characteristic curves are used to characterize the traffic change trend of the data stream in the network transmission node at different time points. Step 2: Based on the traffic characteristic curves of each network transmission node, determine the traffic fluctuation period of each network transmission node. The traffic fluctuation period is used to characterize the regular time period of data flow changes in the network transmission node. Step 3: Based on the traffic fluctuation cycle of each network transmission node, group the network transmission nodes to obtain multiple transmission node groups; Step 4: For each transmission node group, calculate the transmission priority weight of each network transmission node based on the hardware performance parameters of each network transmission node in the transmission node group. Step 5: Allocate bandwidth resources to each network transmission node according to the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. Step 6: Dynamically schedule and transmit the data stream based on the bandwidth resource allocation results of each network transmission node.

[0041] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A data intelligence optimization system based on artificial intelligence, characterized in that, include: The characteristic curve acquisition module is used to acquire the traffic characteristic curves of each data stream in the network transmission node. The traffic characteristic curves are used to characterize the traffic change trend of the data stream in the network transmission node at different time points. The period determination module is used to determine the traffic fluctuation period of each network transmission node based on the traffic characteristic curves of each network transmission node. The traffic fluctuation period is used to characterize the regular time period of data flow changes in the network transmission node. The node grouping module is used to group network transmission nodes according to the traffic fluctuation cycle of each network transmission node, resulting in multiple transmission node groups. The priority calculation module is used to calculate the transmission priority weight of each network transmission node for each transmission node group, taking into account the hardware performance parameters of each network transmission node in the transmission node group. The resource allocation module is used to allocate bandwidth resources to each network transmission node based on the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. The data scheduling module is used to dynamically schedule and transmit data streams based on the bandwidth resource allocation results of each network transmission node.

2. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The acquisition of traffic characteristic curves for each data stream in the network transmission node includes: 101: Initialization: Determine the fixed time interval of the sliding window algorithm, i.e., the sampling period; set the fixed length of the buffer; configure the parameters of the Gaussian filter, including the filter window size and standard deviation; 102: Traffic Data Sampling: A timer is started according to a set fixed time interval. Every time the timer reaches a sampling period, a data sampling operation is triggered. When the timer is triggered, traffic data for the current time period is collected from the network transmission node and stored in a buffer of a set fixed length. When the buffer is not full, data collection and storage continue. When the buffer is full, the next step of traffic value calculation and smoothing processing is performed. 103: Traffic value calculation and smoothing: When the buffer is full, calculate the average value of all data in the buffer, and use a Gaussian filter to smooth the calculated traffic value; 104: Flow characteristic curve generation: After each calculation and smoothing process to obtain the flow value, the current timestamp is recorded. The recorded timestamp and the corresponding smoothed flow value are combined to form a data point. As time goes by, the above process of sampling, calculation, smoothing and recording data points is repeated continuously. The data points at each time point are connected in chronological order to form a flow characteristic curve that can reflect the trend of data flow change over time.

3. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The determination of the traffic fluctuation period of each network transmission node includes: 201: Flow characteristic curve discretization processing: Perform time alignment operation on the acquired flow characteristic curve. According to the set discretization interval, sample the time-aligned flow characteristic curve at equal intervals. Starting from a unified time starting point, read the flow value on the flow characteristic curve in sequence according to the discretization interval, and convert it into a series of equally spaced sampled numerical sequences. At the same time, unify the length of the flow characteristic curve of different data streams. 202: Fast Fourier Transform Calculation: The selected Fast Fourier Transform (FFT) algorithm is used to calculate the preprocessed numerical sequence, transforming the numerical sequence from the time domain to the frequency domain; the spectral information is extracted from the FFT calculation results, the amplitude spectrum corresponding to each frequency component is calculated, and the amplitude spectrum is then plotted as a spectrum diagram. 203: Peak Frequency Extraction and Period Calculation: Find the peak frequency in the spectrum graph, set an amplitude threshold, traverse all frequency components in the spectrum graph, mark the frequency components whose amplitude exceeds the set amplitude threshold as candidate peak frequencies, compare each candidate peak frequency with the set frequency range interval, if a candidate peak frequency is within the set frequency range interval, then the candidate peak frequency will be retained, otherwise the candidate peak frequency will not be retained, thus obtaining the final peak frequency; If a peak frequency exists, according to the formula The corresponding period length is calculated. And this is taken as the traffic fluctuation period, where f represents the peak frequency; If multiple peak frequencies exist, according to the formula The weighted average period was calculated. And take it as the cycle of traffic fluctuation, in which Represented as the z-th peak frequency, This is represented as the amplitude corresponding to the z-th peak frequency; This allows us to determine the traffic fluctuation period of each network transmission node.

4. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The grouping of network transmission nodes includes: 301: Initialize K-means clustering parameters: Determine the number of K-means clusters k using the elbow rule, and initialize k cluster centers. The initial center point is selected by randomly choosing k and the traffic fluctuation period of the network transmission node. 302: Genetic algorithm optimizes the initial center point, including: 302-1: Encoding: Encode the k cluster centers; 302-2: Initial population generation: Randomly generate a certain number of initial populations, where each individual represents a set of k possible cluster centers; 302-3: Fitness Function Design: Design a fitness function to evaluate the quality of each individual; 302-4: Selection operation: The roulette wheel selection method is used to select individuals with high fitness from the current population as parents to generate the next generation of the population; 302-5: Crossover operation: Performs a crossover operation on the selected parent individual to generate a new individual; 302-6: Mutation operation: Perform mutation operation on newly generated individuals to increase the diversity of the population; 302-7: Iterative Optimization: Repeat the selection, crossover, and mutation operations until the preset number of iterations is reached. Then, terminate the genetic algorithm and select the individual with the highest fitness in the current population as the optimal initial center. ; 303: K-means clustering grouping execution: 303-1: Assigning Nodes to Clusters: For each network transmission node, calculate its traffic fluctuation period and the optimized k cluster centers using the Euclidean distance formula. Based on the distance, each node is assigned to the group containing the nearest cluster center according to the minimum distance principle, forming k initial transmission node groups; 303-2: Update cluster centroids: For each cluster, recalculate the average of the traffic fluctuation periods of all nodes in the cluster as the new cluster centroids; 303-3: Iterative Clustering: Repeat the execution steps of 301-1 and 301-2 until the cluster centers reach the preset number of iterations, completing the grouping of network transmission nodes and obtaining multiple transmission node groups. .

5. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The calculation of the transmission priority weights for each network transmission node includes: Collect hardware performance parameters of network transmission nodes, including processor clock speed, memory capacity, and network interface speed; The hardware performance parameters are normalized to their maximum and minimum values, unifying the parameter values ​​to the [0,1] range; The overall performance score is calculated by weighted summation, and the stability coefficient of the flow fluctuation period is calculated at the same time. The stability coefficient is the reciprocal of the standard deviation of the flow fluctuation period. The transmission priority weight is obtained by combining the comprehensive performance score and the stability coefficient.

6. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The allocation of bandwidth resources to each network transmission node includes: 501: Calculate the bandwidth allocation ratio: Use the proportional fair allocation algorithm to calculate the bandwidth allocation ratio of a single network transmission node; 502: Calculate the initial bandwidth allocation value: Calculate the initial bandwidth allocation value based on the total bandwidth resources and allocation ratio; traverse all network transmission nodes, compare the initial bandwidth allocation value with the set minimum bandwidth, and if the initial bandwidth allocation value of a network transmission node is less than the set minimum bandwidth, then the network transmission node is determined to be a low-priority node. In this way, the low-priority nodes are counted and the bandwidth gap of each low-priority node is calculated. Then, the bandwidth gaps of all low-priority nodes are accumulated to obtain the total bandwidth gap. Based on each low-priority node, the remaining nodes are marked as adjustable nodes, and the unused bandwidth potential of all adjustable nodes is added together to obtain the total adjustable bandwidth pool. The adjustment ratio is calculated using the formula based on the total bandwidth gap and the total pool of adjustable bandwidth. For each adjustable node, its bandwidth allocation value is reduced according to the adjustment ratio to obtain a new bandwidth allocation value; 503: Dynamically adjust bandwidth allocation: Monitor network load changes in real time, set a preset threshold for load fluctuations, and recalculate transmission priority weights when load fluctuations exceed the preset threshold. Based on the recalculated transmission priority weights, the bandwidth allocation ratio and initial bandwidth allocation value are recalculated according to the execution steps 501-502, and the bandwidth allocation resources of each network transmission node are dynamically adjusted.

7. The data intelligence optimization system based on artificial intelligence according to claim 1, characterized in that, The dynamic scheduling and transmission of the data stream includes: 601: Constructing a multi-objective optimization model: A multi-objective optimization model is constructed using the deep reinforcement learning algorithm DQN, with latency, packet loss rate, and bandwidth utilization as optimization objectives; 602: Define the state space: Define the state space as real-time network metrics, including bandwidth usage of each node, data stream transmission status, and network latency, and combine these metrics into a state vector. 603: Define Action Space: Define the action space as a data stream scheduling strategy, including transmission path selection and transmission order adjustment; 604: Design a reward function: Quantify the scheduling effect through a reward function. The reward value is positively correlated with the amount of latency reduction, packet loss rate reduction, and bandwidth utilization improvement. 605: Model Training: The model is trained by simulating different network environment scenarios. In each scenario, the model perceives various indicators in the state space based on the current network state and forms the current state vector. ; Based on the current state vector The model uses the DQN algorithm to select an action. The DQN algorithm obtains the Q-value of each possible action by querying the Q-network and selects the action with the largest Q-value as the action to be executed. Perform the selected action This involves scheduling and transmitting data streams. After an action is performed, the network state will be sent to transition, and the model will perceive the next state vector. And calculate the reward value based on the scheduling effect. , Combine the current state, action, reward, and next state into an experience tuple. The data is stored in the experience replay buffer. During training, a batch of experience tuples is randomly sampled from the experience replay buffer to update the parameters of the Q network. Through continuous iteration of this process, the model gradually learns the strategy of taking the optimal action in different states. 606: Dynamic Scheduling and Transmission: After sufficient training, the model perceives the network state in real time, selects the optimal action based on the current state vector, and dynamically schedules and transmits the data stream.

8. A data intelligence optimization method based on artificial intelligence, characterized in that, Includes the following steps: Step 1: Obtain the traffic characteristic curves of each data stream in the network transmission node; Step 2: Determine the traffic fluctuation period of each network transmission node based on the traffic characteristic curves of each network transmission node. Step 3: Based on the traffic fluctuation cycle of each network transmission node, group the network transmission nodes to obtain multiple transmission node groups; Step 4: For each transmission node group, calculate the transmission priority weight of each network transmission node based on the hardware performance parameters of each network transmission node in the transmission node group. Step 5: Allocate bandwidth resources to each network transmission node according to the transmission priority weight of each network transmission node and the number of network transmission nodes in the transmission node group. Step 6: Dynamically schedule and transmit the data stream based on the bandwidth resource allocation results of each network transmission node.

Images