Data transmission method and device and electronic equipment

Abstract

Embodiments of the invention disclose a data transmission method and apparatus, and an electronic device. The method comprises the steps of receiving a data transmission instruction for target data; in response to the data transmission instruction, determining a target network corresponding to the target data, and obtaining data demand information of the target data, current network topology information of the target network and link state information; determining a routing strategy corresponding to the target data based on the data demand information of the target data, the current network topology information of the target network and the link state information by using a pre-trained prediction model; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm through a Q-learning algorithm based on historical sample data; and transmitting the target data through the target network according to the routing policy corresponding to the target data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data transmission method, device and electronic equipment. Background Art

[0002] With the rapid development of network technology, how to improve data transmission efficiency and enhance user satisfaction has become the focus of developers.

[0003] During the route calculation and selection phase, routing decisions can be made based on the shortest path or minimum cost principle. However, as more and more factors affect data transmission efficiency, the shortest path or minimum cost principle suffers from slow algorithm response and cannot meet real-time requirements. Therefore, a technical solution is needed to improve data transmission efficiency and enhance user satisfaction. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a technical solution for improving data transmission efficiency and thus enhancing user satisfaction.

[0005] To solve the above technical problems, the embodiments of the present invention are implemented as follows: In a first aspect, an embodiment of the present invention provides a data transmission method, the method comprising: receiving a data transmission instruction for target data; In response to the data transmission instruction, determining a target network corresponding to the target data, and obtaining data requirement information of the target data, current network topology information of the target network, and link status information; Determine the routing strategy corresponding to the target data using a pre-trained prediction model based on the data demand information of the target data, the current network topology information of the target network, and the link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The target data is transmitted through the target network according to a routing policy corresponding to the target data.

[0006] In a second aspect, an embodiment of the present invention provides a data transmission device, the device comprising: An instruction receiving module, used for receiving a data transmission instruction for target data; an information acquisition module, configured to determine, in response to the data transmission instruction, a target network corresponding to the target data, and acquire data requirement information of the target data, current network topology information of the target network, and link status information; a strategy determination module, configured to determine a routing strategy corresponding to the target data using a pre-trained prediction model based on data demand information of the target data, current network topology information of the target network, and link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The data transmission module is used to transmit the target data through the target network according to the routing policy corresponding to the target data.

[0007] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data transmission method provided in the above embodiment.

[0008] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data transmission method provided in the above embodiment are implemented.

[0009] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the data transmission method provided in the above embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic flow chart of a data transmission method according to the present invention; Figure 2 A schematic diagram of a model training process of the present invention; Figure 3 A schematic diagram of a model training process of the present invention; Figure 4 A schematic diagram of a process for determining a routing strategy of the present invention; Figure 5 A schematic diagram of a data transmission process of the present invention; Figure 6 A schematic diagram of another model training process of the present invention; Figure 7 A schematic diagram of a process for determining network topology information according to the present invention; Figure 8 A schematic diagram of a link status determination process of the present invention; Figure 9 A schematic diagram of a routing decision determination process of the present invention; Figure 10 This is a structural diagram of a data transmission device of the present invention; Figure 11 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0012] Embodiments of the present invention provide a data transmission method, device, and electronic device.

[0013] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0014] The embodiments of this specification provide data transmission methods, devices and equipment. With the rapid development of network technology, how to improve data transmission efficiency to improve user satisfaction has become the focus of developers. In the route calculation and selection stage, routing decisions can be made based on the shortest path or minimum cost principle. However, when there are more and more factors affecting data transmission efficiency, the shortest path or minimum cost principle has the problem of slow algorithm response speed and cannot meet real-time requirements. Therefore, a technical solution is needed to improve data transmission efficiency and improve user satisfaction. In this solution, a data transmission instruction for target data is received, and in response to the data transmission instruction, a target network corresponding to the target data is determined, and data demand information of the target data, current network topology information of the target network and link status information are obtained. A pre-trained prediction model is used to determine the routing strategy corresponding to the target data based on the data demand information of the target data, current network topology information of the target network and link status information. The prediction model can be obtained by training a model constructed by a deep learning algorithm based on historical sample data using a Q-learning algorithm. According to the routing strategy corresponding to the target data, the target data is transmitted through the target network. In this way, on the one hand, based on the data demand information of the target data, the current network topology information of the target network, and the link status information, a routing strategy corresponding to the target data can be determined by comprehensively considering the specific needs of different data and the network status of the target network. On the other hand, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data demand from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction. For specific processing, please refer to the specific content of the following embodiment.

[0015] like Figure 1 As shown, an embodiment of the present invention provides a data transmission method. The execution subject of the method can be a terminal device or a server. The terminal device can be a mobile terminal device such as a mobile phone, tablet computer, smart watch, or a terminal device such as a computer. The server can be an independent server or a server cluster composed of multiple servers. The method can specifically include the following steps: In step S102 , a data transmission instruction for target data is received.

[0016] Among them, the target data can be any data to be transmitted, for example, the target data can be voice data, audio data, video data, image data, web page data, etc. For example, user 1 can make a voice call with user 2 through the terminal device, and the voice communication data between user 1 and user 2 can be the target data. Alternatively, user 1 can also trigger the execution of the video viewing service through the terminal device, and the server can send the video data selected by user 1 to the terminal device corresponding to user 1, and the video data to be transmitted can be the target data.

[0017] In step S104, in response to the data transmission instruction, a target network corresponding to the target data is determined, and data requirement information of the target data, current network topology information of the target network, and link status information are obtained.

[0018] Among them, data demand information can be used to characterize the data transmission requirements of the target data. For example, the data demand information of the target data may include the priority and data volume of the target data. The network topology information can be used to characterize the connection relationship between the nodes in the target network. The link status information can be used to characterize the transmission status such as the transmission performance of each link in the target network.

[0019] During implementation, the server can collect the current network topology information and link status information of the target network. For example, the server can obtain information such as the connection relationship between each node in the target network, link quality (such as bandwidth, delay, packet loss rate, etc.), etc.

[0020] In step S106, a pre-trained prediction model is used to determine a routing strategy corresponding to the target data based on data demand information of the target data, current network topology information of the target network, and link status information.

[0021] The prediction model can be trained using historical sample data using the Q-learning algorithm, a reinforcement learning algorithm used to solve Markov decision process (MDP) problems. The core idea of ​​the Q-learning algorithm is to evaluate the expected utility of taking an action in a given state by learning an action-value function (Q function).

[0022] In implementation, taking the deep learning algorithm as a deep neural network algorithm as an example, the server can construct a prediction model through the deep neural network algorithm, and train the constructed prediction model based on historical sample data and the Q-learning algorithm to obtain a trained prediction model.

[0023] The server can then input the data demand information of the target data, the current network topology information of the target network, and the link status information into the trained prediction model to determine the routing strategy corresponding to the target data based on the output results of the prediction model.

[0024] In addition, there can be multiple target data, and the server can input the data demand information corresponding to each target data into a pre-trained prediction model to determine the routing strategy corresponding to each target data based on the current network topology information and link status information of the target network through the pre-trained prediction model.

[0025] In step S108, the target data is transmitted through the target network according to the routing policy corresponding to the target data.

[0026] In implementation, the server may transmit each target data through the target network according to a routing policy corresponding to each target data.

[0027] In this way, intelligent routing algorithms based on reinforcement learning can better consider factors such as network congestion and link quality, improving the real-time and responsiveness of routing selection. The prediction model can learn the complex relationship between network status and data demand from a large amount of historical sample data and dynamically adjust based on real-time conditions. As a result, this prediction model can provide more accurate and real-time routing selection, further optimizing the efficiency and performance of network data transmission.

[0028] An embodiment of the present invention provides a data transmission method that receives a data transmission instruction for target data, determines a target network corresponding to the target data in response to the data transmission instruction, obtains data requirement information for the target data, current network topology information, and link status information of the target network, and uses a pre-trained prediction model to determine a routing strategy corresponding to the target data based on the data requirement information, current network topology information, and link status information of the target network. The prediction model can be obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data. The target data is transmitted through the target network according to the routing strategy corresponding to the target data. In this way, based on the data requirement information of the target data, the current network topology information, and link status information of the target network, a routing strategy corresponding to the target data can be determined by comprehensively considering the specific requirements of different data and the network status of the target network. Furthermore, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data requirements from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction.

[0029] In practical applications, before using the pre-trained prediction model to determine the routing strategy corresponding to the target data, the prediction model can also be trained. There are many ways to train the prediction model. The following is an optional processing method, such as Figure 2 As shown, the process may specifically include the following steps S202 to S208.

[0030] In step S202, historical sample data is obtained.

[0031] Among them, the historical sample data includes historical data demand information, historical network topology information of the target network, historical link status information, and historical routing strategies.

[0032] In step S204, the prediction model is used to determine a reward value corresponding to the first predicted routing action according to historical data demand information, historical network topology information, and historical link status information.

[0033] In implementation, before model training, the server can determine the model structure and training process of the prediction model, such as Figure 3 As shown, the following steps may be specifically included: 1. Define the problem: Define the problem and clarify the goal. In reinforcement learning, the goal is usually to maximize the cumulative reward, and the problem is how to choose the best action to achieve this goal; 2. State representation: Consider state representation. In routing calculations, the state can include information such as network topology, link status, and data requirements. The server can encode this information into a vector as input to the prediction model. 3. Action space: Define the action space. In routing, the action space can represent which node to choose for data transmission or which path to choose for data transmission. The action space can be represented as a set of nodes or paths. 4. Reward function: Define a reward function. The reward function can be used to measure the quality of each action to help improve the strategy. In intelligent routing, the reward function can be designed based on factors such as data transmission rate, latency, packet loss rate, and network congestion. 5. Deep neural network structure: Design the structure of a deep neural network. A neural network can consist of multiple layers, such as an input layer, a hidden layer, and an output layer. A deep neural network can be used to predict the reward value of each action to help select the optimal action. 6. Training model: You can use reinforcement learning algorithms to train the model. During the training process, the prediction model can select an action based on the current state and update it based on the rewards obtained. You can use the Q-learning algorithm for training, and use techniques such as experience replay to improve training efficiency. 7. Model evaluation: To evaluate the prediction model, you can use the test data set to evaluate the performance of the prediction model and calculate the model's accuracy, recall rate, F1-score and other indicators.

[0034] In step S206 , according to the update rule of the Q-learning algorithm, the target Q value is calculated based on the reward value corresponding to the predicted routing action.

[0035] In practice, training via the Q-learning algorithm may include the following steps: 1. Initialization a. State space: defines all possible network states, including network topology information, link status, and data requirements.

[0036] b. Action space: defines all possible actions, such as choosing a specific routing path.

[0037] c. Q-table: Initialize the Q-table to store the Q value of each state-action pair.

[0038] d. Parameter setting: Set hyperparameters such as learning rate α, discount factor γ, and exploration rate ε.

[0039] 2. Experience Replay Buffer Initialize the buffer: Create an experience replay buffer to store the experienced state, action, reward, and new state (such as a four-tuple (s, a, r, s'), where s is the current state, a is the action corresponding to the current state, r is the reward corresponding to the current state, and s' is the new state (i.e., the next state of the current state)).

[0040] 3. Main Loop For each time step t: a. Select an action Exploration and Exploitation: Select an action based on the current exploration rate ε. If the random number is less than ε, then a random action is selected; otherwise, the action with the highest Q value in the current state is selected.

[0041] b. Perform actions and observe results Execute Action: Execute the selected action in the simulation environment.

[0042] Receive feedback: record whether the immediate reward obtained by the new state reaches the final state.

[0043] c. Storage experience Add to Buffer: Add the experienced state, action, reward, and new state to the experience replay buffer.

[0044] d. Sample extraction and learning Extract samples: Randomly extract a certain number of samples (such as four-tuples (s, a, r, s')) from the experience replay buffer.

[0045] Calculate the target Q value: Use the Q-learning update rule to calculate the target Q value: Q(s,a)←Q(s,a)+α[r+γmaxα`Q(s`,a`)-Q(s,a)]Q(s,a)←Q(s,a)+α[r+γmaxαQ(s`,a`)-Q(s,a)] Update Q-table: Update the corresponding state-action pair in the Q-table according to the calculated target Q value.

[0046] e. Adjust the exploration rate Decay ε: Gradually reduce the exploration rate ε over time to make more use of the learned knowledge.

[0047] In step S208, whether the prediction model has converged is determined based on the change range of the target Q value, and if the prediction model has converged, a trained prediction model is obtained.

[0048] In implementation, when the preset number of training rounds is reached, the convergence of the prediction model can be determined based on the convergence judgment condition, where the convergence judgment condition can be: if the target Q value changes by less than a preset amplitude threshold in multiple consecutive rounds of training, the prediction model can be considered to have converged, and the training can be stopped to obtain the trained prediction model.

[0049] In practical applications, in step S106, a pre-trained prediction model is used to determine the routing strategy corresponding to the target data based on the data demand information of the target data, the current network topology information of the target network, and the link status information. There are many ways to process the routing strategy. The following is an optional processing method, such as Figure 4 As shown, the process may specifically include the following steps S1062 to S1064.

[0050] In step S1062, a pre-trained prediction model is used to determine a reward value corresponding to the second predicted routing action based on data demand information of the target data, current network topology information of the target network, and link status information.

[0051] In step S1064 , a routing strategy corresponding to the target data is determined according to the reward value corresponding to the second predicted routing action.

[0052] In implementation, the server may input current network status information and link status, as well as target data demand information, into a trained prediction model.

[0053] The server can select the action with the highest reward value as the routing decision based on the information in the Q table output by the prediction model. The server can then update the routing table based on the selected routing decision to guide the forwarding of the target data packet.

[0054] In addition, if Figure 5 As shown, the specific steps of the server in performing route calculation and selection may include: 1. Action selection: The server can select the action with the highest reward value as the current routing decision based on the reward value predicted by the prediction model. It can use a greedy strategy to select the action with the highest reward value, or introduce randomness to increase exploration. 2. Route update: Based on the selected action, the corresponding routing information in the routing table or router is updated, including the next hop node or path, so that the target data packet can be transmitted according to the new route; 3. Data transmission: Based on the updated routing information, data transmission begins, and the target data packet is transmitted from the source node to the target node according to the new routing selection. During the transmission process, the network status may change, so it is necessary to periodically recalculate and update the route; 4. Monitoring and adjustment: During data transmission, it is necessary to monitor network status and performance indicators, such as latency and packet loss rate. If network performance is poor or a failure occurs, it may be necessary to recalculate the route or make dynamic adjustments to optimize the quality and efficiency of data transmission.

[0055] In addition, the prediction model can be optimized, such as Figure 6 As shown, the specific steps of the optimization process may include: 1. Policy evaluation: Use a trained prediction model to evaluate the current policy. Based on the current state, the prediction model will predict the reward value of an action. This can be compared with the actual reward to evaluate the performance of the current routing policy. 2. Reward value adjustment: Based on the results of the strategy evaluation, the reward value can be adjusted. If the current strategy performs well, the reward value of the corresponding action can be increased; conversely, if the current strategy performs poorly, the reward value of the corresponding action can be reduced. 3. Experience replay: To improve training efficiency, experience replay technology can be used to store previous routing decision experiences in an experience pool and randomly select a portion of samples from it for training. This can increase sample diversity and prevent the model from overfitting the current data. 4. Update model parameters: Using the collected data and adjusted reward values, you can update the parameters of the prediction model and use optimization algorithms such as gradient descent to minimize the model's loss function, so that the prediction model can better predict the reward value; 5. Iterative training: Repeat steps 2 to 5 for multiple rounds of training and optimization. Through continuous iteration and updating, the performance of the prediction model will gradually improve and the strategy will become more and more accurate.

[0056] In practical applications, the data requirement information of the target data may be type information obtained by classifying the target data according to the priority and data size of the target data.

[0057] In practical applications, there are various processing methods for obtaining the current network topology information of the target network in step S104. An optional processing method is provided below, such as Figure 7 As shown, the process may specifically include the following steps S1042 to S1046.

[0058] In step S1042, a preset controller is used to broadcast a preset data packet to each node in the target network.

[0059] The preset data packet may be a Link Layer Discovery Protocol (LLDP) data packet.

[0060] In implementation, the pre-set controller is an SDN controller, for example. The SDN controller can be an application in a software-defined network (SDN) that is responsible for traffic control to ensure an intelligent network. The SDN controller is based on protocols such as OpenFlow, which allows servers to tell switches where to send data packets.

[0061] The server can deploy an SDN controller, such as OpenDaylight or Floodlight, and ensure it can establish connections with network devices.

[0062] For example, before broadcasting a data packet, the server can determine whether the switches and routers in the target network support the OpenFlow protocol and are correctly configured. If the switches and routers in the target network support the OpenFlow protocol and are correctly configured, the server can communicate with each node in the target network through the SDN controller.

[0063] In addition, the server also needs to configure network devices, such as configuring correct flow table rules on each network device in the target network so that they can send information to the SDN controller.

[0064] In step S1044, feedback information from each node regarding the preset data packet is received.

[0065] In step S1046, the current network topology information of the target network is constructed according to the feedback information.

[0066] The network topology information may include connection relationships and port information between nodes.

[0067] In practice, the server can send LLDP packets through the SDN controller to discover link information in the target network. These packets are broadcast in the target network, and when the nodes in the target network receive these packets, they return the information to the SDN controller.

[0068] The SDN controller can construct the network topology information of the target network based on the returned LLDP information.

[0069] In actual applications, there are various ways to construct the current network topology information of the target network according to the feedback information in step S1046. An optional processing method is provided below, which may specifically include the following steps A1 to A3.

[0070] In step A1, first network topology information is constructed according to the feedback information.

[0071] In step A2, feedback information from different nodes is compared to obtain a comparison result, and integrity verification is performed on the first network topology information based on the comparison result.

[0072] In implementation, the network topology corresponding to each node can be determined based on the feedback information of the node, and the accuracy and completeness of the first network topology information can be verified by comparing the network topologies of different devices (i.e., nodes in the target network).

[0073] In step A3, if the first network topology information passes the integrity check, the first network topology information is determined as the current network topology information of the target network.

[0074] In practical applications, there are various processing methods for obtaining the current link status information of the target network in step S104. The link status information may include the bandwidth and delay of the link. Accordingly, an optional processing method is provided below, such as Figure 8 As shown, the processing may specifically include the following steps S1048 to S10410.

[0075] In step S1048, a preset controller is used to send a preset message to each link in the target network, and the current link delay of the target network is determined according to the round-trip time of the preset message.

[0076] In step S10410, traffic information of ports of each node in the target network is obtained from the switch, and the bandwidth of the current link of the target network is determined based on the traffic information.

[0077] In practice, the link latency between switches can be tested by sending preset messages through the SDN controller. These messages are sent to each device on the current link of the target network, and the round-trip time (RTT) is recorded.

[0078] The statistical messages of the OpenFlow protocol can be used to obtain port traffic information from the switch and analyze the obtained traffic information to calculate the available bandwidth of the link.

[0079] In addition, it can continuously monitor link status information, such as latency, bandwidth utilization and other key indicators, to promptly identify network performance issues or potential failure points.

[0080] Furthermore, by analyzing collected information such as network topology and link status, bottlenecks and redundant paths can be identified within the target network. Based on these analysis results, the configuration of each device in the target network can be adjusted to optimize network performance. For example, routing policies can be replanned and flow table rules adjusted. The optimized configuration can then be applied to the target network and its effectiveness continuously monitored to ensure network stability and efficiency.

[0081] like Figure 9 As shown, the above data transmission method may include the following steps: Step 1: Collect network topology information and link status: Obtain information such as the connection relationship and link quality between each node in the target network, including bandwidth, latency, packet loss rate, etc. Step 2: Data demand analysis: Classify target data into different categories or traffic types based on their priority, size, and other characteristics, so that different requirements can be considered during routing selection. Step 3: Build a reinforcement learning model: Use a deep neural network to build a reinforcement learning model with inputs such as network topology information, link status, and data requirements. Step 4: Strategy Evaluation: This involves simulating multiple data streams passing through the target network, continuously observing and recording information such as the current state, action, reward, and next state, and conducting strategy evaluation. Step 5: Optimize strategy: By learning and training the results of strategy evaluation, the reinforcement learning model is optimized so that it can make accurate routing decisions based on network status and data requirements. Step 6: Route calculation and selection: Select the optimal transmission path and node based on the prediction results of the reinforcement learning model; Step 7: Route update and adaptability: Update the reinforcement learning model in a timely manner according to real-time network conditions and data changes, and adapt to changes in the network environment to ensure routing accuracy and stability.

[0082] In this way, by establishing a reinforcement learning model (i.e., a prediction model), the shortcomings of traditional algorithms can be improved. The prediction model can use a deep neural network. The input of the prediction model can be information such as network topology information, link status, and data demand. By observing and recording the simulated data flow, the current state, action, reward, and next state information can be evaluated, thereby optimizing the routing strategy.

[0083] Embodiments of this specification provide a data transmission method that receives a data transmission instruction for target data, determines a target network corresponding to the target data in response to the data transmission instruction, obtains data requirement information for the target data, current network topology information of the target network, and link status information, and uses a pre-trained prediction model to determine a routing strategy corresponding to the target data based on the data requirement information, current network topology information, and link status information of the target data. The prediction model can be obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data. The target data is transmitted through the target network according to the routing strategy corresponding to the target data. In this way, on the one hand, based on the data requirement information of the target data, the current network topology information of the target network, and the link status information, a routing strategy corresponding to the target data can be determined while comprehensively considering the specific requirements of different data and the network status of the target network. On the other hand, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data requirements from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction.

[0084] The above is a data transmission method provided by the embodiment of this specification. Based on the same idea, the embodiment of this specification also provides a data transmission device, such as Figure 10 shown.

[0085] The data transmission device includes: an instruction receiving module 1001, an information acquisition module 1002, a strategy determination module 1003 and a data transmission module 1004, wherein: The instruction receiving module 1001 is used to receive a data transmission instruction for target data; An information acquisition module 1002 is configured to determine, in response to the data transmission instruction, a target network corresponding to the target data, and acquire data requirement information of the target data, current network topology information of the target network, and link status information; A strategy determination module 1003 is configured to determine a routing strategy corresponding to the target data using a pre-trained prediction model based on data demand information of the target data, current network topology information of the target network, and link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The data transmission module 1004 is configured to transmit the target data through the target network according to a routing policy corresponding to the target data.

[0086] In the embodiment of this specification, the device further includes: A data acquisition module, configured to acquire the historical sample data, wherein the historical sample data includes historical data demand information, historical network topology information of the target network, historical link status information, and historical routing strategies; a first prediction module, configured to determine a reward value corresponding to a first predicted routing action based on the historical data demand information, the historical network topology information, and the historical link state information using the prediction model; A data calculation module, configured to calculate a target Q value based on a reward value corresponding to the predicted routing action according to an update rule of a Q-learning algorithm; A model training module is used to determine whether the prediction model has converged according to the change range of the target Q value, and obtain a trained prediction model if the prediction model has converged; The strategy determination module 1003 is configured to: Determining a reward value corresponding to a second predicted routing action based on data demand information of the target data, current network topology information of the target network, and link status information using the pre-trained prediction model; A routing strategy corresponding to the target data is determined according to a reward value corresponding to the second predicted routing action.

[0087] In the embodiment of this specification, the data requirement information of the target data is type information obtained by classifying the target data according to the priority and data size of the target data.

[0088] In the embodiment of this specification, the information acquisition module 1002 is used to: Broadcasting a preset data packet to each node in the target network using a preset controller; receiving feedback information from each node regarding the preset data packet; The current network topology information of the target network is constructed according to the feedback information, where the network topology information includes the connection relationship and port information between the nodes.

[0089] In the embodiment of this specification, the information acquisition module 1002 is used to: Constructing first network topology information according to the feedback information; Comparing feedback information from different nodes to obtain a comparison result, and performing integrity verification on the first network topology information based on the comparison result; In a case where the first network topology information passes the integrity check, the first network topology information is determined as the current network topology information of the target network.

[0090] In the embodiment of this specification, the link status information includes the bandwidth and latency of the link. The information acquisition module 1002 is configured to: Using a preset controller to send a preset message to each link in the target network, and determining the current link delay of the target network based on the round-trip time of the preset message; The flow information of the port of each node in the target network is obtained from the switch, and the bandwidth of the current link of the target network is determined according to the flow information.

[0091] An embodiment of the present specification provides a data transmission device that receives a data transmission instruction for target data, responds to the data transmission instruction, determines a target network corresponding to the target data, obtains data demand information for the target data, current network topology information of the target network, and link status information, and uses a pre-trained prediction model to determine a routing strategy corresponding to the target data based on the data demand information, current network topology information, and link status information of the target data. The prediction model can be obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data. The target data is transmitted through the target network according to the routing strategy corresponding to the target data. In this way, on the one hand, based on the data demand information of the target data, the current network topology information of the target network, and the link status information, a routing strategy corresponding to the target data can be determined while comprehensively considering the specific requirements of different data and the network status of the target network. On the other hand, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data demand from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction.

[0092] The above is a data transmission device provided in the embodiment of this specification. Based on the same idea, the embodiment of this specification also provides a data transmission device, such as Figure 11 shown.

[0093] The data transmission device may provide a terminal device or a server, etc. for the above-mentioned embodiment.

[0094] Data transmission devices can vary significantly depending on their configuration or performance. They may include one or more processors 1101 and memory 1102. Memory 1102 may store one or more applications or data. Memory 1102 may be either ephemeral or persistent. Applications stored in memory 1102 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for the data transmission device. Furthermore, processor 1101 may be configured to communicate with memory 1102 to execute the series of computer-executable instructions in memory 1102 on the data transmission device. The data transmission device may also include one or more power supplies 1103, one or more wired or wireless network interfaces 1104, one or more input / output interfaces 1105, and one or more keyboards 1106.

[0095] Specifically, in this embodiment, the data transmission device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the data transmission device, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following: receiving a data transmission instruction for target data; In response to the data transmission instruction, determining a target network corresponding to the target data, and obtaining data requirement information of the target data, current network topology information of the target network, and link status information; Determine the routing strategy corresponding to the target data using a pre-trained prediction model based on the data demand information of the target data, the current network topology information of the target network, and the link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The target data is transmitted through the target network according to a routing policy corresponding to the target data.

[0096] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the data transmission device embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0097] Embodiments of this specification provide a data transmission device that receives a data transmission instruction for target data, responds to the data transmission instruction, determines a target network corresponding to the target data, obtains data demand information for the target data, current network topology information of the target network, and link status information, and uses a pre-trained prediction model to determine a routing strategy corresponding to the target data based on the data demand information, current network topology information, and link status information of the target data. The prediction model can be obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data. The target data is transmitted through the target network according to the routing strategy corresponding to the target data. In this way, on the one hand, based on the data demand information of the target data, the current network topology information of the target network, and the link status information, a routing strategy corresponding to the target data can be determined while comprehensively considering the specific requirements of different data and the network status of the target network. On the other hand, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data demand from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction.

[0098] Furthermore, based on the above Figures 1 to 9 In one or more embodiments of the present specification, a storage medium is provided for storing computer-executable instruction information. In a specific embodiment, the storage medium may be a USB flash drive, an optical disk, a hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, the following process can be implemented: receiving a data transmission instruction for target data; In response to the data transmission instruction, determining a target network corresponding to the target data, and obtaining data requirement information of the target data, current network topology information of the target network, and link status information; Determine the routing strategy corresponding to the target data using a pre-trained prediction model based on the data demand information of the target data, the current network topology information of the target network, and the link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The target data is transmitted through the target network according to a routing policy corresponding to the target data.

[0099] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the aforementioned storage medium embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0100] An embodiment of the present specification provides a storage medium that receives a data transmission instruction for target data, determines a target network corresponding to the target data in response to the data transmission instruction, obtains data demand information of the target data, current network topology information of the target network, and link status information, and uses a pre-trained prediction model to determine a routing strategy corresponding to the target data based on the data demand information of the target data, the current network topology information of the target network, and the link status information. The prediction model can be obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data. The target data is transmitted through the target network according to the routing strategy corresponding to the target data. In this way, on the one hand, based on the data demand information of the target data, the current network topology information of the target network, and the link status information, a routing strategy corresponding to the target data can be determined while comprehensively considering the specific requirements of different data and the network status of the target network. On the other hand, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data demand from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction.

[0101] Furthermore, based on the above Figures 1 to 9 In addition to the method shown in FIG. 1 , one or more embodiments of this specification further provide a computer program product, including a computer program. When the computer program in the computer program product is executed by a processor, the computer program can implement the following process: receiving a data transmission instruction for target data; In response to the data transmission instruction, determining a target network corresponding to the target data, and obtaining data requirement information of the target data, current network topology information of the target network, and link status information; Determine the routing strategy corresponding to the target data using a pre-trained prediction model based on the data demand information of the target data, the current network topology information of the target network, and the link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The target data is transmitted through the target network according to a routing policy corresponding to the target data.

[0102] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the aforementioned computer program product embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, reference can be made to the description of the method embodiment.

[0103] Embodiments of this specification provide a computer program product that receives a data transmission instruction for target data, determines a target network corresponding to the target data in response to the data transmission instruction, obtains data demand information for the target data, current network topology information of the target network, and link status information, and uses a pre-trained prediction model to determine a routing strategy corresponding to the target data based on the data demand information, current network topology information, and link status information of the target data. The prediction model can be obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data. The target data is transmitted through the target network according to the routing strategy corresponding to the target data. In this way, on the one hand, based on the data demand information of the target data, the current network topology information of the target network, and the link status information, a routing strategy corresponding to the target data can be determined while comprehensively considering the specific requirements of different data and the network status of the target network. On the other hand, the Q-learning algorithm has strong nonlinear fitting capabilities and can learn the complex relationship between network status and data demand from a large amount of historical sample data. Therefore, the prediction model trained by this algorithm can provide accurate and real-time routing selection, improve network transmission efficiency, and enhance user satisfaction.

[0104] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using physical hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly performed using software called a "logic compiler." This is similar to the software compilers used during program development. Before compilation, the original code must be written in a specific programming language, called a Hardware Description Language (HDL). There are many types of HDL, including ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that simply by programming a method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0106] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the memory control logic. Those skilled in the art will also appreciate that, in addition to implementing the controller purely in computer-readable program code, the controller can also be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, an embedded microcontroller, etc. by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing the various functions included therein can also be considered as structures within the hardware component. Alternatively, the means for implementing the various functions can be considered both a software module implementing the method and a structure within the hardware component.

[0107] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0108] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0109] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The embodiments of this specification are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable fraud case serial and parallel device to produce a machine, so that the instructions executed by the processor of the computer or other programmable fraud case serial and parallel device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable fraud case serial and parallel device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, the instruction device being implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0112] These computer program instructions may also be loaded onto a computer or other programmable device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0113] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0114] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0115] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0116] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0117] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0118] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0119] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0120] The foregoing description is merely an example of the present invention and is not intended to limit this document. Various modifications and variations are possible within the scope of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A data transmission method, characterized in that: The method comprises: receiving a data transmission instruction for target data; In response to the data transmission instruction, determining a target network corresponding to the target data, and obtaining data requirement information of the target data, current network topology information of the target network, and link status information; Determine the routing strategy corresponding to the target data using a pre-trained prediction model based on the data demand information of the target data, the current network topology information of the target network, and the link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The target data is transmitted through the target network according to a routing policy corresponding to the target data.

2. The method according to claim 1, characterized in that Before determining a routing strategy corresponding to the target data based on data demand information of the target data, current network topology information of the target network, and link status information using a pre-trained prediction model, the method further includes: Acquire the historical sample data, wherein the historical sample data includes historical data demand information, historical network topology information of the target network, historical link status information, and historical routing strategy; Determining a reward value corresponding to a first predicted routing action using the prediction model and based on the historical data demand information, the historical network topology information, and the historical link state information; Calculating a target Q value based on the reward value corresponding to the predicted routing action according to the update rule of the Q-learning algorithm; Determining whether the prediction model has converged based on the change amplitude of the target Q value, and obtaining a trained prediction model if the prediction model has converged; The method of using a pre-trained prediction model to determine a routing strategy corresponding to the target data based on data demand information of the target data, current network topology information of the target network, and link status information includes: Determining a reward value corresponding to a second predicted routing action based on data demand information of the target data, current network topology information of the target network, and link status information using the pre-trained prediction model; A routing strategy corresponding to the target data is determined according to a reward value corresponding to the second predicted routing action.

3. The method according to claim 2, characterized in that The data requirement information of the target data is type information obtained by classifying the target data according to the priority and data size of the target data.

4. The method according to claim 2, characterized in that The obtaining of the current network topology information of the target network includes: Broadcasting a preset data packet to each node in the target network using a preset controller; receiving feedback information from each node regarding the preset data packet; The current network topology information of the target network is constructed according to the feedback information, where the network topology information includes the connection relationship and port information between the nodes.

5. The method according to claim 4, characterized in that The constructing the current network topology information of the target network according to the feedback information includes: Constructing first network topology information according to the feedback information; Comparing feedback information from different nodes to obtain a comparison result, and performing integrity verification on the first network topology information based on the comparison result; In a case where the first network topology information passes the integrity check, the first network topology information is determined as the current network topology information of the target network.

6. The method according to claim 1, wherein The link status information includes the bandwidth and delay of the link, and obtaining the current link status information of the target network includes: Using a preset controller to send a preset message to each link in the target network, and determining the current link delay of the target network based on the round-trip time of the preset message; The flow information of the port of each node in the target network is obtained from the switch, and the bandwidth of the current link of the target network is determined according to the flow information.

7. A data transmission device, characterized in that: The device comprises: An instruction receiving module, used for receiving a data transmission instruction for target data; an information acquisition module, configured to determine, in response to the data transmission instruction, a target network corresponding to the target data, and acquire data requirement information of the target data, current network topology information of the target network, and link status information; a strategy determination module, configured to determine a routing strategy corresponding to the target data using a pre-trained prediction model based on data demand information of the target data, current network topology information of the target network, and link status information; wherein the prediction model is obtained by training a model constructed by a deep learning algorithm using a Q-learning algorithm based on historical sample data; The data transmission module is used to transmit the target data through the target network according to the routing policy corresponding to the target data.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the data transmission method according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the data transmission method according to any one of claims 1 to 6 when the computer program is executed by a processor.

Images