Data transmission method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data transmission method and device, electronic equipment and a storage medium, and belongs to the technical field of communication. The method comprises the following steps: inputting network topology information and network state information of a target network and a data transmission demand of initial data required to be transmitted by each source node into a pre-trained data scheduling model, and outputting target node information corresponding to each source node; determining path feature information corresponding to each candidate path between each source node and the corresponding target node according to the network topology information; inputting the network topology information, the network state information and the path feature information corresponding to each candidate path into a pre-trained path selection model, and outputting target path information corresponding to each source node; and sending the target node information and the target path information to the corresponding source node, so that the corresponding source node sends initial data to the target node according to the target node information and the target path information. According to the invention, the cross-domain transmission efficiency of data can be improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to a data transmission method, apparatus, electronic device, and storage medium. Background Technology

[0002] As the scale of Artificial Intelligence (AI) models continues to expand and application scenarios become more complex, task processing places higher demands on computing power. To overcome the resource limitations of a single computing center, cross-domain collaboration of distributed computing and network resources across multiple locations is often employed to support various AI tasks, such as large-scale AI models. Related technologies predefine the transmission paths between various computing centers and implement data transmission based on these predetermined paths during model training.

[0003] However, computing networks are constantly changing dynamically. Static transmission paths cannot adapt to dynamic network changes. Data transmission according to a predetermined path can easily lead to data not being transmitted to the corresponding nodes in a timely manner. Furthermore, when data streams of different sizes share limited network transmission resources, it can easily cause local overload of the link, leading to transmission congestion and preventing subsequent tasks from being carried out. In other words, related technologies suffer from low efficiency in cross-domain data transmission. Summary of the Invention

[0004] This application provides a data transmission method, apparatus, electronic device, and storage medium that can improve the efficiency of cross-domain data transmission.

[0005] To achieve the above objectives, one embodiment of this application provides a data transmission method, including: Obtain the data transmission requirements of the initial data to be transmitted for each source node, and obtain the network topology and network status information of the target network where each source node is located; The network topology information, network state information, and data transmission requirements corresponding to each source node are input into the pre-trained data scheduling model, and the target node information corresponding to each source node is output. The pre-trained data scheduling model is trained based on the sample network topology information, sample network state information, and sample data transmission requirements corresponding to the sample source nodes. Based on the network topology information, multiple candidate paths are determined between each source node and the corresponding target node, and the path feature information corresponding to each candidate path is determined based on the network topology information and network state information. The network topology information, network state information, and path feature information corresponding to each candidate path are input into the pre-trained path selection model, and the target path information corresponding to each source node is output. The pre-trained path selection model is trained based on the sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path in the sample data. The target node information and target path information are sent to the corresponding source node, so that the corresponding source node can send the initial data to the target node based on the target node information and target path information.

[0006] In some embodiments, network topology information, network state information, and path feature information corresponding to each candidate path are input into a pre-trained path selection model, and the output is the target path information corresponding to each source node, including: The network topology information, network state information, and path feature information corresponding to each candidate path are input into the pre-trained path selection model, and the output is the initial path information corresponding to each source node. When there is at least partial overlap between the initial path information corresponding to any two source nodes, obtain the data transmission type of the initial data to be transmitted between any two source nodes. For each data transmission type, determine the data transmission priority of the corresponding initial data; The initial path information of the corresponding source node is updated according to the data transmission priority of each data transmission to obtain the target path information of the corresponding source node, so as to obtain the target path information corresponding to each source node.

[0007] In some embodiments, multiple candidate paths between each source node and its corresponding target node are determined based on network topology information, including: Obtain path constraint information; For each source node, multiple initial paths between the source node and the corresponding target node are determined based on path constraint information and network topology information; The path cost value corresponding to each initial path is determined based on the network state information. The path cost value corresponding to each initial path is then filtered according to a preset cost threshold to obtain multiple candidate paths between each source node and the corresponding target node.

[0008] In some embodiments, before inputting network topology information, network state information, and data transmission requirements corresponding to each source node into the pre-trained data scheduling model, the method further includes: Obtain the sample data transmission requirements of each sample source node, and obtain the sample network topology information and sample network status information of the sample network in which each sample source node is located. The sample network topology information, sample network state information, and sample data transmission requirements corresponding to each sample source node are input into the data scheduling model, and the sample node information corresponding to each sample source node is output. Obtain the verification node information corresponding to each sample source node, and determine the differences in node information based on the sample node information and the verification node information; The data scheduling model is iteratively trained based on the differences in node information until the differences in node information obtained again do not exceed the preset threshold for differences in node information, thus obtaining an intermediate data scheduling model.

[0009] In some embodiments, after obtaining the intermediate data scheduling model, the method further includes: Based on the sample network topology information, multiple candidate sample paths are determined between each sample source node and the corresponding sample node, and the sample path feature information corresponding to each candidate sample path is determined based on the sample network topology information and the sample network state information. The sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path are input into the path selection model, and the sample path information corresponding to each sample source node is output. Based on the sample network topology information and sample network state information, the latency satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information are determined. The path selection reward function value is determined based on the latency satisfaction and bandwidth satisfaction. The path selection model is then iteratively trained based on the reward function value until the re-obtained latency satisfaction does not exceed the preset latency satisfaction and the re-obtained bandwidth satisfaction does not exceed the preset bandwidth satisfaction. This results in an intermediate path selection model.

[0010] In some embodiments, the latency satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information are determined based on the sample network topology information and the sample network state information, including: Obtain the latency threshold; Based on the sample network topology information and sample network status information, the bandwidth threshold of the sample path corresponding to the sample path information is determined, and the sample data transmission scale of the sample data is determined according to the sample data transmission requirements. The minimum bandwidth required to transmit sample data is determined based on the sample data transmission scale and latency threshold. The bandwidth satisfaction of the sample path is determined based on the bandwidth threshold and minimum bandwidth. The estimated transmission delay is determined based on the bandwidth threshold and the sample data transmission scale, and the delay satisfaction of the sample path is determined based on the estimated transmission delay and the delay threshold.

[0011] In some embodiments, after obtaining the intermediate path selection model, the method further includes: Based on the sample path information, sample data transmission requirements, and sample network status information corresponding to each sample source node, the computing resource utilization rate and average task latency are determined. Based on the sample path information and sample network status information corresponding to each sample source node, the link bandwidth utilization is determined; The task success rate is determined based on at least one of computing resource utilization, link bandwidth utilization, and average task latency. Based on the utilization rate of computing resources, the utilization rate of link bandwidth, the average latency of tasks, and the success rate of tasks, the data scheduling model and the path selection model are jointly iteratively trained until the utilization rate of computing resources, the utilization rate of link bandwidth, the average latency of tasks, and the success rate of tasks are all no more than the corresponding preset thresholds, thus obtaining the pre-trained data scheduling model and the pre-trained path selection model.

[0012] To achieve the above objectives, one embodiment of this application provides a data transmission apparatus, including: The acquisition module is used to acquire the data transmission requirements of the initial data to be transmitted by each source node, and to acquire the network topology information and network status information of the target network where each source node is located. The target node information determination module is used to input network topology information, network status information, and data transmission requirements corresponding to each source node into the pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on sample network topology information, sample network status information, and sample data transmission requirements corresponding to sample source nodes. The path feature information determination module is used to determine multiple candidate paths between each source node and the corresponding target node based on network topology information, and to determine the path feature information corresponding to each candidate path based on network topology information and network status information. The target path information determination module is used to input network topology information, network state information, and path feature information corresponding to each candidate path into the pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path in the sample data. The data transmission module is used to send target node information and target path information to the corresponding source node, so that the corresponding source node can send initial data transmission to the target node based on the target node information and target path information.

[0013] To achieve the above objectives, one aspect of this application provides a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the steps in the data transmission method provided in this application.

[0014] To achieve the above objectives, one aspect of this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps in the data transmission method provided in this application.

[0015] To achieve the above objectives, one aspect of this application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in the data transmission method provided in this application.

[0016] The data transmission method, apparatus, electronic device, and storage medium proposed in this application obtain the data transmission requirements of the initial data to be transmitted by each source node, and obtain the network topology and network state information of the target network in which each source node is located; input the network topology information, network state information, and data transmission requirements corresponding to each source node into a pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on sample network topology information, sample network state information, and sample data transmission requirements corresponding to sample source nodes; determine multiple candidate paths between each source node and the corresponding target node based on the network topology information, and determine the path feature information corresponding to each candidate path based on the network topology information and network state information; input the network topology information, network state information, and path feature information corresponding to each candidate path into a pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path of sample data; send the target node information and target path information to the corresponding source node, so that the corresponding source node sends the initial data transmission to the target node according to the target node information and target path information.

[0017] This application's embodiments are based on a two-stage intelligent decision-making mechanism. First, using a pre-trained data scheduling model, the optimal target node is dynamically selected for each source node, taking into account global awareness of network topology, real-time status, and data transmission requirements. Second, after determining the target node, multiple candidate paths are dynamically generated, and a pre-trained path selection model is used to select the optimal transmission path based on the path characteristics of each path. Thus, data transmission is performed based on the dynamically determined target node and target path, avoiding the poor data transmission efficiency caused by blindly sending data to resource-constrained or poorly connected nodes based on static paths in traditional methods. This application significantly reduces network congestion risk and greatly improves the efficiency of cross-domain data transmission through a two-stage dynamic joint optimization strategy of selecting points first and then routes.

[0018] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the system framework corresponding to the data transmission method provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the data transmission method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the target network for the data transmission method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the data processing flow of the data transmission method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the module structure of the data transmission device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] It should be noted that in each specific embodiment of this application, when data transmission requirements involve obtaining the initial data to be transmitted by the source node, permission or consent from the relevant personnel managing the source node is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when this application embodiment needs to obtain sensitive personal information of relevant personnel, separate permission or consent from the relevant personnel is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the separate permission or consent of the relevant personnel is the data transmission requirement for the necessary initial data to enable the normal operation of this application embodiment obtained. Other data obtained in this application embodiment are all authorized and legal data, and will not be described in detail here.

[0023] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, programmable consumer computer devices, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0024] The technical problems existing in the related technologies are as follows: As the scale of artificial intelligence (AI) models continues to expand and application scenarios become more complex, task processing places higher demands on computing power. To overcome the resource limitations of a single computing center, cross-domain collaboration of distributed computing and network resources across multiple locations is often employed to support various AI tasks, such as large-scale AI models. Related technologies predefine the transmission paths between various computing centers and implement data transmission based on these predetermined paths during model training.

[0025] However, computing networks are constantly changing dynamically. Static transmission paths cannot adapt to dynamic network changes. Data transmission according to a predetermined path can easily lead to data not being transmitted to the corresponding nodes in a timely manner. Furthermore, when data streams of different sizes share limited network transmission resources, it can easily cause local overload of the link, leading to transmission congestion and preventing subsequent tasks from being carried out. In other words, related technologies suffer from low efficiency in cross-domain data transmission.

[0026] For example, in scenarios highly sensitive to the timeliness of data transmission and the accuracy of network resource scheduling, such as real-time sensing in smart cities, cross-regional large-scale model inference services, or collaborative simulation in scientific computing, AI tasks often need to frequently exchange massive amounts of intermediate data (such as feature tensors, intermediate results, or parameter updates) between multiple geographically dispersed computing centers. Because related technologies rely on statically preset transmission paths (such as fixed routing strategies or pre-configured communication topologies), when encountering network bandwidth fluctuations, link failures, sudden increases in node load, or sudden high-priority traffic competition during actual operation, the predetermined paths cannot perceive the dynamic changes in the current computing network status. This causes critical data flows to queue on congested links, or even be dropped due to timeouts. At this point, downstream nodes are forced to halt due to a lack of necessary input, disrupting the task pipeline; some high-priority small-scale tasks may be severely delayed because they share the same bottleneck link with large-scale data flows, disrupting the overall task scheduling order. More seriously, the static path mechanism lacks the ability to assess link utilization, queue depth, and remaining bandwidth in real time, making it unable to dynamically select the optimal transmission channel among multiple feasible paths. This results in some high-bandwidth, low-load links being idle, while local hotspot links remain continuously overloaded, further exacerbating transmission latency and resource waste. Therefore, when facing high-concurrency, heterogeneous, and dynamically evolving cross-domain computing network environments, related technologies struggle to achieve efficient, elastic, and disturbance-resistant data transmission, severely restricting the end-to-end execution efficiency and system scalability of large-scale AI tasks.

[0027] The data transmission method, apparatus, electronic device, and storage medium proposed in this application obtain the data transmission requirements of the initial data to be transmitted by each source node, and obtain the network topology and network state information of the target network in which each source node is located; input the network topology information, network state information, and data transmission requirements corresponding to each source node into a pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on sample network topology information, sample network state information, and sample data transmission requirements corresponding to sample source nodes; determine multiple candidate paths between each source node and the corresponding target node based on the network topology information, and determine the path feature information corresponding to each candidate path based on the network topology information and network state information; input the network topology information, network state information, and path feature information corresponding to each candidate path into a pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path of sample data; send the target node information and target path information to the corresponding source node, so that the corresponding source node sends the initial data transmission to the target node according to the target node information and target path information.

[0028] This application's embodiments are based on a two-stage intelligent decision-making mechanism. First, using a pre-trained data scheduling model, the optimal target node is dynamically selected for each source node, taking into account global awareness of network topology, real-time status, and data transmission requirements. Second, after determining the target node, multiple candidate paths are dynamically generated, and a pre-trained path selection model is used to select the optimal transmission path based on the path characteristics of each path. Thus, data transmission is performed based on the dynamically determined target node and target path, avoiding the poor data transmission efficiency caused by blindly sending data to resource-constrained or poorly connected nodes based on static paths in traditional methods. This application significantly reduces network congestion risk and greatly improves the efficiency of cross-domain data transmission through a two-stage dynamic joint optimization strategy of selecting points first and then routes.

[0029] The specific details regarding the data transmission method, apparatus, electronic device, and storage medium provided in the embodiments of this application will be described in detail below.

[0030] Please see Figure 1 , Figure 1 This is a schematic diagram of the system framework corresponding to the data transmission method provided in the embodiments of this application. The data transmission method provided in the embodiments of this application can be applied to this system framework.

[0031] It includes terminal 140, Internet 130, gateway 120, server 110, etc.

[0032] Terminal 140 or server 110 can be a device that performs a data transmission method.

[0033] Terminal 140 includes, but is not limited to, mobile phones, tablets, computers, and intelligent computing centers. Terminal 140 can be a single device or a collection of multiple devices. For example, multiple computers can be interconnected via a local area network, sharing a single monitor to work collaboratively, thus forming a terminal 140. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.

[0034] Server 110 refers to a computer system that can provide certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0035] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.

[0036] The embodiments of this application can be applied to various scenarios, such as distributed training of ultra-large-scale language models, collaborative perception and decision-making in cross-regional intelligent driving, real-time multi-source data fusion in financial risk control, cross-domain collaborative simulation of scientific computing and climate simulation, and a series of data transmission tasks such as a multimodal data fusion platform for smart cities. The data transmission method proposed in the embodiments of this application can be used in other scenarios; it is only used as an example here and does not mean that the embodiments of this application limit the scenarios in which the data transmission method is applied.

[0037] Next, we will describe it from the perspective of data transmission devices, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating the data transmission method provided in an embodiment of this application. The data transmission method is applied to a data transmission device. Figure 2The method may include, but is not limited to, the following steps 210 to 250. When the data transmission device executes the data transmission method, the specific process is as follows. It should be noted first that this embodiment... Figure 2 The order of steps 210 to 250 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0038] Step 210: Obtain the data transmission requirements of the initial data to be transmitted by each source node, and obtain the network topology information and network status information of the target network where each source node is located; Step 220: Input the network topology information, network state information and data transmission requirements corresponding to each source node into the pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on the sample network topology information, sample network state information and sample data transmission requirements corresponding to the sample source nodes. Step 230: Determine multiple candidate paths between each source node and the corresponding target node based on the network topology information, and determine the path feature information corresponding to each candidate path based on the network topology information and network status information; Step 240: Input the network topology information, network state information and path feature information corresponding to each candidate path into the pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on the sample network topology information, sample network state information and sample path feature information corresponding to each sample candidate path of the sample data. Step 250: Send the target node information and target path information to the corresponding source node so that the corresponding source node can send the initial data to the target node based on the target node information and target path information.

[0039] Steps 210 to 250 are described in detail below.

[0040] In step 210, the data transmission requirements of the initial data to be transmitted by each source node are obtained, and the network topology information and network status information of the target network where each source node is located are obtained.

[0041] In this context, the source node refers to the node in the target network that initiates or leads cross-domain data transmission. The source node stores the initial data used to execute the cross-domain task. The source node executes the cross-domain task by transmitting this initial data to any other node in the target network (the target node). Typically, there are multiple cross-domain tasks to be executed in the target network, with each task corresponding to one source node and one target node.

[0042] Here, initial data refers to the raw input data originally held by the source node for performing cross-domain tasks, such as images, text, or samples collected by sensors. The initial data has not yet undergone any local computation or processing. Data transmission requirements refer to the data processing requirements of the initial data.

[0043] The target network refers to the communication network to which the source node intends to transmit data. Typically, the source node and the corresponding target node reside within the same target network. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of the target network for the data transmission method provided in the embodiments of this application. Figure 3 The target network comprises multiple city nodes, such as Beijing, Shanghai, Guangzhou, and Wuhan. Each node includes a corresponding computing center and a core switch (such as a computing network gateway). These city nodes are interconnected via the core switch. Each computing center provides computing resources and performs data processing and gradient calculations in model training tasks. For example, the Shanghai node can act as a source node, initiating cross-domain model training tasks. The target network, consisting of multiple nodes and the links connecting them, can be abstracted as a weighted undirected graph. ,in Represents a set of nodes. This represents the set of physical links connecting the nodes.

[0044] Network topology information refers to the connection structure and logical relationships between nodes in the target network at the current moment, including the link layout, path reachability, hierarchical architecture, and possible redundant or bottleneck links. Network status information refers to the status of available communication and computing resources in the target network at the current moment, including the bandwidth capacity, current utilization, remaining bandwidth, transmission latency, packet loss rate, queue length of each link, as well as the CPU load, memory usage, and cache status of nodes.

[0045] In some embodiments, before inputting network topology information, network state information, and data transmission requirements corresponding to each source node into the pre-trained data scheduling model, the method further includes: (1.1) Obtain the sample data transmission requirements of each sample source node, and obtain the sample network topology information and sample network status information of the sample network where each sample source node is located; (1.2) Input the sample network topology information, sample network status information and sample data transmission requirements corresponding to each sample source node into the data scheduling model, and output the sample node information corresponding to each sample source node; (1.3) Obtain the verification node information corresponding to each sample source node, and determine the differences in node information based on the sample node information and the verification node information; (1.4) The data scheduling model is iteratively trained based on the differences in node information until the differences in node information obtained again do not exceed the preset threshold for differences in node information, and then an intermediate data scheduling model is obtained.

[0046] In some embodiments, after obtaining the data transmission requirements of the initial data to be transmitted by each source node, as well as the network topology and network state information of the target network where each source node is located, this data needs to be input into a pre-trained data scheduling model to output the target node information corresponding to each source node, and the target node information is input into a pre-trained path selection model to output the target path information corresponding to each source node. To utilize these two models for data processing during inference, the data scheduling model and the path selection model need to be pre-trained separately to obtain pre-trained data scheduling models and pre-trained path selection models.

[0047] Furthermore, we first obtain the sample data transmission requirements for each sample source node, the sample network topology information, and the sample network state information of the sample network in which each sample source node is located. The definitions of sample source nodes, sample data, sample data transmission requirements, sample network, target network, sample network topology information, and sample network state information are similar to those of the target network, and will not be elaborated further here. It should be noted that the multiple sample data collected during training conform to a normal distribution, the purpose of which is to more realistically reflect the statistical characteristics of different communication behaviors in real-world scenarios.

[0048] Furthermore, the sample network topology information, sample network state information, and sample data transmission requirements corresponding to each sample source node are input into the untrained data scheduling model. The output is sample node information corresponding to each sample source node, where the sample node information represents the node determined by the data scheduling model that is suitable for the sample source node to send the corresponding sample data. After determining the sample node information corresponding to each sample source node, validation node information is obtained for each sample source node to compare with the corresponding sample node information. The validation node information represents the expected output information of the data scheduling model pre-set during training, and is used to represent the node that the expected data scheduling model outputs that is suitable for the sample source node to send the corresponding sample data to.

[0049] Furthermore, by calculating the degree of difference between sample node information and validation node information, the node information difference is obtained. This difference can be determined by calculating the cosine similarity or mean square error between the two, so that the data scheduling model can be trained based on the node information difference in the future. This application does not limit the specific method for determining the node information difference; the method can be selected according to the actual situation.

[0050] Furthermore, the training of the data scheduling model includes multiple rounds. In each round, the difference between the sample node information output by the data scheduling model and the verification node information used as labels is calculated, and the model parameters of the data scheduling model are adjusted according to the difference. The next round of training continues based on the data scheduling model with adjusted parameters to achieve iterative training of the data scheduling model. When the difference between the node information output by the data scheduling model is less than or equal to a preset threshold, an intermediate data scheduling model is obtained.

[0051] It should be noted that the intermediate data scheduling model refers to the model obtained after training the data scheduling model alone. Subsequently, the intermediate data scheduling model will be trained together with the intermediate path selection model to obtain the pre-trained data scheduling model and the pre-trained path selection model.

[0052] In some embodiments, after obtaining the intermediate data scheduling model, the method further includes: (2.1) Based on the sample network topology information, determine multiple sample candidate paths between each sample source node and the corresponding sample node, and determine the sample path feature information corresponding to each sample candidate path based on the sample network topology information and sample network state information; (2.2) Input the sample network topology information, sample network state information and sample path feature information corresponding to each sample candidate path into the path selection model, and output the sample path information corresponding to each sample source node; (2.3) Based on the sample network topology information and sample network state information, determine the delay satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information; (2.4) Determine the path selection reward function value based on the latency satisfaction and bandwidth satisfaction, and iteratively train the path selection model based on the reward function value until the re-obtained latency satisfaction does not exceed the preset latency satisfaction and the re-obtained bandwidth satisfaction does not exceed the preset bandwidth satisfaction, thus obtaining the intermediate path selection model.

[0053] First, steps (2.1) to (2.3) will be explained in detail: In some embodiments, after obtaining the intermediate number scheduling model, the sample node to be transmitted for each sample source node is determined based on the sample node information corresponding to each sample source node output by the intermediate number scheduling model; and multiple optional sample candidate paths between each sample source node and its corresponding sample node are determined based on each sample source node and its corresponding sample node, as well as the sample network topology information. Since each sample candidate path passes through different nodes, and the bandwidth between different nodes is different, that is, the transmission efficiency of each sample candidate path is different, in order to determine the optimal sample path, the sample network topology information, the sample network state information, and the sample path feature information corresponding to each sample candidate path need to be input into the untrained path selection model, and the path selection model outputs the sample path information corresponding to each sample source node. The sample path information represents the path selected from multiple sample candidate paths that is most suitable for transmitting sample data from the sample source node to the sample node.

[0054] Latency satisfaction refers to the time required to transmit sample data from the source node to the sample node along the selected sample path. Generally, the closer the latency is to or below the required threshold, the higher the satisfaction. Latency satisfaction reflects the path's ability to meet the service quality requirements in terms of latency. Bandwidth satisfaction refers to the degree of matching between the available bandwidth provided by the selected sample path and the transmission bandwidth required by the source node. When the path can support sufficient or even redundant bandwidth, bandwidth satisfaction is high. High bandwidth satisfaction indicates that the corresponding sample path can effectively support the throughput requirements of the data stream, thereby avoiding transmission bottlenecks or packet loss caused by insufficient bandwidth.

[0055] The following sections explain how latency satisfaction and bandwidth satisfaction are determined: In some embodiments, the latency satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information are determined based on the sample network topology information and the sample network state information, including: (3.1) Obtain the latency threshold; (3.2) Determine the bandwidth threshold of the sample path corresponding to the sample path information based on the sample network topology information and sample network status information, and determine the sample data transmission scale based on the sample data transmission requirements; (3.3) Determine the minimum bandwidth required to transmit sample data based on the sample data transmission scale and latency threshold; (3.4) Determine the bandwidth satisfaction of the sample path based on the bandwidth threshold and minimum bandwidth; (3.5) Determine the estimated transmission delay based on the bandwidth threshold and the sample data transmission scale, and determine the delay satisfaction of the sample path based on the estimated transmission delay and the delay threshold.

[0056] The latency threshold information refers to the maximum end-to-end transmission latency allowed for transmitting sample data. It represents the user's desired transmission time for the sample data, reflecting the application's requirements for real-time performance or response speed (such as the latency requirements for data transmission in scenarios like video calls and industrial control). The latency threshold information is usually set by the user or determined autonomously by the device based on the sample data transmission requirements. This application embodiment does not limit the method for determining the latency threshold information, and it can be adaptively adjusted according to the actual situation.

[0057] The bandwidth threshold refers to the upper limit of bandwidth that the sample network can provide for the selected sample path under the current network state. It is determined through the sample network topology information and sample network state information. Specifically, based on the sample network topology information and the selected sample path, the nodes that the sample path must pass through to transmit sample data are determined. Then, combined with the sample network state information, the bandwidth between every two adjacent nodes is determined, thereby determining the remaining available bandwidth of the sample path, and thus obtaining the bandwidth threshold of the sample path corresponding to the sample path information.

[0058] The sample data transmission scale refers to the size of the sample data to be transmitted, which is usually expressed in units such as bytes (Byte), kilobytes (KB), or bits (bit). The sample data transmission scale is usually determined directly based on the sample data transmission requirements, that is, the data payload information declared or carried by each sample source node when initiating the transmission task, such as the size of the file to be uploaded, the size of the data packets collected by the sensor, or the amount of feature data in a batch during AI training.

[0059] Furthermore, the minimum bandwidth required to transmit sample data is determined using the following formula. : .in, Indicates the scale of sample data transmission; This indicates the time delay threshold.

[0060] Furthermore, bandwidth satisfaction is determined using the following formula. : .in, Indicates the bandwidth threshold; This represents the minimum bandwidth; it's a non-zero decimal set to prevent division by a denominator of 0, when the bandwidth threshold is reached. Greater than the minimum bandwidth required by the request The bandwidth satisfaction rate is 1.

[0061] Furthermore, the estimated transmission delay is determined using the following formula: .in, This indicates the inherent delay of path transmission.

[0062] Furthermore, the time delay satisfaction is determined using the following formula: Among them, from 0 and ( The maximum value is selected as the latency satisfaction. Generally, the smaller the estimated transmission latency, the higher the latency satisfaction.

[0063] Next, we will continue with a detailed explanation of step (2.4): In some embodiments, the path selection reward function value is determined using the following formula. : .in, This is the delay penalty coefficient. This is the bandwidth penalty factor, typically... and All values ​​are set to 0.5 by default; this is just an example. and The specific value can be determined according to the actual situation, and the embodiments of this application do not limit it; The preset delay weight value, The preset bandwidth weight value, and The specific value can be set according to the actual situation.

[0064] Furthermore, based on the reward function value The path selection model is iteratively trained until the re-obtained latency satisfaction does not exceed the preset latency satisfaction and the re-obtained bandwidth satisfaction does not exceed the preset bandwidth satisfaction, at which point an intermediate path selection model is obtained. The specific values ​​of latency satisfaction and bandwidth satisfaction can be set according to actual conditions, and this application embodiment does not impose any restrictions on them.

[0065] It is understood that the embodiments of this application introduce a reward mechanism centered on latency satisfaction and bandwidth satisfaction to iteratively optimize the path selection model. In this way, the embodiments of this application can adaptively learn scheduling strategies that satisfy both latency and bandwidth constraints during the training process, which significantly improves the accuracy and robustness of the trained intermediate path selection model in terms of path selection.

[0066] In some embodiments, after obtaining the intermediate path selection model, the method further includes: (4.1) Based on the sample path information, sample data transmission requirements and sample network status information corresponding to each sample source node, determine the computing resource utilization rate and average task latency; (4.2) Determine the link bandwidth utilization based on the sample path information and sample network status information corresponding to each sample source node; (4.3) Determine the task success rate based on at least one of computing power resource utilization, link bandwidth utilization, and average task latency; (4.4) Based on the computing power resource utilization rate, link bandwidth utilization rate, average task latency and task success rate, the data scheduling model and path selection model are jointly iteratively trained until the computing power resource utilization rate, link bandwidth utilization rate, average task latency and task success rate obtained again do not exceed the corresponding preset thresholds, thus obtaining the pre-trained data scheduling model and the pre-trained path selection model.

[0067] In some embodiments, after completing the first stage of independent training to obtain the intermediate data scheduling model and the intermediate path selection model, the second stage of joint training is initiated to obtain the pre-trained data scheduling model and the pre-trained path selection model. The computing resource utilization rate is determined using the following formula: .in, This represents the set of nodes corresponding to all nodes in the sample network; This represents the actual computing power used by node i. It can be determined based on the requirements of sample path information, sample data transmission, and sample network status information. This represents the maximum computing power usage of node i. It can be determined from the sample network state information.

[0068] The link bandwidth utilization rate is determined using the following formula: .in, This represents the set of links corresponding to all links in the sample network; This represents the available bandwidth capacity of link (i, j), where i and j represent the two end nodes of the link; This represents the bandwidth already used on link (i, j). The used bandwidth is determined based on the sample path information and the sample network status.

[0069] The average task latency is determined using the following formula: .in, This represents the total number of cross-domain tasks currently being performed by the sample network. Each cross-domain task includes a pair of sample source nodes and a corresponding sample node. This represents the task processing latency when each cross-domain task is processed on the corresponding sample node; This represents the estimated transmission delay for transmitting each sample data to its corresponding sample node.

[0070] The success rate of the task is determined by the following formula: .in, The successful execution of a sample cross-domain task can be judged by the following criteria: the average task latency meets the preset completion time requirement; the computing resource utilization rate meets the preset computing resource utilization rate requirement; the link bandwidth utilization rate meets the preset link bandwidth utilization rate requirement; or the average task latency, computing resource utilization rate, and link bandwidth utilization rate all meet the preset requirements. It should be noted that this is only an example for illustration. In actual applications, the task success criteria can be set according to the actual situation to determine the task success rate. This application embodiment does not impose any restrictions on this.

[0071] In addition, the second phase of model training also needs to meet some constraints: This constraint means that in the target network, any link... The total bandwidth consumed by the initial data traffic that has been successfully deployed and allocated a certain bandwidth on the link must not exceed the bandwidth limit that the link can use. .

[0072] This constraint states that the sum of the estimated delays of all links on the path output by each request orchestration decision cannot exceed the maximum tolerable delay of the initial data transmitted in the corresponding request.

[0073] This constraint means that the currently available computing resources of the computing node selected to deploy a cross-domain task must meet the computing power requirements of the cross-domain task corresponding to the initial data.

[0074] This constraint means that the currently available storage resources of the computing node selected to deploy a cross-domain task must meet the storage space required by the task.

[0075] Thus, this application embodiment verifies the feasibility of each orchestration output by constraints such as bandwidth limit, end-to-end latency, computing power and storage availability, avoiding deployment failures caused by link congestion over-configuration, latency default and insufficient node resources, thereby improving the task deployment success rate and reducing the additional overhead caused by retries / rollbacks.

[0076] Furthermore, the overall training objective for the second phase of joint training is determined using the following formula: .in, This represents the utilization rate of computing resources, and is the preset first weight. This represents the link bandwidth utilization rate, which is a preset second weight. This represents the average task latency, which is a preset third weight. This represents the task success rate, which is the preset fourth weight. , , and The specific value can be set according to the actual situation.

[0077] That is, based on the overall training objective determined by computing resource utilization, link bandwidth utilization, average task latency, and task success rate, the data scheduling model and path selection model are jointly iteratively trained until the computing resource utilization, link bandwidth utilization, average task latency, and task success rate obtained again do not exceed the corresponding preset thresholds, thus obtaining the pre-trained data scheduling model and the pre-trained path selection model.

[0078] It is understood that the embodiments of this application dynamically evaluate the utilization rate of computing resources, the utilization rate of link bandwidth, and the average latency of tasks by comprehensively considering sample path information, data transmission requirements, and sample network status information. Based on this, a quantitative indicator of task success rate is constructed, and then the data scheduling model and the path selection model are jointly iteratively optimized to achieve the coordinated and efficient utilization of network resources and computing resources. While ensuring controllable task latency, the success rate of task execution is improved. Finally, through a closed-loop training mechanism under multi-objective constraints, the system automatically converges to a stable strategy that meets the preset performance threshold, significantly enhancing the system's adaptive scheduling capability and overall service quality in complex dynamic network environments. Under this training environment, a pre-trained data scheduling model and a pre-trained path selection model are obtained.

[0079] In step 220, the network topology information, network state information, and data transmission requirements corresponding to each source node are input into the pre-trained data scheduling model, and the target node information corresponding to each source node is output. The pre-trained data scheduling model is trained based on the sample network topology information, sample network state information, and sample data transmission requirements corresponding to the sample source nodes.

[0080] In some embodiments, after obtaining the pre-trained data scheduling model, during the inference process, network topology information, network state information, and data transmission requirements corresponding to each source node are input into the pre-trained data scheduling model, and the target node information corresponding to each source node is output. The target node information and the sample node information are defined similarly, both representing the optimal node to which the source node, determined by the model, will transmit the initial data.

[0081] In step 230, multiple candidate paths between each source node and the corresponding target node are determined based on the network topology information, and path feature information corresponding to each candidate path is determined based on the network topology information and network status information.

[0082] In some embodiments, the target node to which the source node needs to be transmitted is determined based on the target node information. After the target node is determined, multiple candidate paths between each source node and its corresponding target node are determined based on the network topology information. A candidate path refers to a feasible path that meets preset transmission requirements and is determined based on the network topology between a source node and its corresponding target node in the target network, so that the optimal target path can be selected from multiple candidate paths.

[0083] The following sections explain how multiple candidate paths are determined for each source node to its corresponding target node: In some embodiments, multiple candidate paths between each source node and its corresponding target node are determined based on network topology information, including: (5.1) Obtain path constraint information; (5.2) For each source node, determine multiple initial paths between the source node and the corresponding target node based on path constraint information and network topology information; (5.3) Determine the path cost value corresponding to each initial path based on the network status information, and filter the path cost value corresponding to each initial path according to the preset cost threshold to obtain multiple candidate paths between each source node and the corresponding target node.

[0084] Path constraint information refers to the restrictive conditions or service requirements imposed on feasible paths during the path planning process, used to exclude transmission paths that do not meet specific needs. In this embodiment, path constraint information refers to generating a maximum of k paths with a hop count not exceeding h. In practical applications, path constraint information includes, but is not limited to, maximum allowed hop count limits, end-to-end latency limits, minimum bandwidth requirements, requirements to avoid faulty links or specific areas, requirements for nodes that must be passed through or are prohibited from being passed through, energy consumption limits, etc., that is, path constraint information can be set according to actual conditions.

[0085] Furthermore, multiple initial paths between the source node and its corresponding target node are determined by comprehensively considering path constraint information and network topology information. Specifically, feasible initial paths that meet the conditions between the source node and its corresponding target node can be determined using generation algorithms such as depth-first search and heuristic cost functions.

[0086] Furthermore, for each initial path, the path cost corresponding to each initial path is determined based on the network state information using the following formula. : .in, It is a small positive number used to avoid the denominator being zero; it represents any initial path. This represents the forwarding delay between any two adjacent nodes on the initial path; This represents the available bandwidth between any two adjacent nodes on the initial path.

[0087] Furthermore, the path cost corresponding to each initial path is compared with a preset cost threshold. If the cost of an initial path is less than or equal to the cost threshold, it is considered that the initial path meets the acceptable range of performance or resource consumption under the target network state and is retained as a candidate path. Conversely, if the path cost exceeds the cost threshold, it indicates that the path may have excessive latency, excessive congestion risk, or does not meet business requirements, and is therefore eliminated. The remaining initial paths after screening are used as candidate paths. In this way, by prioritizing the elimination of inferior paths with substandard performance or excessive resource consumption, the complexity of the subsequent model action space is greatly reduced, ensuring that subsequent scheduling decisions are based on a high-quality and feasible set of candidate paths.

[0088] Next, we will continue to explain how, after identifying multiple candidate paths, we determine the path characteristic information corresponding to each candidate path based on network topology and network state information: Furthermore, based on network topology and network status information, the following network attribute information is determined: (1) Hop count: The hop count is numerically equal to the number of nodes in the corresponding candidate path minus one; (2) Total delay: The total delay is numerically equal to the sum of the delays of all links in the corresponding candidate path; (3) Average bandwidth: The average bandwidth of the links in the corresponding candidate path; (4) Minimum bandwidth: The available bandwidth of the bottleneck link in the corresponding candidate path. These network attribute information are then organized into path feature information. This allows the fixed-length vector to be input into the pre-trained path selector, which then outputs the target path information related to the optimal target path.

[0089] Among these metrics, hop count reflects the topological length of the candidate path, i.e., the number of intermediate nodes the data must traverse. A lower hop count typically means less forwarding overhead, lower processing latency, and higher reliability. Total latency is the sum of propagation, queuing, and transmission delays across all links on the candidate path, reflecting end-to-end response speed. Total latency is crucial for real-time services such as video calls and industrial control. Average bandwidth reflects the overall bandwidth supply level of the candidate path; a higher average bandwidth usually means stronger concurrent carrying capacity. Minimum bandwidth determines the maximum throughput limit of the entire path. Accurately identifying bottleneck bandwidth is crucial for ensuring the stability of high-volume or high-throughput services such as file transfers and video streaming. This application's embodiments construct comprehensive and refined path quality assessment features by considering multi-dimensional path characteristics such as hop count, total latency, average bandwidth, and minimum bandwidth. This allows subsequent path selection models to identify the optimal target path in complex and dynamic network environments based on rich path feature information, thereby significantly improving the efficiency of initial data transmission based on the target path.

[0090] In step 240, network topology information, network state information, and path feature information corresponding to each candidate path are input into the pre-trained path selection model, and the target path information corresponding to each source node is output. The pre-trained path selection model is trained based on the sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path in the sample data.

[0091] In some embodiments, network topology information, network state information, and path feature information corresponding to each candidate path are input into a pre-trained path selection model. The processing flow of the path selection model is as follows: (1) For each candidate path i, the network topology information, network state information, and path feature information corresponding to each candidate path are concatenated using the following formula: .in, This represents a feature obtained by concatenating network topology information and network state information. This represents the path feature information of the i-th candidate path.

[0092] (2) Scoring is performed based on the fully connected neural network (MLP) in the path selection model, and the path score corresponding to each candidate path is output. : .

[0093] (3) Perform a softmax normalization operation on the scores of all candidate paths to obtain the final path selection probability distribution. And select the candidate path with the highest probability as the target path: .in, This represents the exponential sum of the scores for all candidate paths; This represents the exponential sum of the scores corresponding to the current candidate path.

[0094] It is understood that the embodiments of this application input network topology information, real-time network status information, and refined path features of each candidate path into a pre-trained path selection model. The model intelligently outputs the target path corresponding to each source node, which not only significantly improves the accuracy and adaptability of path selection, but also takes into account multi-dimensional path quality evaluation indicators such as latency, bandwidth, and reliability, effectively avoiding the limitations of traditional static or single-objective routing strategies. At the same time, this method supports end-to-end optimization, can dynamically respond to network changes, improve overall transmission efficiency, resource utilization, and system robustness, and provides efficient and scalable technical support for intelligent network scheduling.

[0095] In some embodiments, network topology information, network state information, and path feature information corresponding to each candidate path are input into a pre-trained path selection model, and the output is the target path information corresponding to each source node, including: (6.1) Input the network topology information, network state information and path feature information corresponding to each candidate path into the pre-trained path selection model, and output the initial path information corresponding to each source node; (6.2) When there is at least some overlap between the initial path information corresponding to any two source nodes, obtain the data transmission type of the initial data to be transmitted between any two source nodes; (6.3) Determine the data transmission priority of the corresponding initial data according to each data transmission type; (6.4) Update the initial path information of the corresponding source node according to the data transmission priority of each data transmission to obtain the target path information of the corresponding source node, so as to obtain the target path information corresponding to each source node.

[0096] In some embodiments, the path information determined based on network topology information, network status information, and path feature information corresponding to each candidate path is used as the initial path information corresponding to each source node. Then, in response to resource contention or link sharing conflicts that may occur during the path selection process, this application embodiment introduces a data service semantic awareness mechanism: when it is detected that the initial paths selected by any two source nodes have at least partial overlap in topology, that is, when any two initial paths share a certain link, switch, or node at the same time, the device will actively obtain the data transmission type of the data to be transmitted by the two source nodes, so as to determine the transmission priority of the corresponding initial data according to the data transmission type.

[0097] For example, as shown in Table 1, different data transmission types of the initial data result in different data characteristics, and these different data characteristics will affect the subsequent evaluation of the transmission priority of the initial data: Table 1

[0098] Furthermore, the data transmission priority of the corresponding initial data is determined based on the data transmission type. In this embodiment, the data transmission priority is a priority weight value; the higher the value, the higher the urgency of transmitting the corresponding initial data. For example, industrial control data is given the highest priority to ensure real-time performance and reliability, audio and video conferencing data is given the next highest priority to ensure a smooth experience, while ordinary file downloads or background synchronization tasks are assigned lower priorities. In this way, this embodiment maps data transmission types to explicit priorities, enabling differentiated services to be implemented when there are multiple cross-domain tasks on the target network and path resources are prioritized, effectively avoiding performance degradation of critical services due to network congestion or path conflicts.

[0099] Furthermore, the initial path information of the corresponding source node is updated according to the data transmission priority of each data transmission to obtain the target path information of the corresponding source node. This ensures that the target path information of each source node not only indicates the target path that can reach the target node, but also indicates the data transmission order when there is a conflict between the current data transmission and other cross-domain task data transmissions.

[0100] It is understood that the embodiments of this application significantly improve the network resource coordination capability and service quality assurance level in multi-source concurrent transmission scenarios by introducing a collaborative scheduling mechanism driven by path conflict awareness and service priority. First, a pre-trained model is used to select initial paths for each source node. When path overlap is detected as potentially causing resource contention, its service semantics are further identified by combining data transmission type and mapped to a preset data transmission priority. Finally, the paths of low-priority flows are dynamically adjusted according to priority to free up bandwidth for high-priority services or avoid congestion. This process not only effectively alleviates link contention and performance interference problems but also achieves differentiated protection for critical services (such as industrial control and real-time communication), improving overall network throughput efficiency while enhancing the system's robustness and intelligence in complex, high-load environments.

[0101] In step 250, the target node information and target path information are sent to the corresponding source node so that the corresponding source node can send the initial data to the target node based on the target node information and target path information.

[0102] In some embodiments, after determining the target node information and the target path information, these two pieces of information are sent to the corresponding source node, so that each source node transmits the initial data to the corresponding target node along the target path according to the corresponding target node information and target path information.

[0103] For example, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the data processing flow of the data transmission method provided in this application embodiment. The network topology information, network state information, and data transmission requirements corresponding to each source node of the target network obtained through global perception are input into a data scheduling model. The model outputs target node information corresponding to each source node and sends this target node information to the corresponding source node. Simultaneously, multiple candidate paths between each source node and its corresponding target node are determined based on the network topology information. Path feature information corresponding to each candidate path is determined based on the network topology information and network state information. The network topology information, network state information, and path feature information corresponding to each candidate path are input into a pre-trained path selection model. The model outputs target path information corresponding to each source node and sends this target path information to the corresponding source node. Thus, each source node can transmit the initial data required for the cross-domain task to the target node based on the target node information and the target path information.

[0104] Furthermore, when there is at least partial overlap between the initial path information corresponding to any two source nodes, the bandwidth allocation for each initial data determined in steps (6.1) to (6.4) is determined using the following formula: .in, represents the available bandwidth of the current link, and wi represents the priority weight of the i-th initial data stream.

[0105] It is understood that the embodiments of this application can dynamically determine the optimal target node corresponding to the real-time network status and task requirements through a pre-trained data scheduling model; then, through a pre-trained path selection model, an optimal target path is dynamically selected from multiple candidate paths, realizing real-time adaptive transmission routing and effectively avoiding congested links in the network. That is, the embodiments of this application, based on a two-stage dynamic intelligent decision-making mechanism of "selecting the point first, then the route," make data transmission no longer limited to fixed paths, and can flexibly adapt to dynamic changes in the network. This not only significantly improves the efficiency and success rate of cross-domain data transmission, but also effectively manages data flows of different scales, reduces the risk of local link overload and transmission congestion, thereby ensuring the smooth progress of subsequent tasks, and ultimately achieving the synergistic and efficient utilization of computing power and network resources.

[0106] like Figure 5As shown, Figure 5 This is a schematic diagram of the module structure of the data transmission device provided in this application embodiment. The data transmission device 300 may include the following modules 310 to 350: The acquisition module 310 is used to acquire the data transmission requirements of the initial data to be transmitted by each source node, and to acquire the network topology information and network status information of the target network where each source node is located. The target node information determination module 320 is used to input network topology information, network status information and data transmission requirements corresponding to each source node into the pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on sample network topology information, sample network status information and sample data transmission requirements corresponding to sample source nodes. The path feature information determination module 330 is used to determine multiple candidate paths between each source node and the corresponding target node based on network topology information, and to determine the path feature information corresponding to each candidate path based on network topology information and network status information. The target path information determination module 340 is used to input network topology information, network state information and path feature information corresponding to each candidate path into the pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on the sample network topology information, sample network state information and sample path feature information corresponding to each sample candidate path of the sample data. The data transmission module 350 is used to send the target node information and target path information to the corresponding source node, so that the corresponding source node can send the initial data transmission to the target node according to the target node information and target path information.

[0107] In some embodiments, the target path information determination module 340 is used for: The network topology information, network state information, and path feature information corresponding to each candidate path are input into the pre-trained path selection model, and the output is the initial path information corresponding to each source node. When there is at least partial overlap between the initial path information corresponding to any two source nodes, obtain the data transmission type of the initial data to be transmitted between any two source nodes. For each data transmission type, determine the data transmission priority of the corresponding initial data; The initial path information of the corresponding source node is updated according to the data transmission priority of each data transmission to obtain the target path information of the corresponding source node, so as to obtain the target path information corresponding to each source node.

[0108] In some embodiments, the path feature information determination module 330 is used for: Obtain path constraint information; For each source node, multiple initial paths between the source node and the corresponding target node are determined based on path constraint information and network topology information; The path cost value corresponding to each initial path is determined based on the network state information. The path cost value corresponding to each initial path is then filtered according to a preset cost threshold to obtain multiple candidate paths between each source node and the corresponding target node.

[0109] In some embodiments, the target node information determination module 320 is used for: Obtain the sample data transmission requirements of each sample source node, and obtain the sample network topology information and sample network status information of the sample network in which each sample source node is located. The sample network topology information, sample network state information, and sample data transmission requirements corresponding to each sample source node are input into the data scheduling model, and the sample node information corresponding to each sample source node is output. Obtain the verification node information corresponding to each sample source node, and determine the differences in node information based on the sample node information and the verification node information; The data scheduling model is iteratively trained based on the differences in node information until the differences in node information obtained again do not exceed the preset threshold for differences in node information, thus obtaining an intermediate data scheduling model.

[0110] In some embodiments, the target node information determination module 320 is further configured to: Based on the sample network topology information, multiple candidate sample paths are determined between each sample source node and the corresponding sample node, and the sample path feature information corresponding to each candidate sample path is determined based on the sample network topology information and the sample network state information. The sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path are input into the path selection model, and the sample path information corresponding to each sample source node is output. Based on the sample network topology information and sample network state information, the latency satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information are determined. The path selection reward function value is determined based on the latency satisfaction and bandwidth satisfaction. The path selection model is then iteratively trained based on the reward function value until the re-obtained latency satisfaction does not exceed the preset latency satisfaction and the re-obtained bandwidth satisfaction does not exceed the preset bandwidth satisfaction. This results in an intermediate path selection model.

[0111] In some embodiments, the target node information determination module 320 is further configured to: Obtain the latency threshold; Based on the sample network topology and sample network status, determine the bandwidth threshold of the sample path corresponding to the sample path information, and determine the sample data transmission scale based on the sample data transmission requirements. The minimum bandwidth required to transmit sample data is determined based on the sample data transmission scale and latency threshold. The bandwidth satisfaction of the sample path is determined based on the bandwidth threshold and minimum bandwidth. The estimated transmission delay is determined based on the bandwidth threshold and the sample data transmission scale, and the delay satisfaction of the sample path is determined based on the estimated transmission delay and the delay threshold.

[0112] In some embodiments, the target node information determination module 320 is further configured to: Based on the sample path information, sample data transmission requirements, and sample network status information corresponding to each sample source node, the computing resource utilization rate and average task latency are determined. Based on the sample path information and sample network status information corresponding to each sample source node, the link bandwidth utilization is determined; The task success rate is determined based on at least one of computing resource utilization, link bandwidth utilization, and average task latency. Based on the utilization rate of computing resources, the utilization rate of link bandwidth, the average latency of tasks, and the success rate of tasks, the data scheduling model and the path selection model are jointly iteratively trained until the utilization rate of computing resources, the utilization rate of link bandwidth, the average latency of tasks, and the success rate of tasks are all no more than the corresponding preset thresholds, thus obtaining the pre-trained data scheduling model and the pre-trained path selection model.

[0113] The data transmission method, apparatus, electronic device, and storage medium proposed in this application obtain the data transmission requirements of the initial data to be transmitted by each source node, and obtain the network topology and network state information of the target network in which each source node is located; input the network topology information, network state information, and data transmission requirements corresponding to each source node into a pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on sample network topology information, sample network state information, and sample data transmission requirements corresponding to sample source nodes; determine multiple candidate paths between each source node and the corresponding target node based on the network topology information, and determine the path feature information corresponding to each candidate path based on the network topology information and network state information; input the network topology information, network state information, and path feature information corresponding to each candidate path into a pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path of sample data; send the target node information and target path information to the corresponding source node, so that the corresponding source node sends the initial data transmission to the target node according to the target node information and target path information.

[0114] This application's embodiments are based on a two-stage intelligent decision-making mechanism. First, using a pre-trained data scheduling model, the optimal target node is dynamically selected for each source node, taking into account global awareness of network topology, real-time status, and data transmission requirements. Second, after determining the target node, multiple candidate paths are dynamically generated, and a pre-trained path selection model is used to select the optimal transmission path based on the path characteristics of each path. Thus, data transmission is performed based on the dynamically determined target node and target path, avoiding the poor data transmission efficiency caused by blindly sending data to resource-constrained or poorly connected nodes based on static paths in traditional methods. This application significantly reduces network congestion risk and greatly improves the efficiency of cross-domain data transmission through a two-stage dynamic joint optimization strategy of selecting points first and then routes.

[0115] like Figure 6 As shown, Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: The processor 401 can be implemented using a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 using the data transmission method of the embodiments of this application. Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0116] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data transmission method.

[0117] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0118] The data transmission method, apparatus, electronic device, and storage medium proposed in this application obtain the data transmission requirements of the initial data to be transmitted by each source node, and obtain the network topology and network state information of the target network in which each source node is located; input the network topology information, network state information, and data transmission requirements corresponding to each source node into a pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on sample network topology information, sample network state information, and sample data transmission requirements corresponding to sample source nodes; determine multiple candidate paths between each source node and the corresponding target node based on the network topology information, and determine the path feature information corresponding to each candidate path based on the network topology information and network state information; input the network topology information, network state information, and path feature information corresponding to each candidate path into a pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on sample network topology information, sample network state information, and sample path feature information corresponding to each sample candidate path of sample data; send the target node information and target path information to the corresponding source node, so that the corresponding source node sends the initial data transmission to the target node according to the target node information and target path information.

[0119] This application's embodiments are based on a two-stage intelligent decision-making mechanism. First, using a pre-trained data scheduling model, the optimal target node is dynamically selected for each source node, taking into account global awareness of network topology, real-time status, and data transmission requirements. Second, after determining the target node, multiple candidate paths are dynamically generated, and a pre-trained path selection model is used to select the optimal transmission path based on the path characteristics of each path. Thus, data transmission is performed based on the dynamically determined target node and target path, avoiding the poor data transmission efficiency caused by blindly sending data to resource-constrained or poorly connected nodes based on static paths in traditional methods. This application significantly reduces network congestion risk and greatly improves the efficiency of cross-domain data transmission through a two-stage dynamic joint optimization strategy of selecting points first and then routes.

[0120] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0121] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0124] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0125] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0127] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A data transmission method, characterized in that, include: Obtain the data transmission requirements of the initial data to be transmitted for each source node, and obtain the network topology and network status information of the target network where each source node is located; The network topology information, the network state information, and the data transmission requirements corresponding to each source node are input into the pre-trained data scheduling model, and the target node information corresponding to each source node is output. The pre-trained data scheduling model is trained based on the sample network topology information, the sample network state information, and the sample data transmission requirements corresponding to the sample source nodes. Based on the network topology information, multiple candidate paths are determined between each source node and the corresponding target node, and path feature information corresponding to each candidate path is determined based on the network topology information and the network state information. The network topology information, the network state information, and the path feature information corresponding to each candidate path are input into a pre-trained path selection model, and the target path information corresponding to each source node is output. The pre-trained path selection model is trained based on the sample network topology information, the sample network state information, and the sample path feature information corresponding to each sample candidate path of the sample data. The target node information and the target path information are sent to the corresponding source node, so that the corresponding source node sends the initial data to the target node according to the target node information and the target path information.

2. The data transmission method according to claim 1, characterized in that, The step of inputting the network topology information, the network state information, and the path feature information corresponding to each candidate path into a pre-trained path selection model, and outputting the target path information corresponding to each source node, includes: The network topology information, the network state information, and the path feature information corresponding to each candidate path are input into the pre-trained path selection model, and the initial path information corresponding to each source node is output. When there is at least partial overlap between the initial path information corresponding to any two source nodes, obtain the data transmission type of the initial data to be transmitted between the two source nodes. Based on each of the data transmission types, the corresponding data transmission priority of the initial data is determined; The initial path information of the corresponding source node is updated according to the data transmission priority of each source node to obtain the target path information of the corresponding source node, so as to obtain the target path information corresponding to each source node.

3. The data transmission method according to claim 1, characterized in that, The step of determining multiple candidate paths between each source node and its corresponding target node based on the network topology information includes: Obtain path constraint information; For each source node, multiple initial paths between the source node and the corresponding target node are determined based on the path constraint information and the network topology information. The path cost value corresponding to each initial path is determined based on the network state information. The path cost value corresponding to each initial path is then filtered according to a preset cost threshold to obtain multiple candidate paths between each source node and the corresponding target node.

4. The data transmission method according to claim 1, characterized in that, Before inputting the network topology information, the network state information, and the data transmission requirements corresponding to each source node into the pre-trained data scheduling model, the method further includes: Obtain the sample data transmission requirements of each sample source node, and obtain the sample network topology information and sample network status information of the sample network in which each sample source node is located. The sample network topology information, the sample network status information, and the sample data transmission requirements corresponding to each sample source node are input into the data scheduling model, and the sample node information corresponding to each sample source node is output. Obtain the verification node information corresponding to each sample source node, and determine the differences in node information based on the sample node information and the verification node information; The data scheduling model is iteratively trained based on the differences in node information until the differences in node information obtained again do not exceed a preset threshold for differences in node information, thus obtaining an intermediate data scheduling model.

5. The data transmission method according to claim 4, characterized in that, After obtaining the intermediate data scheduling model, the following is also included: Based on the sample network topology information, multiple candidate sample paths are determined between each sample source node and the corresponding sample node, and sample path feature information corresponding to each candidate sample path is determined based on the sample network topology information and the sample network state information. The sample network topology information, the sample network state information, and the sample path feature information corresponding to each sample candidate path are input into the path selection model, and the sample path information corresponding to each sample source node is output. Based on the sample network topology information and the sample network state information, the latency satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information are determined. The path selection reward function value is determined based on the latency satisfaction and the bandwidth satisfaction, and the path selection model is iteratively trained based on the reward function value until the re-obtained latency satisfaction does not exceed the preset latency satisfaction and the re-obtained bandwidth satisfaction does not exceed the preset bandwidth satisfaction, thus obtaining an intermediate path selection model.

6. The data transmission method according to claim 5, characterized in that, The step of determining the latency satisfaction and bandwidth satisfaction of the sample path corresponding to the sample path information based on the sample network topology information and the sample network state information includes: Obtain the latency threshold; Based on the sample network topology information and the sample network status information, the bandwidth threshold of the sample path corresponding to the sample path information is determined, and the sample data transmission scale of the sample data is determined based on the sample data transmission requirements. The minimum bandwidth required to transmit the sample data is determined based on the sample data transmission scale and the latency threshold. The bandwidth satisfaction of the sample path is determined based on the bandwidth threshold and the minimum bandwidth. The estimated transmission delay is determined based on the bandwidth threshold and the sample data transmission scale, and the delay satisfaction of the sample path is determined based on the estimated transmission delay and the delay threshold.

7. The data transmission method according to claim 5, characterized in that, After obtaining the intermediate path selection model, the following is also included: Based on the sample path information, sample data transmission requirements, and sample network status information corresponding to each sample source node, the computing resource utilization rate and average task latency are determined. The link bandwidth utilization rate is determined based on the sample path information and sample network status information corresponding to each sample source node; The task success rate is determined based on at least one of the computing power resource utilization rate, the link bandwidth utilization rate, and the average task latency. Based on the computing resource utilization rate, the link bandwidth utilization rate, the average task latency, and the task success rate, the data scheduling model and the path selection model are jointly iteratively trained until the computing resource utilization rate, the link bandwidth utilization rate, the average task latency, and the task success rate are all obtained again without exceeding the corresponding preset thresholds, thus obtaining the pre-trained data scheduling model and the pre-trained path selection model.

8. A data transmission device, characterized in that, include: The acquisition module is used to acquire the data transmission requirements of the initial data to be transmitted by each source node, and to acquire the network topology information and network status information of the target network where each source node is located. The target node information determination module is used to input the network topology information, the network state information, and the data transmission requirements corresponding to each source node into the pre-trained data scheduling model, and output the target node information corresponding to each source node. The pre-trained data scheduling model is trained based on the sample network topology information, the sample network state information, and the sample data transmission requirements corresponding to the sample source nodes. The path feature information determination module is used to determine multiple candidate paths between each source node and the corresponding target node based on the network topology information, and to determine the path feature information corresponding to each candidate path based on the network topology information and the network state information. The target path information determination module is used to input the network topology information, the network state information, and the path feature information corresponding to each candidate path into a pre-trained path selection model, and output the target path information corresponding to each source node. The pre-trained path selection model is trained based on the sample network topology information, the sample network state information, and the sample path feature information corresponding to each sample candidate path of the sample data. The data transmission module is used to send the target node information and the target path information to the corresponding source node, so that the corresponding source node can send the initial data transmission to the target node according to the target node information and the target path information.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the data transmission method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the data transmission method according to any one of claims 1 to 7.

Images