Data transmission method, device, system, equipment, storage medium and program product

Abstract

The invention relates to the technical field of communication, and provides a data transmission method, device, system and equipment, a storage medium and a program product, and the method comprises the steps: receiving a data transmission request of a terminal; the data sending request carries to-be-transmitted data and a target address; determining a target path based on the path detection frequency, the path validity period of the cache path and the network index information of the edge node; the target path is a data transmission path with the optimal transmission delay; and transmitting the to-be-transmitted data to the target address based on the target path. Through the above mode, a pre-scheduling mechanism and a real-time path reasoning mechanism are introduced, the defect that path selection in a traditional articulated naturality web data transmission method is difficult to dynamically adjust and only depends on generation of a static routing mechanism is avoided, the flexibility of path scheduling is improved, and data transmission time delay can be effectively reduced.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data transmission method, apparatus, system, device, storage medium, and program product. Background Technology

[0002] As the scale of the visual network expands, the number of video-related terminals accessing it also gradually increases, leading to a more complex, highly distributed, and multi-layered network structure. Based on this, NGVI (Next Generation Visual Internet) has emerged. NGVI is a media service infrastructure designed to meet the transmission needs of various emerging immersive audio and video services. However, how to achieve high-quality data transmission within this complex real-time network (RTN) of NGVI has become a pressing issue.

[0003] Currently, video networks typically include multiple network nodes, and various general video terminals can transmit data with different business systems through data transmission paths formed by different network nodes.

[0004] However, this data transmission method for traditional video network architecture suffers from high data transmission latency and low path scheduling flexibility. Summary of the Invention

[0005] This application provides a data transmission method, apparatus, system, device, storage medium, and program product to solve the technical problems of high data transmission latency and low path scheduling flexibility in existing data transmission methods.

[0006] In a first aspect, embodiments of this application provide a data transmission method applied to an edge node cluster, the edge node cluster including edge nodes, the data transmission method including: receiving a data transmission request from a terminal; the data transmission request carrying data to be transmitted and a target address; determining a target path based on path detection frequency, the path validity period of the cached path and network indicator information of the edge node; the target path being the data transmission path with optimal transmission latency; and transmitting the data to be transmitted to the target address based on the target path.

[0007] In one embodiment, determining a target path based on path detection frequency, the path validity period of a cached path, and network indicator information of edge nodes includes: if the path validity period has expired, collecting network indicator information of edge nodes based on the path detection frequency; determining a target candidate path based on the network indicator information; if the transmission latency of the target candidate path is better than the transmission latency of the cached path, then using the target candidate path as the target path, updating the target path to a cached path, and updating the path validity period.

[0008] In one embodiment, determining a target path based on path detection frequency, the validity period of a cached path, and network indicator information of edge nodes includes: if the path validity period has not expired, acquiring resource information of the edge node cluster according to a preset period; if the resource information meets the path detection frequency adjustment conditions, updating the path detection frequency based on the resource information; collecting network indicator information of edge nodes based on the updated path detection frequency; determining a target candidate path based on the network indicator information; if the transmission latency of the target candidate path is better than the transmission latency of the cached path, using the target candidate path as the target path, updating the target path to a cached path, and updating the path validity period.

[0009] In one embodiment, determining a target candidate path based on network indicator information includes: inputting the network indicator information into an edge scheduling model to perform path filtering and obtain a candidate path set output by the edge scheduling model; the candidate path set includes at least two candidate paths; sending the candidate path set and network indicator information to a cloud node; using the candidate path set and network indicator information together in an intelligent scheduling model to perform real-time path reasoning and generate a target candidate path; receiving the target candidate path sent by the cloud node; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set.

[0010] In one embodiment, the edge scheduling model is obtained by pre-training the first initial model based on the pre-trained parameters of the intelligent scheduling model, and the first initial model is obtained by pruning the XGBoost model.

[0011] In one embodiment, determining the target path based on the path detection frequency, the path validity period of the cached path, and the network indicator information of the edge nodes further includes: if the path validity period has not expired, obtaining the resource information of the edge node cluster according to a preset period; if the resource information does not meet the path detection frequency adjustment conditions, then using the cached path as the target path.

[0012] Secondly, embodiments of this application provide a data transmission method applied to cloud nodes. The data transmission method includes: receiving a candidate path set and network indicator information of edge nodes; the candidate path set includes at least two candidate paths, which are determined by the edge node cluster based on path detection frequency, the validity period of cached paths, and network indicator information; inputting the candidate path set and network indicator information into an intelligent scheduling model to perform real-time path reasoning and obtain a target candidate path output by the intelligent scheduling model; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set; sending the target candidate path to the edge node cluster; the target candidate path is used to update the cached paths of the edge node cluster, and the cached paths are used to transmit data to be transmitted by the terminal.

[0013] In one embodiment, the intelligent scheduling model is a model built based on a deep neural network and a pre-trained path scheduling model. The path scheduling model is trained based on the following steps: pre-training a second initial model based on sample feasible paths, sample network indicator information corresponding to sample feasible paths, and sample target variables corresponding to sample feasible paths and sample network indicator information to obtain an initial path scheduling model; the sample network indicator information includes multiple sample network indicators, and the sample target variables include sample transmission latency, sample service quality indicators, and sample path classification labels; pre-training the initial path scheduling model to obtain the path scheduling model.

[0014] In one embodiment, pre-training an initial path scheduling model to obtain a path scheduling model includes: performing feature importance analysis on multiple sample network indicators based on the initial path scheduling model to determine at least one key indicator; the key indicator is a sample network indicator whose feature importance is higher than a preset threshold, and the feature importance is used to characterize the predictive contribution to the target variable of the sample; performing feature optimization processing on the key indicator to generate optimized network indicators; and pre-training the initial path scheduling model based on the optimized network indicators to obtain a path scheduling model.

[0015] In one embodiment, the sample feasible path, the sample network indicator information corresponding to the sample feasible path, and the sample target variable corresponding to the sample feasible path and the sample network indicator information are obtained based on the following steps: data preprocessing is performed on the initial sample feasible path and the initial sample network indicator information corresponding to the initial sample feasible path to obtain the sample feasible path and the sample network indicator information corresponding to the sample feasible path; the edge node cluster includes multiple edge nodes, the initial sample feasible path is the data transmission path between the access edge node and the target edge node, and the access edge node and the target edge node are any edge node in the edge node cluster; the sample target variable corresponding to the sample feasible path and the sample network indicator information is determined according to the path scheduling scenario.

[0016] In one embodiment, before receiving the candidate path set and the network indicator information of the edge nodes, the method further includes: sending the pre-training parameters of the intelligent scheduling model to the edge node cluster; the pre-training parameters are used to train the edge scheduling model of the edge node cluster, and the edge scheduling model is used to generate the candidate path set.

[0017] In one embodiment, the sample network metrics include at least two of the following: sample bandwidth, sample bandwidth utilization, sample queuing latency, sample network jitter rate, sample edge node load, sample network packet loss rate, and sample packet out-of-order rate.

[0018] Thirdly, embodiments of this application provide a data transmission device deployed in an edge node cluster, the edge node cluster including edge nodes, the data transmission device including: a first receiving module, used to receive a data transmission request from a terminal; the data transmission request carries data to be transmitted and a target address; a path determination module, used to determine a target path based on path detection frequency, the path validity period of the cached path and network indicator information of the edge node; the target path is a data transmission path with optimal transmission latency; and a data transmission module, used to transmit the data to be transmitted to the target address based on the target path.

[0019] Fourthly, embodiments of this application provide a data transmission device deployed on a cloud node. The data transmission device includes: a second receiving module, used to receive a candidate path set and network indicator information of edge nodes; the candidate path set includes at least two candidate paths, and the candidate path set is determined by the edge node cluster based on path detection frequency, path validity period of cached paths, and network indicator information; a path filtering module, used to input the candidate path set and network indicator information into an intelligent scheduling model, perform real-time path reasoning, and obtain the target candidate path output by the intelligent scheduling model; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set; and a data sending module, used to send the target candidate path to the edge node cluster; the target candidate path is used to update the cached paths of the edge node cluster, and the cached paths are used to transmit data to be transmitted by the terminal.

[0020] Fifthly, embodiments of this application provide a data transmission system, including an edge node cluster and a cloud node. The edge node cluster executes any of the above-described data transmission methods applied to the edge node cluster, and the cloud node executes any of the above-described data transmission methods applied to the cloud node.

[0021] Sixthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the data transmission methods described above.

[0022] In a seventh aspect, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the data transmission methods described above.

[0023] Eighthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements any of the data transmission methods described above.

[0024] The data transmission method, apparatus, system, device, storage medium, and program product provided in this application introduce a pre-scheduling mechanism and a real-time path reasoning mechanism. The pre-scheduling mechanism refers to the fact that the edge node cluster pre-stores cached paths for transmission. When the edge node cluster receives a data transmission request from a terminal, it can dynamically determine the target path with the optimal transmission latency based on the path detection frequency, the validity period of the cached path, and the network index information of the edge node. This allows the selection of the target path to dynamically adapt to the real-time network changes of the edge node, thereby avoiding the defect in traditional video network data transmission methods where path selection is difficult to adjust dynamically and can only rely on static routing mechanisms. This improves the flexibility of path scheduling, and then transmits the data to be transmitted carried by the data transmission request to the target address based on the target path, which can effectively reduce data transmission latency. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is one of the flowcharts illustrating the data transmission method provided in the embodiments of this application.

[0027] Figure 2 This is one of the structural schematic diagrams of the data transmission system provided in the embodiments of this application.

[0028] Figure 3 This is a flowchart illustrating the intelligent scheduling mechanism provided in the embodiments of this application.

[0029] Figure 4 This is a schematic diagram of the real-time path reasoning process provided in the embodiments of this application.

[0030] Figure 5 This is a flowchart illustrating the pre-scheduling mechanism provided in an embodiment of this application.

[0031] Figure 6 This is the second flowchart illustrating the data transmission method provided in the embodiments of this application.

[0032] Figure 7 This is a schematic diagram of the training process of the path scheduling model provided in the embodiments of this application.

[0033] Figure 8 This is one of the structural schematic diagrams of the data transmission device provided in the embodiments of this application.

[0034] Figure 9This is a second schematic diagram of the data transmission device provided in the embodiments of this application.

[0035] Figure 10 This is the second schematic diagram of the data transmission system provided in the embodiments of this application.

[0036] Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Please see Figures 1 to 3 , Figure 1 This is one of the flowcharts illustrating the data transmission method provided in the embodiments of this application. Figure 2 This is one of the structural schematic diagrams of the data transmission system provided in the embodiments of this application. Figure 3 This is a flowchart illustrating the intelligent scheduling mechanism provided in the embodiments of this application.

[0039] like Figure 2 As shown in the embodiments of this application, the data transmission system for NGVI includes an edge node cluster ( Figure 2 NGVI edge node clusters and cloud nodes (in the middle) Figure 2 (NGVI cloud node in the middle).

[0040] Various video-related terminals ( Figure 2 The NGVI terminal in the system can transmit data with different NGVI business systems through data transmission paths formed by different edge nodes.

[0041] Optionally, NGVI terminals include, but are not limited to, devices such as mobile phones, cameras, projectors, and tablets that can be used to shoot or play audio and video.

[0042] Optionally, the NGVI business system includes, but is not limited to, conferencing systems, monitoring systems, video calling systems, and other business systems that can be used for audio and video interaction.

[0043] The cloud nodes include the NGVI cloud control node and training nodes.

[0044] Optionally, the training node includes a model training module, a feature extraction module, and a result distribution module.

[0045] Optionally, the model training module can traverse all feasible paths of the edge node cluster through a network probing application, collect end-to-end network transmission data in all feasible paths, and use the data to train a machine learning model.

[0046] Optionally, the feature extraction module can analyze the collected data through machine learning models, identify various network metrics, and label the training data.

[0047] Optionally, network metrics include, but are not limited to, bandwidth, latency, jitter, node load, packet loss rate, and out-of-order rate.

[0048] Optionally, the result distribution module can distribute the trained routing data or pre-trained parameters to the transmission path decision system.

[0049] Optionally, the NGVI cloud-based central control node includes a transmission path decision system, a transmission scheduling service system, and a link awareness service system.

[0050] Optionally, the transmission path decision system can generate optimal path recommendations based on path data initially screened by the edge node cluster, pre-trained data, real-time collected network metrics, and other data, combined with an AI large language model (such as DeepSeek-MoE).

[0051] Optionally, the transmission scheduling service system can distribute the optimal path recommendation generated by the transmission path decision system to each edge node in the edge node cluster. For the terminal, the first access edge node used to send data is the originating network node, the target edge node is the destination network node that directly sends the data to the NGVI service system, and the remaining edge nodes used to forward data are relay network nodes. The access edge node can send data according to the optimal path distributed by the transmission scheduling service system, and the relay network node can forward data according to the optimal path distributed by the transmission scheduling service system and the scheduling strategy until the data is sent to the target edge node. Finally, the target edge node sends the data to the NGVI service system.

[0052] Optionally, the link-aware service system can periodically or on demand receive status information and network metrics information of each edge node in the edge node cluster, such as bandwidth utilization, queuing latency, jitter, packet loss rate, load, etc.

[0053] Optionally, the edge node cluster includes an edge-side transmission path decision module, an edge-side link awareness service module, and multiple network nodes. Since the network nodes are deployed on the edge side, they are also called edge nodes.

[0054] Optionally, the edge-side transmission path decision module can deploy a lightweight model to perform preliminary path screening, and then submit the screened path data to the transmission path decision system of the cloud node. The transmission path decision system can use an AI large language model to make refined path recommendations.

[0055] Optionally, the edge-side link awareness service module can collect real-time status information and network metrics information of each edge node in the edge node cluster, such as bandwidth utilization, queuing latency, packet loss rate, jitter rate, and node load.

[0056] Optionally, multiple network nodes are responsible for transmitting data according to the optimal path and scheduling strategy issued by the transmission scheduling service system.

[0057] like Figure 1 As shown in this embodiment, the data transmission method is applied to an edge node cluster in a data transmission system. The data transmission method includes steps S110 to S130, each step of which is as follows: S110: Receive terminal's data transmission request.

[0058] Specifically, if an NGVI terminal needs to transmit data to the NGVI business system, it can generate a data transmission request and send the data transmission request to the edge node cluster. The edge nodes in the edge node cluster can receive the terminal's data transmission request.

[0059] The data transmission request includes the data to be transmitted and the target address.

[0060] S120: Determine the target path based on path detection frequency, path validity period of cached paths, and network indicator information of edge nodes.

[0061] The target path is the data transmission path with the optimal transmission latency.

[0062] Specifically, such as Figure 3 As shown, this application embodiment introduces a pre-scheduling mechanism and a real-time path reasoning (i.e., real-time scheduling) mechanism into the data transmission process of the data transmission system. The pre-scheduling mechanism refers to the fact that all edge nodes in the edge node cluster pre-store cached paths for data transmission. This is because in the actual data transmission process, if the transmission path needs to be regenerated through the large language model of the cloud node every time data transmission is performed, the real-time path reasoning of the large language model will generate reasoning latency, and the transmission path will also generate additional transmission latency from the cloud node to all edge nodes in the edge node cluster. This will inevitably increase the data transmission latency between the terminal and the business system.

[0063] Based on this, the embodiments of this application adopt a combination of pre-scheduling mechanism and real-time path reasoning mechanism. A hierarchical reasoning approach is adopted between cloud nodes and edge node clusters. That is, simple path filtering calculation is set on edge node clusters, and precise real-time path reasoning calculation is set on cloud nodes. Edge node clusters can perform preliminary path detection and filtering at a preset frequency, and cache the recommended paths generated by real-time path reasoning of cloud nodes on each edge node according to a preset strategy as cached paths, and set corresponding path validity periods for cached paths.

[0064] Furthermore, if any edge node in the edge node cluster receives a data transmission request from a terminal, it can determine whether the current cached path is the data transmission path with the optimal transmission latency during this data transmission based on the path detection frequency, the path validity period of the cached path, and the current network indicator information of each edge node.

[0065] If the current cached path is still the data transmission path with the optimal transmission latency, then the current cached path can be directly used as the target path.

[0066] If the current cached path is no longer the optimal data transmission path in terms of transmission latency, the edge node cluster can interact with the cloud node to request the cloud node to perform real-time path reasoning, obtain the target path, update the target path to the cached path, and update the path validity period.

[0067] S130: Based on the target path, transmit the data to be transmitted to the target address.

[0068] This combination of pre-scheduling and real-time path reasoning (i.e., real-time scheduling) avoids the problem of increased data transmission latency caused by frequent real-time path reasoning. At the same time, since the cached paths pre-saved by the edge node cluster are also the paths with the optimal transmission latency obtained from the previous round of real-time path reasoning, and considering that the path with the optimal transmission latency may not change within a certain period of time, even if the cached paths are not updated, using the cached paths as the target paths for data transmission can maintain high transmission efficiency for a period of time, thus achieving a balance between high-precision real-time path reasoning and transmission efficiency.

[0069] The data transmission method provided in this application introduces a pre-scheduling mechanism and a real-time path reasoning mechanism. The pre-scheduling mechanism refers to the edge node cluster pre-saving cached paths for transmission. When the edge node cluster receives a data transmission request from a terminal, it can dynamically determine the target path with the optimal transmission latency based on the path detection frequency, the validity period of the cached path, and the network index information of the edge node. This allows the selection of the target path to dynamically adapt to the real-time network changes of the edge node, thereby avoiding the defect in traditional video network data transmission methods where path selection is difficult to adjust dynamically and can only rely on static routing mechanisms. This improves the flexibility of path scheduling, and then transmits the data to be transmitted carried by the data transmission request to the target address based on the target path, which can effectively reduce data transmission latency.

[0070] In some embodiments, determining a target path based on path detection frequency, the path validity period of a cached path, and network indicator information of edge nodes includes: if the path validity period has expired, collecting network indicator information of edge nodes based on the path detection frequency; determining a target candidate path based on the network indicator information; if the transmission latency of the target candidate path is better than the transmission latency of the cached path, then using the target candidate path as the target path, updating the target path to a cached path, and updating the path validity period.

[0071] Please see Figure 4 , Figure 4 This is a schematic diagram of the real-time path reasoning process provided in the embodiments of this application.

[0072] Specifically, such as Figure 4 As shown, if any edge node in the edge node cluster receives a data transmission request from a terminal, the edge node cluster can send routing request information to the cloud node to inform the cloud node to prepare for routing scheduling and real-time path reasoning.

[0073] Furthermore, the edge node cluster can first determine whether the current cache path's validity period is valid, that is, whether the path validity period has expired.

[0074] If the path validity period has expired, network indicator information of all edge nodes will be collected based on the preset path detection frequency.

[0075] Furthermore, the edge node cluster can interact with the cloud node using network indicator information to trigger the real-time path reasoning process of the cloud node. After the real-time path reasoning process of the cloud node is triggered, it can use a large language model to generate target candidate paths. At the same time, the cloud node can also send a route scheduling request to the edge node cluster to inform the edge node cluster that real-time path reasoning has been performed and route scheduling is ready.

[0076] Furthermore, the cloud node can send the generated target candidate path to the edge node cluster; after receiving the target candidate path, the edge node cluster can compare the target candidate path with the current cached path to determine whether the transmission latency of the target candidate path is better than that of the cached path.

[0077] Optionally, if the transmission latency of the target candidate path is less than the transmission latency of the current cached path, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0078] Preferably, a comparison threshold is determined based on the transmission latency of the current cached path. For example, 90% of the transmission latency of the current cached path is used as the comparison threshold. If the transmission latency of the target candidate path is less than the comparison threshold, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0079] This is because real-time path inference and updating the cached paths of each edge node will generate additional latency. When the transmission latency of the target candidate path is very close to that of the current cached path, the data transmission latency saved by updating the cached path may not be enough to offset the additional latency caused by updating the cached path. Therefore, it is possible to consider the transmission latency of the target candidate path to be better than that of the cached path only when the transmission latency of the target candidate path is much smaller than that of the current cached path.

[0080] Furthermore, if the transmission latency of the target candidate path is better than that of the cached path, then the target candidate path is adopted as the target path, the target path is updated to the cached path, and the path validity period is updated to ensure optimal transmission latency.

[0081] Specifically, each time the cache path is updated, a path validity period (TTL) tag must be added to the updated cache path to indicate the validity period of the cache path.

[0082] Optionally, the path validity period can be determined based on the path detection period and the random disturbance factor, then the path validity period label TTL can be expressed as: .

[0083] Optionally, the random disturbance factor can be dynamically determined based on the resource information of the edge node cluster.

[0084] Optionally, after the edge node cluster updates the target path to the cache path, the updated cache path can be written back to the cloud node to update the global cache of the cloud node.

[0085] In some embodiments, determining a target path based on path detection frequency, the path validity period of a cached path, and network indicator information of edge nodes includes: if the path validity period has not expired, acquiring resource information of the edge node cluster according to a preset period; if the resource information meets the path detection frequency adjustment conditions, updating the path detection frequency based on the resource information; collecting network indicator information of edge nodes based on the updated path detection frequency; determining a target candidate path based on the network indicator information; if the transmission latency of the target candidate path is better than the transmission latency of the cached path, using the target candidate path as the target path, updating the target path to a cached path, and updating the path validity period.

[0086] Please see Figure 5 , Figure 5 This is a flowchart illustrating the pre-scheduling mechanism provided in an embodiment of this application.

[0087] like Figure 2 and Figure 5 As shown, the link awareness service system deployed on the NGVI cloud control node and the edge-side link awareness service module deployed on the edge node cluster can be used to perceive rapid changes in the network status of each edge node in the edge node cluster. Among them, the edge-side link awareness service module can monitor network indicators such as bandwidth utilization, queue length, and load of each edge node and its adjacent nodes. When there are multiple available paths in the edge node cluster, the edge-side link awareness service module can periodically detect the status information of each relay network node and feed these dynamic indicators back to the link awareness service system deployed on the NGVI cloud control node. The link awareness service system will input the received data into the AI ​​large language model at predetermined time intervals to perform real-time path reasoning, obtain the current optimal path, and distribute it to the edge node cluster. The core of the pre-scheduling mechanism lies in the fact that the edge node cluster can dynamically adjust the path probing frequency based on the resource information of the edge nodes, balancing resource consumption and path freshness (i.e., path validity period). By dynamically adjusting the path probing frequency, the edge node cluster can periodically (e.g., using sub-second, second, or minute-level periods) probe the network status of the edge nodes so that the cloud nodes can infer the optimal path and cache the optimal path inferred by the cloud nodes. If the edge node cluster has data transmission needs within the path validity period, it can directly use the pre-cached optimal path, thereby reducing the additional latency caused by real-time path inference of the cloud nodes.

[0088] Understandably, the pre-scheduling mechanism is introduced to minimize the frequency of real-time path inference and reduce the overall computing power consumption of the data transmission system. However, if changes in the resources of the edge node cluster affect the actual data transmission, real-time path inference can also be performed within the path validity period to dynamically adjust the frequency of real-time path inference.

[0089] Specifically, if any edge node in the edge node cluster receives a data transmission request from a terminal, it can first determine whether the current cache path's validity period is valid, that is, whether the path validity period has expired.

[0090] Furthermore, if the path validity period has not expired, resource information of the edge node cluster can be collected according to a preset period (i.e., the path detection period).

[0091] If the resource information meets the conditions for adjusting the path detection frequency, then the path detection frequency is updated based on the resource information.

[0092] Optionally, resource information includes the system load of the edge node cluster.

[0093] If the system load of the edge node cluster is less than the first load threshold, it means that the edge node cluster has enough system resources for path detection. Therefore, the current system load can be considered to meet the path detection frequency adjustment conditions. At this time, the path detection frequency can be increased (for example, the path detection frequency can be set to detect once per second).

[0094] Similarly, if the system load of the edge node cluster is greater than or equal to the second load threshold, it means that the system resources of the edge node cluster are insufficient to support frequent path probing. It can also be considered that the current system load meets the conditions for adjusting the path probing frequency. In this case, the path probing frequency can be reduced (for example, the path probing frequency can be set to once per minute).

[0095] The second load threshold is greater than or equal to the first load threshold.

[0096] Optionally, the first load threshold and the second load threshold can be adjusted according to the actual situation, for example, both the first load threshold and the second load threshold can be 70%.

[0097] Optionally, the resource information includes network stability information of the edge node cluster, and the network stability information includes network jitter rate.

[0098] If the standard deviation of the network jitter rate of the edge node cluster is greater than the jitter rate threshold multiple times (e.g., 3 times), it indicates that the current network environment of the edge node cluster is unstable. The existing cached paths may become invalid due to the unstable network environment. Path probing needs to be performed frequently to ensure that the edge node cluster can obtain the latest available data transmission paths. Therefore, it can be considered that the current network jitter rate meets the conditions for adjusting the path probing frequency. At this time, the path probing frequency can be temporarily increased (e.g., the path probing frequency can be set to sub-second level).

[0099] Optionally, the jitter rate threshold can be adjusted according to the actual situation, for example, the jitter rate threshold is 15%.

[0100] Furthermore, based on the updated path detection frequency, the preset period (i.e., the path detection period) is adjusted to collect the current network indicator information of the edge nodes.

[0101] Furthermore, the edge node cluster can interact with the cloud node using network indicator information to trigger the real-time path reasoning process of the cloud node; after the real-time path reasoning process of the cloud node is triggered, it can use a large language model to generate target candidate paths.

[0102] Furthermore, the cloud node can send the generated target candidate path to the edge node cluster; after receiving the target candidate path, the edge node cluster can compare the target candidate path with the current cached path to determine whether the transmission latency of the target candidate path is better than that of the cached path.

[0103] Optionally, if the transmission latency of the target candidate path is less than the transmission latency of the current cached path, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0104] Preferably, a comparison threshold is determined based on the transmission latency of the current cached path. For example, 90% of the transmission latency of the current cached path is used as the comparison threshold. If the transmission latency of the target candidate path is less than the comparison threshold, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0105] Furthermore, if the transmission latency of the target candidate path is better than that of the cached path, then the target candidate path is adopted as the target path, the target path is updated to the cached path, and the path validity period is updated to ensure optimal transmission latency.

[0106] The data transmission method provided in this application, by introducing a pre-scheduling and caching mechanism, can reduce the impact of additional latency caused by frequent real-time path inference by cloud nodes while ensuring the accuracy of the target path.

[0107] In some embodiments, determining a target candidate path based on network indicator information includes: inputting network indicator information into an edge scheduling model to perform path filtering and obtain a candidate path set output by the edge scheduling model; the candidate path set includes at least two candidate paths; sending the candidate path set and network indicator information to a cloud node; using the candidate path set and network indicator information together in an intelligent scheduling model to perform real-time path reasoning and generate a target candidate path; receiving the target candidate path sent by the cloud node; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set.

[0108] like Figure 2As shown, the NGVI cloud control node deploys an AI large language model as the core inference engine for real-time path reasoning. The edge node cluster can collect network indicator information from the edge nodes and execute scheduling strategies issued by the cloud nodes. However, in real-time decision-making scenarios, the AI ​​large language model is difficult to call frequently due to its high inference overhead. Therefore, this embodiment adopts a cloud-edge collaboration and hierarchical inference mechanism, that is, placing the computationally intensive real-time path reasoning task on the cloud node, while the edge node cluster only performs lightweight path prediction and data collection, thereby ensuring the efficiency of cloud-edge collaborative path reasoning and adapting to the distributed edge node scenario of NGVI. By deploying a lightweight model in the edge node cluster for rapid prediction or path filtering, the range of candidate paths can be effectively narrowed, reducing the number of candidate paths that need to be submitted to the cloud AI large language model for deep inference, thereby reducing the overall response latency. This allows the cloud node to call the AI ​​large language model to perform more refined calculations and evaluations on the few candidate paths that are most likely to become the optimal path, improving the efficiency of real-time path reasoning on the cloud node.

[0109] This cloud-edge collaborative hybrid scheduling method combines the stability and interpretability of lightweight machine learning models at the edge with the high-complexity learning capabilities of large language models in cloud-based AI. It has strong application value in the field of network path scheduling and optimization, and can help data transmission systems achieve more accurate and real-time optimal path recommendations, thereby effectively reducing end-to-end latency in the edge data transmission process and improving overall network performance.

[0110] Specifically, after the edge node cluster collects the current network indicator information of all edge nodes, the network indicator information can be input into the pre-trained edge scheduling model to perform path filtering and obtain the candidate path set output by the edge scheduling model.

[0111] Optionally, network metrics information includes, but is not limited to, bandwidth utilization, current queuing latency, network packet loss rate, network jitter rate, and node load of each edge node; the edge node cluster can construct a simplified network status diagram on the edge side based on these network metrics information to quickly reflect the local status of the current edge side network environment.

[0112] It should be noted that the edge scheduling model deployed in the edge node cluster is a simplified or specially trained lightweight model. The edge scheduling model can be a trimmed XGBoost lightweight model, the purpose of which is to achieve fast inference on devices with limited computing resources.

[0113] The edge scheduling model can be trained based on the pre-trained parameters of the AI ​​large language model distributed from the cloud node, or it can be trained using offline data from each edge node.

[0114] Specifically, the edge scheduling model can perform fast path filtering based on the network indicator information of each edge node currently collected, and generate an initial candidate path set, which includes at least two initial candidate paths.

[0115] Furthermore, the edge scheduling model can calculate a preliminary score for each initial candidate path, and filter out initial candidate paths that obviously do not meet the requirements according to a pre-set scoring threshold or sorting rule, retaining only a few initial candidate paths with higher preliminary scores (e.g., only retaining the top 3 or top 5 initial candidate paths with preliminary scores), generating a candidate path set, which includes at least two candidate paths.

[0116] Furthermore, the edge node cluster can send the candidate path set and network indicator information to the cloud node; after receiving the candidate path set and network indicator information, the cloud node can call the AI ​​large language model.

[0117] To ensure inference accuracy, the AI ​​large language model here is not a general large language model, but a pre-trained intelligent scheduling model. That is, the intelligent scheduling model is a large language model that integrates a deep neural network and a pre-trained path scheduling model. Because the intelligent scheduling model integrates a pre-trained path scheduling model, it has efficient real-time path inference capabilities.

[0118] The intelligent scheduling model can perform real-time path reasoning based on the candidate path set and network indicator information to generate target candidate paths. The target candidate path is the candidate path with the optimal transmission latency in the candidate path set.

[0119] Furthermore, the cloud node can send the target candidate path to the edge node cluster; after receiving the target candidate path sent by the cloud node, the edge node cluster can compare the target candidate path with the current cached path to determine whether the transmission latency of the target candidate path is better than that of the cached path.

[0120] Optionally, if the transmission latency of the target candidate path is less than the transmission latency of the current cached path, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0121] Preferably, a comparison threshold is determined based on the transmission latency of the current cached path. For example, 90% of the transmission latency of the current cached path is used as the comparison threshold. If the transmission latency of the target candidate path is less than the comparison threshold, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0122] Furthermore, if the transmission latency of the target candidate path is better than that of the cached path, then the target candidate path is adopted as the target path, the target path is updated to the cached path, and the path validity period is updated to ensure optimal transmission latency.

[0123] Optionally, the edge scheduling model can also periodically receive updated pre-trained parameters or feedback data from the intelligent scheduling model at the cloud node, and perform iterative training based on this data. This allows the model to continuously align with the judgment criteria of the cloud-based intelligent scheduling model when performing rapid path selection, ensuring the accuracy of the initial path selection. This training strategy not only guarantees the real-time performance and accuracy of the initial path selection at the edge, but also facilitates the continuous optimization of the global scheduling strategy by the edge based on the cloud-based intelligent scheduling model.

[0124] The data transmission method provided in this application selects multiple edge nodes to form the optimal target path through a cloud-edge collaborative intelligent scheduling method, which can effectively reduce network transmission latency between the terminal and the business system. At the same time, it introduces a system mode that separates edge-side transmission and cloud-side control. The cloud node is responsible for fine-grained real-time path inference and routing scheduling decisions, while the edge side is only responsible for executing the scheduling decisions of the cloud node, which can achieve efficient cloud-edge collaboration. In addition, the introduction of a combination of cloud-based large language model inference and pre-scheduling can effectively reduce the frequency of cloud-based large language model calls, which is conducive to reducing the additional overhead caused by real-time path inference, thereby further reducing data transmission latency.

[0125] In some embodiments, the edge scheduling model is obtained by pre-training the first initial model based on the pre-trained parameters of the intelligent scheduling model, and the first initial model is obtained by pruning the XGBoost model.

[0126] The edge scheduling model deployed in an edge node cluster is a simplified or specially trained lightweight model. The edge scheduling model can be a trimmed XGBoost lightweight model, which aims to achieve fast inference on computing-constrained devices.

[0127] Among them, the edge scheduling model is obtained by pre-training the first initial model based on the pre-trained parameters of the intelligent scheduling model issued by the cloud node. This training method enables the edge scheduling model to closely match the judgment criteria of the cloud intelligent scheduling model when performing fast path selection, thus ensuring the accuracy of the initial path selection.

[0128] In some embodiments, determining the target path based on the path detection frequency, the path validity period of the cached path, and the network indicator information of the edge nodes further includes: if the path validity period has not expired, obtaining the resource information of the edge node cluster according to a preset period; if the resource information does not meet the path detection frequency adjustment conditions, then using the cached path as the target path.

[0129] Specifically, if any edge node in the edge node cluster receives a data transmission request from a terminal, it can first determine whether the current cache path's validity period is valid, that is, whether the path validity period has expired.

[0130] If the path validity period has not expired, the resource information of the edge node cluster will be obtained according to the preset period.

[0131] If the resource information does not meet the conditions for adjusting the path detection frequency, the directly cached path can be used as the target path, thereby reducing the frequency of real-time path inference and reducing the overall computing power consumption of the data transmission system.

[0132] This application also provides a data transmission method. Please refer to the embodiments therein. Figure 6 , Figure 6 This is a second schematic flowchart of the data transmission method provided in the embodiments of this application. Figure 6 As shown in this embodiment, the data transmission method is applied to a cloud node in the data transmission system. The data transmission method includes steps S610 to S630, each of which is detailed below: S610: Receives network metric information for candidate path sets and edge nodes.

[0133] The candidate path set includes at least two candidate paths, which are determined by the edge node cluster based on path detection frequency, cached path validity period, and network metric information.

[0134] like Figure 2 As shown, the NGVI cloud control node deploys an AI large language model as the core inference engine for real-time path reasoning. The edge node cluster can collect network indicator information from the edge nodes and execute scheduling strategies issued by the cloud nodes. However, in real-time decision-making scenarios, the AI ​​large language model is difficult to call frequently due to its high inference overhead. Therefore, this embodiment adopts a cloud-edge collaboration and hierarchical inference mechanism, that is, placing the computationally intensive real-time path reasoning task on the cloud node, while the edge node cluster only performs lightweight path prediction and data collection, thereby ensuring the efficiency of cloud-edge collaborative path reasoning and adapting to the distributed edge node scenario of NGVI. By deploying a lightweight model in the edge node cluster for rapid prediction or path filtering, the range of candidate paths can be effectively narrowed, reducing the number of candidate paths that need to be submitted to the cloud AI large language model for deep inference, thereby reducing the overall response latency. This allows the cloud node to call the AI ​​large language model to perform more refined calculations and evaluations on the few candidate paths that are most likely to become the optimal path, improving the efficiency of real-time path reasoning on the cloud node.

[0135] This cloud-edge collaborative hybrid scheduling method combines the stability and interpretability of lightweight machine learning models at the edge with the high-complexity learning capabilities of large language models in cloud-based AI. It has strong application value in the field of network path scheduling and optimization, and can help data transmission systems achieve more accurate and real-time optimal path recommendations, thereby effectively reducing end-to-end latency in the edge data transmission process and improving overall network performance.

[0136] Specifically, after the edge node cluster collects the current network indicator information of all edge nodes, the network indicator information can be input into the pre-trained edge scheduling model to perform path filtering and obtain the candidate path set output by the edge scheduling model. The candidate path set includes at least two candidate paths.

[0137] Furthermore, the edge node cluster can send candidate path sets and network metric information to the cloud nodes.

[0138] S620: Input the candidate path set and network indicator information into the intelligent scheduling model to perform real-time path reasoning and obtain the target candidate path output by the intelligent scheduling model.

[0139] The target candidate path is the candidate path with the optimal transmission latency among the candidate paths.

[0140] Specifically, after receiving the candidate path set and network indicator information, the cloud node can invoke the intelligent scheduling model.

[0141] Among them, the intelligent scheduling model is a large language model that integrates a deep neural network and a pre-trained path scheduling model. Because the intelligent scheduling model integrates a pre-trained path scheduling model, it has efficient real-time path reasoning capabilities.

[0142] Furthermore, the candidate path set and network indicator information are input into the intelligent scheduling model. The intelligent scheduling model can perform real-time path reasoning based on the candidate path set and network indicator information to generate target candidate paths. The target candidate path is the candidate path with the optimal transmission latency in the candidate path set.

[0143] S630: Sends the target candidate path to the edge node cluster.

[0144] The target candidate path is used to update the cache path of the edge node cluster, and the cache path is used to transmit the data to be transmitted by the terminal.

[0145] Specifically, cloud nodes can send target candidate paths to the edge node cluster.

[0146] Furthermore, after receiving the target candidate path sent by the cloud node, the edge node cluster can compare the target candidate path with the current cached path to determine whether the transmission latency of the target candidate path is better than that of the cached path.

[0147] Optionally, if the transmission latency of the target candidate path is less than the transmission latency of the current cached path, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0148] Preferably, a comparison threshold is determined based on the transmission latency of the current cached path. For example, 90% of the transmission latency of the current cached path is used as the comparison threshold. If the transmission latency of the target candidate path is less than the comparison threshold, then the transmission latency of the target candidate path can be considered to be better than the transmission latency of the cached path.

[0149] Furthermore, if the transmission latency of the target candidate path is better than that of the cached path, the edge node cluster can use the target candidate path as the target path, update the target path to the cached path, and update the path validity period to ensure optimal transmission latency.

[0150] Furthermore, the edge node cluster can transmit data to the target address based on the target path.

[0151] The data transmission method provided in this application, when real-time path reasoning is required, firstly, the edge node cluster collects network indicator information of the edge nodes and generates a candidate path set. Then, the candidate path set and the network indicator information of the edge nodes are sent to the cloud node. The cloud node then uses an intelligent scheduling model to perform real-time path reasoning based on the candidate path set and the network indicator information of the edge nodes, selects the candidate path with the optimal transmission latency as the target candidate path, and sends the target candidate path to the edge node cluster. This allows the edge node cluster to update paths and transmit data based on the real-time path reasoning results, avoiding the problem of increased data transmission latency caused by frequent real-time path reasoning.

[0152] In some embodiments, the intelligent scheduling model is a model built on deep neural networks and a pre-trained path scheduling model.

[0153] The path scheduling model is trained based on the following steps: The second initial model is pre-trained based on sample feasible paths, sample network indicator information corresponding to the sample feasible paths, and sample target variables corresponding to the sample feasible paths and sample network indicator information to obtain the initial path scheduling model; the sample network indicator information includes multiple sample network indicators, and the sample target variables include sample transmission latency, sample quality of service indicators, and sample path classification labels; the initial path scheduling model is pre-trained to obtain the path scheduling model.

[0154] Please see Figure 7 , Figure 7 This is a schematic diagram of the training process of the path scheduling model provided in the embodiments of this application.

[0155] Specifically, such as Figure 7 As shown, a suitable machine learning algorithm should be selected before training the path scheduling model.

[0156] In this embodiment, the second initial model is constructed based on the GBDT (Gradient Boosting Decision Tree) algorithm. This is because the GBDT algorithm is suitable for numerical features and relatively structured network metrics, such as bandwidth, latency, and load. It can also achieve stable performance when the sample size is not extremely large. It is robust to missing values ​​and outliers of network metrics. In addition, the training speed of the GBDT algorithm is relatively controllable, and the model has good interpretability, which is beneficial for subsequent analysis of the main factors affecting latency.

[0157] It should be noted that the GBDT algorithm can be implemented using various model frameworks. The algorithm principles of different model frameworks are similar, but different model frameworks have different training efficiencies and parameter tuning methods. Therefore, you can choose the appropriate framework based on the scale and requirements of the training data.

[0158] Preferably, XGBoost is selected as the model framework for the GBDT algorithm, that is, the second initial model is also an XGBoost model.

[0159] Specifically, network probing applications can be deployed in edge node clusters to collect sample feasible paths and sample network indicator information corresponding to the sample feasible paths in the edge node clusters.

[0160] During the training phase, the input data for the second initial model consists of feasible sample paths and sample network indicator information corresponding to the feasible sample paths. The sample network indicator information includes multiple sample network indicators.

[0161] However, the output data of the second initial model may differ depending on the path scheduling scenario.

[0162] Path scheduling scenarios include regression scenarios and classification scenarios. Classification scenarios can be further divided into single-classification scenarios and multi-classification scenarios.

[0163] For regression scenarios, the model is mainly used to predict relevant evaluation indicators. Therefore, during the training phase, the output data of the second initial model are sample transmission delay (numerical data), sample service quality indicators, etc. The true values ​​of sample transmission delay and sample service quality indicators can be used as sample labels.

[0164] For classification scenarios, the model is mainly used for path classification and filtering. Therefore, during the training phase, the output data of the second initial model is the sample path classification label.

[0165] Optionally, for single-class scenarios, the sample path classification label is used to identify whether the sample feasible path is a "low-latency path" ("0" can be used to indicate that the sample feasible path is not a low-latency path, and "1" can be used to indicate that the sample feasible path is a low-latency path).

[0166] Optionally, for multi-class scenarios, there are multiple feasible paths for a sample. In this case, the sample path classification label is used to identify whether the sample feasible path is the "optimal path" among the multiple paths ("0" can be used to indicate that the sample feasible path is not the optimal path, and "1" can be used to indicate that the sample feasible path is the optimal path).

[0167] Therefore, after determining the feasible paths of the samples and the corresponding sample network indicators, it is also necessary to label them according to the path scheduling scenario, and determine the sample target variables corresponding to the feasible paths and sample network indicators. The sample target variables include sample transmission latency, sample service quality indicators, and sample path classification labels.

[0168] Furthermore, based on the sample feasible path, the sample network index information corresponding to the sample feasible path, and the sample target variable corresponding to the sample feasible path and the sample network index information, the second initial model is pre-trained to obtain the initial path scheduling model.

[0169] Optionally, the labeled training data can be divided into training and validation sets for pre-training according to a certain ratio (e.g., 8:2 or 7:3).

[0170] Optionally, if the amount of training data with completed labeling is large enough, the test set can be further divided or cross-validation can be used to evaluate the training effect of the initial path scheduling model.

[0171] Optionally, if the second initial model is an XGBoost model, its learning rate can be set between 0.01 and 0.1 during the training phase. Too large a learning rate can easily lead to model overfitting, while too small a learning rate can easily lead to slow training speed. At the same time, the decision tree depth of the second initial model should be set reasonably to control model complexity, and the subsampling rate and feature sampling rate should be set reasonably to prevent model overfitting to a certain extent. In addition, the number of decision trees and the number of iterations of the second initial model should be adjusted in conjunction with the learning rate to ensure model training performance.

[0172] Optionally, during training, the model can iteratively generate new decision trees on the training set to fit the residuals or gradients. At this time, the training performance of the model can be monitored according to the validation set, and relevant parameters can be adjusted in a timely manner to obtain better training results.

[0173] In some embodiments, pre-training an initial path scheduling model to obtain a path scheduling model includes: performing feature importance analysis on multiple sample network indicators based on the initial path scheduling model to determine at least one key indicator; the key indicator is a sample network indicator whose feature importance is higher than a preset threshold, and the feature importance is used to characterize the predictive contribution to the target variable of the sample; performing feature optimization processing on the key indicator to generate optimized network indicators; and pre-training the initial path scheduling model based on the optimized network indicators to obtain a path scheduling model.

[0174] Understandably, since the sample network metrics information includes multiple sample network metrics during pre-training, but not all sample network metrics may be helpful for model prediction, after obtaining the initial path scheduling model, feature importance analysis can be performed on multiple sample network metrics to determine at least one key metric, and the initial path scheduling model can be pre-trained again using the key metric to obtain a more accurate path scheduling model.

[0175] For models built based on the GBDT algorithm, feature importance analysis can be performed by implementing the corresponding model framework.

[0176] The feature importance analysis principle of the GBDT algorithm is to integrate multiple decision trees. The splitting of nodes in each decision tree brings indices such as "information gain" or "loss reduction." Each node in a decision tree represents an input sample feature. If an input sample feature splits frequently in the decision tree and generates a large information gain, it indicates a high contribution to the prediction of the sample's output variable. Based on this principle, GBDT algorithm model frameworks typically provide external interfaces to support the export of sample features that contribute significantly to the model's predictions during training, for user analysis.

[0177] Therefore, after training the initial path scheduling model, feature importance analysis can be performed on multiple sample network indicators through the initial path scheduling model, and the feature importance corresponding to each sample network indicator can be derived through the external interface provided by the initial path scheduling model.

[0178] Furthermore, based on the feature importance corresponding to each sample network indicator, at least one key indicator is selected. The key indicator is the sample network indicator whose feature importance is higher than a preset threshold. Feature importance is used to characterize the predictive contribution to the target variable of the sample.

[0179] For example, the network metrics of the top 10 or top 20 samples in terms of feature importance can be selected as key metrics.

[0180] Preferably, after obtaining all key indicators, cross-validation of all key indicators can be performed by combining relevant domain knowledge to verify that all key indicators are related to the model prediction.

[0181] Optionally, for sample network metrics that have low importance and no obvious business significance, it may be considered to remove them or perform feature dimensionality reduction, thereby reducing the model's focus on these sample network metrics and improving training and prediction efficiency.

[0182] Furthermore, for all key metrics, feature optimization processing can be performed during subsequent training data collection and annotation to generate optimized network metrics.

[0183] Specifically, all key indicators are labeled and refined to obtain detailed indicators.

[0184] Optionally, increase the sampling frequency or precision of all key indicators to ensure the accuracy of the key indicator data.

[0185] Optionally, relevant features can be added to all key indicators, such as the mean, peak value, and variance of the key indicators, to improve the model's learning performance on the key indicators.

[0186] Optionally, the label definitions corresponding to key indicators can be improved: if the labels corresponding to key indicators are too simple, the labels can be re-labeled based on feature importance.

[0187] After the above label refinement process, the refined label indicators corresponding to all key indicators can be obtained. At this point, further data cleaning and feature optimization processing can be performed on the refined label indicators.

[0188] Specifically, based on the feature importance corresponding to all refined label indicators, data cleaning is performed again on all refined label indicators to remove invalid features.

[0189] It should be noted that invalid features here refer to features with very low importance ranking. Removing these invalid features and retaining only the refined label indicators that are strongly correlated with the prediction can reduce the complexity of model training.

[0190] Furthermore, the remaining refined label indicators that are strongly correlated with the prediction are subjected to feature optimization processing. The refined label indicators with the highest feature importance are combined to form new optimized network indicators.

[0191] For example, if the refined label metrics that are strongly correlated with prediction include sample bandwidth and sample queuing delay, then sample bandwidth and sample queuing delay can be combined as network optimization metrics.

[0192] Furthermore, based on optimized network metrics, the initial path scheduling model is pre-trained to obtain the path scheduling model.

[0193] Specifically, the optimized network metrics are re-inputted into the initial path scheduling model for pre-training, and the performance changes of the initial path scheduling model on the validation or test set are observed.

[0194] If the accuracy, AUC, F1, and other performance metrics of the initial path scheduling model are improved, it indicates that the feature optimization process is effective.

[0195] If the performance metrics do not improve, the business logic and data quality need to be checked again, the features need to be optimized again, new optimized network metrics need to be generated, and the initial path scheduling model needs to be pre-trained based on the new optimized network metrics.

[0196] The process involves repeatedly generating and optimizing network metrics and pre-training the model until the initial path scheduling model achieves the expected performance, thus obtaining the path scheduling model.

[0197] In some embodiments, the sample feasible path, the sample network indicator information corresponding to the sample feasible path, and the sample target variable corresponding to the sample feasible path and the sample network indicator information are obtained based on the following steps: data preprocessing is performed on the initial sample feasible path and the initial sample network indicator information corresponding to the initial sample feasible path to obtain the sample feasible path and the sample network indicator information corresponding to the sample feasible path; the edge node cluster includes multiple edge nodes, the initial sample feasible path is the data transmission path between the access edge node and the target edge node, and the access edge node and the target edge node are any edge node in the edge node cluster; the sample target variable corresponding to the sample feasible path and the sample network indicator information is determined according to the path scheduling scenario.

[0198] Specifically, a network probing application can be deployed in the edge node cluster to traverse and test all feasible paths between different access edge nodes and the target edge node. At the same time, relevant network indicator data can be collected through the log system to obtain all initial sample feasible paths and the initial sample network indicator information corresponding to each initial sample feasible path.

[0199] Furthermore, data preprocessing is performed on all feasible paths of the initial samples and the initial sample network index information corresponding to each feasible path of the initial samples to obtain all feasible paths of the samples and the sample network index information corresponding to each feasible path of the samples.

[0200] Optionally, data preprocessing includes deduplication, missing value removal, and outlier detection to ensure data quality.

[0201] Furthermore, based on the path scheduling scenario, the feasible path for each sample and the target variable corresponding to the network indicator information for each sample are determined.

[0202] The data transmission method provided in this application fully utilizes network probing applications to obtain sufficient training data and combines machine learning algorithms to model key sample network indicators, enabling the model to have good sample learning capabilities, which is beneficial for subsequent accurate prediction and selection of the optimal data transmission path.

[0203] In some embodiments, the sample network metrics include at least two of the following: sample bandwidth, sample bandwidth utilization, sample queuing latency, sample network jitter rate, sample edge node load, sample network packet loss rate, and sample packet out-of-order rate.

[0204] In some embodiments, before receiving the candidate path set and the network indicator information of the edge nodes, the method further includes: sending the pre-training parameters of the intelligent scheduling model to the edge node cluster; the pre-training parameters are used to train the edge scheduling model of the edge node cluster, and the edge scheduling model is used to generate the candidate path set.

[0205] With the development of artificial intelligence technology, large language models (such as DeepSeek) have achieved remarkable results in fields such as image recognition, natural language processing, and recommendation systems, and are therefore gradually being applied to network path scheduling and optimization. Although general-purpose large language models have good natural language processing capabilities, their processing performance is not ideal when directly applied to specialized fields such as path reasoning.

[0206] Based on this, after obtaining the pre-trained path scheduling model, the path scheduling model and other deep neural networks can be integrated into a general large language model to form an intelligent scheduling model. This allows the intelligent scheduling model to have good path reasoning ability while maintaining its original natural language processing capabilities.

[0207] For intelligent scheduling models, during real-time path reasoning, the prediction results of the integrated path scheduling model can be used as some features of the intelligent scheduling model to guide the path reasoning of the intelligent scheduling model. This can improve the accuracy and reasoning speed of the intelligent scheduling model in network scheduling, and achieve more accurate latency prediction and path planning.

[0208] Furthermore, the intelligent scheduling model is deployed on the NGVI cloud central control node to enable intelligent scheduling of network paths on the edge side.

[0209] Understandably, before using the intelligent scheduling model for intelligent scheduling, the cloud nodes need to send the pre-trained parameters of the intelligent scheduling model to the edge node cluster, so that the edge node cluster can use these pre-trained parameters to train the edge scheduling model and generate a candidate path set.

[0210] Since the edge scheduling model is trained based on the pre-trained parameters of the intelligent scheduling model, it can closely match the judgment criteria of the cloud-based intelligent scheduling model when performing fast path selection, thus ensuring the accuracy of the candidate path set.

[0211] This application also provides a data transmission device. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is one of the structural schematic diagrams of the data transmission device provided in the embodiments of this application. In this embodiment, the data transmission device is deployed in an edge node cluster, which includes multiple edge nodes. The data transmission device includes a first receiving module 810, a path determination module 820, and a data transmission module 830.

[0212] The first receiving module 810 is used to receive data transmission requests from the terminal.

[0213] The data transmission request carries the data to be transmitted and the destination address.

[0214] The path determination module 820 is used to determine the target path based on the path detection frequency, the path validity period of the cached path, and the network indicator information of the edge nodes.

[0215] The target path is the data transmission path with the optimal transmission latency.

[0216] The data transmission module 830 is used to transmit data to the target address based on the target path.

[0217] In some embodiments, the path determination module 820 is used to collect network indicator information of edge nodes based on the path detection frequency if the path validity period has expired; determine a target candidate path based on the network indicator information; if the transmission latency of the target candidate path is better than the transmission latency of the cached path, then use the target candidate path as the target path, update the target path to the cached path, and update the path validity period.

[0218] In some embodiments, the path determination module 820 is configured to: if the path validity period has not expired, acquire resource information of the edge node cluster according to a preset period; if the resource information meets the path detection frequency adjustment conditions, update the path detection frequency based on the resource information; collect network indicator information of the edge nodes based on the updated path detection frequency; determine the target candidate path based on the network indicator information; if the transmission latency of the target candidate path is better than the transmission latency of the cached path, use the target candidate path as the target path, update the target path to the cached path, and update the path validity period.

[0219] In some embodiments, the path determination module 820 is used to input network indicator information into the edge scheduling model, perform path filtering, and obtain a candidate path set output by the edge scheduling model; the candidate path set includes at least two candidate paths; send the candidate path set and network indicator information to the cloud node; the candidate path set and network indicator information are used together by the intelligent scheduling model to perform real-time path reasoning and generate a target candidate path; receive the target candidate path sent by the cloud node; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set.

[0220] In some embodiments, the edge scheduling model is obtained by pre-training the first initial model based on the pre-trained parameters of the intelligent scheduling model, and the first initial model is obtained by pruning the XGBoost model.

[0221] In some embodiments, the path determination module 820 is used to obtain the resource information of the edge node cluster according to a preset period if the path validity period has not expired; if the resource information does not meet the path detection frequency adjustment conditions, the cached path is used as the target path.

[0222] This application also provides a data transmission device. Please refer to [link to relevant documentation]. Figure 9 , Figure 9 This is a second schematic diagram of the data transmission device provided in this application embodiment. In this application embodiment, the data transmission device is deployed on a cloud node, and the data transmission device includes a second receiving module 910, a path filtering module 920, and a data sending module 930.

[0223] The second receiving module 910 is used to receive candidate path sets and network indicator information of edge nodes.

[0224] The candidate path set includes at least two candidate paths, which are determined by the edge node cluster based on path detection frequency, cached path validity period, and network metric information.

[0225] The path filtering module 920 is used to input the candidate path set and network indicator information into the intelligent scheduling model to perform real-time path reasoning and obtain the target candidate path output by the intelligent scheduling model.

[0226] The target candidate path is the candidate path with the optimal transmission latency among the candidate paths.

[0227] The data sending module 930 is used to send the target candidate path to the edge node cluster.

[0228] The target candidate path is used to update the cache path of the edge node cluster, and the cache path is used to transmit the data to be transmitted by the terminal.

[0229] In some embodiments, the intelligent scheduling model is a model constructed based on a deep neural network and a pre-trained path scheduling model. The path scheduling model is trained based on the following steps: pre-training a second initial model based on sample feasible paths, sample network indicator information corresponding to sample feasible paths, and sample target variables corresponding to sample feasible paths and sample network indicator information to obtain an initial path scheduling model; the sample network indicator information includes multiple sample network indicators, and the sample target variables include sample transmission delay, sample service quality indicators, and sample path classification labels; pre-training the initial path scheduling model to obtain the path scheduling model.

[0230] In some embodiments, pre-training an initial path scheduling model to obtain a path scheduling model includes: performing feature importance analysis on multiple sample network indicators based on the initial path scheduling model to determine at least one key indicator; the key indicator is a sample network indicator whose feature importance is higher than a preset threshold, and the feature importance is used to characterize the predictive contribution to the target variable of the sample; performing feature optimization processing on the key indicator to generate optimized network indicators; and pre-training the initial path scheduling model based on the optimized network indicators to obtain a path scheduling model.

[0231] In some embodiments, the sample feasible path, the sample network indicator information corresponding to the sample feasible path, and the sample target variable corresponding to the sample feasible path and the sample network indicator information are obtained based on the following steps: data preprocessing is performed on the initial sample feasible path and the initial sample network indicator information corresponding to the initial sample feasible path to obtain the sample feasible path and the sample network indicator information corresponding to the sample feasible path; the edge node cluster includes multiple edge nodes, the initial sample feasible path is the data transmission path between the access edge node and the target edge node, and the access edge node and the target edge node are any edge node in the edge node cluster; the sample target variable corresponding to the sample feasible path and the sample network indicator information is determined according to the path scheduling scenario.

[0232] In some embodiments, the data sending module 930 is used to send the pre-training parameters of the intelligent scheduling model to the edge node cluster; the pre-training parameters are used to train the edge scheduling model of the edge node cluster, and the edge scheduling model is used to generate a candidate path set.

[0233] In some embodiments, the sample network metrics include at least two of the following: sample bandwidth, sample bandwidth utilization, sample queuing latency, sample network jitter rate, sample edge node load, sample network packet loss rate, and sample packet out-of-order rate.

[0234] This application also provides a data transmission system. Please refer to [link / reference]. Figure 10 , Figure 10 This is a second schematic diagram of the data transmission system provided in this application embodiment. In this application embodiment, the data transmission system includes an edge node cluster 1010 and a cloud node 1020. The edge node cluster 1010 executes any of the above-mentioned data transmission methods applied to the edge node cluster, and the cloud node executes any of the above-mentioned data transmission methods applied to the cloud node.

[0235] This application also provides an electronic device. Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 11 As shown, the electronic device may include a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other through the communication bus 1140. The processor 1110 can call logical instructions in the memory 1130 to execute data transmission methods.

[0236] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0237] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the data transmission methods provided by the above methods.

[0238] This application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the data transmission methods provided by the above methods.

[0239] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0240] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0241] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data transmission method, characterized in that, Applied to an edge node cluster, wherein the edge node cluster includes edge nodes, the data transmission method includes: The receiving terminal sends a data transmission request; the data transmission request carries the data to be transmitted and the target address. Based on the path detection frequency, the path validity period of the cached path, and the network indicator information of the edge node, the target path is determined; the target path is the data transmission path with the optimal transmission latency. Based on the target path, the data to be transmitted is transmitted to the target address.

2. The data transmission method according to claim 1, characterized in that, The determination of the target path based on path detection frequency, path validity period of cached paths, and network indicator information of the edge nodes includes: If the path validity period has expired, then based on the path detection frequency, the network indicator information of the edge node is collected; Based on the network indicator information, the target candidate path is determined; If the transmission latency of the target candidate path is better than the transmission latency of the cached path, then the target candidate path is adopted as the target path, the target path is updated to the cached path, and the validity period of the path is updated.

3. The data transmission method according to claim 1, characterized in that, The determination of the target path based on path detection frequency, path validity period of cached paths, and network indicator information of the edge nodes includes: If the path validity period has not expired, the resource information of the edge node cluster is obtained according to the preset period. If the resource information meets the path detection frequency adjustment conditions, then the path detection frequency is updated based on the resource information; Based on the updated path detection frequency, the network indicator information of the edge nodes is collected; Based on the network indicator information, the target candidate path is determined; If the transmission latency of the target candidate path is better than the transmission latency of the cached path, then the target candidate path is adopted as the target path, the target path is updated to the cached path, and the validity period of the path is updated.

4. The data transmission method according to claim 2 or 3, characterized in that, The step of determining the target candidate path based on the network indicator information includes: The network metric information is input into the edge scheduling model to perform path filtering, thereby obtaining a candidate path set output by the edge scheduling model; the candidate path set includes at least two candidate paths. The candidate path set and the network indicator information are sent to the cloud node; the candidate path set and the network indicator information are used together by the intelligent scheduling model to perform real-time path reasoning and generate target candidate paths; Receive the target candidate path sent by the cloud node; the target candidate path is the candidate path with the optimal transmission latency among the candidate paths.

5. The data transmission method according to claim 4, characterized in that, The edge scheduling model is obtained by pre-training the first initial model based on the pre-trained parameters of the intelligent scheduling model. The first initial model is obtained by pruning the XGBoost model.

6. The data transmission method according to claim 1, characterized in that, The determination of the target path based on path detection frequency, path validity period of cached paths, and network indicator information of the edge nodes also includes: If the path validity period has not expired, the resource information of the edge node cluster is obtained according to the preset period. If the resource information does not meet the path detection frequency adjustment conditions, then the cached path will be used as the target path.

7. A data transmission method, characterized in that, Applied to cloud nodes, the data transmission method includes: Receive a candidate path set and network indicator information of edge nodes; the candidate path set includes at least two candidate paths, and the candidate path set is determined by the edge node cluster based on path detection frequency, path validity period of cached paths, and the network indicator information; The candidate path set and the network indicator information are input into the intelligent scheduling model to perform real-time path reasoning and obtain the target candidate path output by the intelligent scheduling model; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set. The target candidate path is sent to the edge node cluster; the target candidate path is used to update the cached path of the edge node cluster, and the cached path is used to transmit the terminal's data to be transmitted.

8. The data transmission method according to claim 7, characterized in that, The intelligent scheduling model is a model built based on deep neural networks and pre-trained path scheduling models; The path scheduling model is trained based on the following steps: Based on the feasible sample paths, the sample network indicator information corresponding to the feasible sample paths, and the sample target variables corresponding to the feasible sample paths and the sample network indicator information, the second initial model is pre-trained to obtain the initial path scheduling model; the sample network indicator information includes multiple sample network indicators, and the sample target variables include sample transmission latency, sample quality of service indicators, and sample path classification labels. The initial path scheduling model is pre-trained to obtain the path scheduling model.

9. The data transmission method according to claim 8, characterized in that, The step of pre-training the initial path scheduling model to obtain the path scheduling model includes: Based on the initial path scheduling model, feature importance analysis is performed on multiple sample network indicators to determine at least one key indicator; the key indicator is the sample network indicator whose feature importance is higher than a preset threshold, and the feature importance is used to characterize the predictive contribution to the target variable of the sample. The key indicators are subjected to feature optimization processing to generate optimized network indicators; Based on the optimized network metrics, the initial path scheduling model is pre-trained to obtain the path scheduling model.

10. The data transmission method according to claim 8, characterized in that, The feasible path of the sample, the sample network indicator information corresponding to the feasible path of the sample, and the sample target variable corresponding to the feasible path of the sample and the sample network indicator information are obtained based on the following steps: Data preprocessing is performed on the initial sample feasible path and the initial sample network index information corresponding to the initial sample feasible path to obtain the sample feasible path and the sample network index information corresponding to the sample feasible path; the edge node cluster includes multiple edge nodes, the initial sample feasible path is the data transmission path between the access edge node and the target edge node, and the access edge node and the target edge node are any edge node in the edge node cluster; Based on the path scheduling scenario, determine the sample target variable corresponding to the sample feasible path and the sample network indicator information.

11. The data transmission method according to claim 7, characterized in that, Before receiving the candidate path set and the network indicator information of the edge nodes, the process also includes: The pre-training parameters of the intelligent scheduling model are sent to the edge node cluster; the pre-training parameters are used to train the edge scheduling model of the edge node cluster, and the edge scheduling model is used to generate the candidate path set.

12. The data transmission method according to claim 8, characterized in that, The sample network metrics include at least two of the following: sample bandwidth, sample bandwidth utilization, sample queuing latency, sample network jitter rate, sample edge node load, sample network packet loss rate, and sample packet out-of-order rate.

13. A data transmission device, characterized in that, Deployed in an edge node cluster, the edge node cluster including edge nodes, the data transmission device includes: The first receiving module is used to receive a data transmission request from the terminal; the data transmission request carries the data to be transmitted and the target address. The path determination module is used to determine the target path based on the path detection frequency, the path validity period of the cached path, and the network indicator information of the edge node; the target path is the data transmission path with the optimal transmission latency. The data transmission module is used to transmit the data to be transmitted to the target address based on the target path.

14. A data transmission device, characterized in that, Deployed on a cloud node, the data transmission device includes: The second receiving module is used to receive a candidate path set and network indicator information of edge nodes; the candidate path set includes at least two candidate paths, and the candidate path set is determined by the edge node cluster based on the path detection frequency, the path validity period of the cached path, and the network indicator information; The path filtering module is used to input the candidate path set and the network indicator information into the intelligent scheduling model to perform real-time path reasoning and obtain the target candidate path output by the intelligent scheduling model; the target candidate path is the candidate path with the optimal transmission latency in the candidate path set. A data sending module is used to send the target candidate path to the edge node cluster; the target candidate path is used to update the cached path of the edge node cluster, and the cached path is used to transmit the terminal's data to be transmitted.

15. A data transmission system, characterized in that, It includes an edge node cluster and a cloud node, wherein the edge node cluster performs the data transmission method as described in any one of claims 1 to 6, and the cloud node performs the data transmission method as described in any one of claims 7 to 12.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data transmission method as described in any one of claims 1 to 12.

17. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data transmission method as described in any one of claims 1 to 12.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the data transmission method as described in any one of claims 1 to 12.

Images