Data transmission optimization method considering data slicing and multi-path routing

Abstract

The present invention discloses a data transmission optimization method that comprehensively considers data slicing and multi-path routing selection, belongs to the field of data transmission technology, and solves the problems of low security and transmission efficiency in existing data transmission methods. The method includes: using a data slicing training set to train a data slicing model to generate data slicing rules, using data transmission results based on the data slicing rules to calculate a loss function, and optimizing model parameters of the data slicing model; training a multi-path routing planning model based on the multi-path routing training set to obtain a trained multi-path routing planning model; performing data slicing processing on real-time data to be transmitted using the data slicing rules predicted by the trained data slicing model; performing multi-path routing transmission on each data slice of the real-time data to be transmitted using the optimal multi-path routing algorithm predicted by the trained multi-path routing planning model; and recombining each data slice to complete the transmission.

Description

Technical Field

[0001] The present invention relates to the field of data transmission technology, and in particular to a data transmission optimization method that comprehensively considers data slicing and multi-path routing selection. Background Art

[0002] With the rapid development of the internet and information technology, the demands and complexity of data transmission continue to increase. In modern network environments, especially in business scenarios involving sensitive information, the security and efficiency of data transmission are paramount. To address these challenges, network slicing and multipath routing technologies are becoming key means of improving data transmission performance.

[0003] Currently, the main drawback of data transmission is that the data slicing process and the path routing process are not tightly integrated, making it difficult for them to work together to implement the data transmission process. Specifically, in the data slicing research process, data security is enhanced by dividing the data into multiple independent slices. However, this research does not involve how to route the sliced ​​data to achieve data transmission, that is, it does not consider how to efficiently and securely transmit the sliced ​​data. In fact, the relevant parameter settings for data transmission may affect the way data is sliced. In the research on multi-path routing planning, only reinforcement learning methods are used to plan multi-path routing, and it does not involve slicing and other pre-processing processes of the data before transmission, and does not address the different requirements for multi-path routing algorithms in different complex scenarios.

[0004] In summary, while both research directions in the existing technology have their own advantages, they lack a comprehensive consideration of data transmission mechanisms, resulting in significant deficiencies in practical applications. Therefore, designing a data transmission optimization method that comprehensively considers data slicing and multi-path routing to improve data transmission security, efficiency, and user experience is a pressing technical issue that needs to be addressed. Summary of the Invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a data transmission optimization method that comprehensively considers data slicing and multi-path routing selection to solve the problems of low security and transmission efficiency in existing data transmission methods.

[0006] The present invention discloses a data transmission optimization method that comprehensively considers data slicing and multi-path routing selection, the method comprising:

[0007] The data slicing training set is used to train the data slicing model to generate data slicing rules. The data transmission results based on the data slicing rules are used to calculate the comprehensive loss function and optimize the model parameters of the data slicing model. After multiple iterations of training, a trained data slicing model is obtained.

[0008] Training a multi-path routing planning model based on a multi-path routing training set to obtain a trained multi-path routing planning model;

[0009] The real-time data to be transmitted is processed by slicing data using the data slicing rules predicted by the trained data slicing model. The optimal multi-path routing algorithm predicted by the trained multi-path routing planning model is used to perform multi-path routing transmission on each data slice of the real-time data to be transmitted. The receiver reorganizes each data slice according to the predicted data slicing rules to complete the transmission.

[0010] On the basis of the above solution, the present invention also makes the following improvements:

[0011] Furthermore, each data slice training sample in the data slice training set includes a data sample to be transmitted, its confidentiality level, and transmission parameters.

[0012] Further, the data slice model is trained by performing the following operations:

[0013] In each training process, each data slice training sample in the data slice training set is used as input, and the data slice big model outputs the matching data slice rules;

[0014] Performing data slicing processing on the data sample to be transmitted according to the generated data slicing rules to obtain a number of data slices; and transmitting data to each data slice according to the transmission parameters of the data sample to be transmitted;

[0015] Calculate the comprehensive loss function based on the transmission performance index and confidentiality level during the data transmission process, and optimize the model parameters of the data slicing model based on the comprehensive loss function;

[0016] Jump to the next training until the preset training end conditions are met, and finally obtain a large data slice model that has passed the training.

[0017] Furthermore, the comprehensive loss function Expressed as:

[0018] (1)

[0019] in, 、 、 、 They represent the resource utilization loss function, data transmission delay loss function, packet loss rate loss function, and confidentiality level loss function respectively; 、 、 、 They represent the weight parameters of resource utilization loss function, data transmission delay loss function, packet loss rate loss function, and confidentiality level loss function respectively.

[0020] Furthermore, the transmission performance indicators include actual transmission volume, actual average delay and number of packet losses.

[0021] Furthermore, the resource utilization loss function Expressed as:

[0022] (2)

[0023] in, 、 They represent the actual transmission volume and theoretical maximum transmission volume of all data slices of the data sample to be transmitted respectively;

[0024] Data transmission delay loss function Expressed as:

[0025] (3)

[0026] in, 、 They represent the actual average delay and theoretical average delay of all data slices of the data samples to be transmitted respectively;

[0027] Packet loss rate loss function Expressed as:

[0028] (4)

[0029] in, 、 They represent the number of lost packets and the total number of transmitted packets of the data slice of the data sample to be transmitted respectively.

[0030] Furthermore, the confidentiality level loss function is expressed as for:

[0031] (5)

[0032] in, Indicates the actual number of data slices, Indicates the minimum number of data slices corresponding to the confidentiality level of the currently transmitted data sample.

[0033] Furthermore, in the multi-path routing training set, the multi-path routing samples include: transmission parameters of each routing path, the data volume and confidentiality level of the sample data to be transmitted, and the optimal multi-path routing algorithm label.

[0034] Furthermore, the multi-path routing planning model is trained by performing the following operations:

[0035] During the training process of the multi-path routing planning large model, the network status of each routing path, the amount of sample data to be transmitted and the confidentiality level are used as input, and the corresponding optimal multi-path routing algorithm label is used as output to train the multi-path routing planning large model.

[0036] Furthermore, the real-time data to be transmitted is transmitted by performing the following operations:

[0037] The data sender receives and determines whether data slicing can be performed on the real-time data to be transmitted. If so, the real-time data to be transmitted, its confidentiality level, and transmission parameters are input into the trained data slicing model to predict and output data slicing rules.

[0038] The data sender performs data slicing processing on the real-time data to be transmitted according to the data slicing rules to obtain several data slices;

[0039] Input the transmission parameters of each routing path from the data sender to the data receiver, the data volume of each data slice to be transmitted, and the confidentiality level into the trained multi-path routing planning model, and predict and output the optimal multi-path routing algorithm;

[0040] Obtain a multipath route for the data to be transmitted according to the predicted optimal multipath routing algorithm, and transmit each data slice of the real-time data to be transmitted to the data receiver according to the multipath route;

[0041] The data receiver reassembles the received data slices according to the data slicing rules to complete the data transmission.

[0042] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0043] The data transmission optimization method provided by the present invention, which comprehensively considers data slicing and multi-path routing, can effectively solve the problems of low security and transmission efficiency in existing data transmission methods, and has the following technical effects:

[0044] (1) Security improvement

[0045] 1) Data slicing combined with confidentiality levels: The data slicing training set contains the data samples to be transmitted and their confidentiality levels. The trained data slicing model generates corresponding slicing rules based on the confidentiality level of the data. For data with high confidentiality levels, more refined and dispersed slicing rules can be generated. This makes it difficult for attackers to recover the complete sensitive information even if the data is intercepted during transmission, thereby enhancing data confidentiality.

[0046] 2) Multi-path transmission reduces risk: With multi-path routing, different slices of data are sent over multiple paths. Even if some paths are attacked or fail, as long as some paths can still transmit data slices, the receiver can reconstruct the complete data according to the slicing rules. This reduces the risk of complete data loss or leakage due to single-path failure or attack, and improves the security and reliability of data transmission.

[0047] (2) Improvement of transmission efficiency

[0048] 1) Optimized data slicing rules: By continuously training and optimizing the data slicing model using the data slicing training set, the generated slicing rules can comprehensively consider factors such as transmission parameters. For example, based on network bandwidth, latency, and other conditions, data is sliced ​​into sizes and quantities that are more suitable for the transmission network. This makes the transmission of sliced ​​data more efficient, avoids transmission redundancy or transmission congestion caused by unreasonable slicing, and thus improves overall transmission efficiency.

[0049] 2) Intelligent Multipath Routing Planning: A large multipath routing planning model, trained with a multipath routing training set, predicts the optimal multipath routing algorithm based on the data volume, confidentiality level, and transmission parameters of each routing path for each data slice. This algorithm rationally allocates the transmission of each data slice along different paths, fully utilizing network resources and achieving load balancing. This avoids the bottleneck of single-path transmission and effectively improves data transmission speed and efficiency.

[0050] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference symbols denote the same components.

[0052] Figure 1 A flowchart of a data transmission optimization method that comprehensively considers data slicing and multi-path routing, provided by an embodiment of the present invention;

[0053] Figure 2 A flowchart of a data transmission method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0055] A specific embodiment of the present invention discloses a data transmission optimization method that comprehensively considers data slicing and multi-path routing. The flowchart of the method is as follows: Figure 1 shown.

[0056] Step S1: Use the data slicing training set to train the data slicing model to generate data slicing rules, use the data transmission results based on the data slicing rules to calculate the comprehensive loss function, and optimize the model parameters of the data slicing model; after multiple iterative training, a trained data slicing model is obtained.

[0057] During data transmission, for data with a higher confidentiality level, more data slicing operations are required to enhance data transmission security. Furthermore, when the data to be transmitted exceeds the transmission capacity of the network transmission unit, data slicing is required. Furthermore, data transmission is also affected by other parameters such as network status. Therefore, to address this issue, this embodiment designs the following data slicing model training method to achieve intelligent data slicing of data to be transmitted.

[0058] Step S11: Constructing a data slicing training set for training a data slicing large model.

[0059] Preferably, in this embodiment, each data slice training sample in the data slice training set includes a data sample (data packet) to be transmitted, its confidentiality level, and transmission parameters (including network status and transmission capacity of the network transmission unit). Detailed description is as follows.

[0060] (1) Data packet

[0061] A data packet consists of two parts: a header and a data portion. The header contains key information about the packet, guiding its transmission and processing within the network. This information may include the source address, destination address, protocol type, packet length, and checksum. The format and content of the header vary depending on the network protocol (such as IP or TCP). The data portion is the actual payload of the packet, containing user data or application data. The composition of each data slice is processed based on the header's identifier, flags, and slice offset.

[0062] Identification: All data slices of the same data packet use the same identification.

[0063] Flag: When DF (Don't Fragment) = 1, slicing is prohibited; when DF = 0, slicing is allowed. That is, for data packets formed by data slicing, if its DF = 0, slicing can still be performed.

[0064] When MF (More Fragment) = 1, there are subsequent slices. When MF = 0, there are no subsequent slices, that is, this slice is the last slice of the group. MF is meaningful only when DF = 0.

[0065] Fragment offset: The relative position of a slice in the original IP packet.

[0066] A simple data slicing rule example is shown in Table 1. As can be seen from Table 1:

[0067] Data packet to be transmitted: 20 bytes for header and 3800 bytes for data.

[0068] Rule: Each slice cannot exceed 1420 bytes (transmission capacity of the network transmission unit);

[0069] ID: 666;

[0070] Flag: DF=0, indicating that slicing is allowed; MF=0, indicating that no slicing will follow;

[0071] Slice offset: The slice offset of the first data slice is 0, and the slice offset of subsequent data slices depends on the length of the data portion of the previous data slice.

[0072] Table 1 Data slicing rule examples

[0073] .

[0074] (2) Confidentiality level

[0075] Before actual data transmission, the data sender will classify and grade the data to be transmitted based on its importance (primarily considering data transmission security) and determine the confidentiality level of the data to be transmitted. In specific implementations, the confidentiality level can be adaptively set based on the specific application scenario, and this embodiment does not impose any additional restrictions on this. Considering data transmission security, data with a higher confidentiality level tends to be transmitted using more data slices, while data with a lower confidentiality level tends to be transmitted using fewer data slices. Therefore, the confidentiality level also has a certain impact on the setting of data slicing rules.

[0076] (3) Network status

[0077] Network status includes information such as bandwidth, latency, and packet loss rate, which can be obtained through network monitoring tools and has a direct impact on the formulation of data slicing rules.

[0078] In the specific implementation process, given that the network status will change dynamically over time, the following methods can be used to process the network status during the transmission of data samples: (1) Time series vectorization / matrix representation: The time series data of the network status changing over time is presented in the form of vectors or matrices to facilitate subsequent mathematical operations and model processing; (2) Sliding window aggregation analysis: Set a fixed time window length (for example, per second or per minute) and perform aggregation analysis on the network status within the window. For example, calculate key indicators such as the average bandwidth, delay, and packet loss rate in the past 1 second, so as to more intuitively grasp the dynamic change trend of the network status; (3) Eigenvalue normalization processing: Scale the value of the network status to a fixed range (such as 0-1 or -1 to 1) to avoid adverse effects on model performance due to excessive differences in feature scales, ensuring that the model can more accurately evaluate and predict the network status.

[0079] (4) Transmission capacity of network transmission unit

[0080] The transmission capacity of the network transmission unit determines the maximum data length allowed during the transmission process, that is, the data to be transmitted must be transmitted within the transmission capacity of the network transmission unit to perform a single data transmission.

[0081] Preferably, in this embodiment, the data slicing rules include: the number of data slices, the size of each data slice, and the offset. The data slicing rules can help the data slicing model learn slicing rules in different scenarios.

[0082] It's important to note that the data samples to be transmitted in the data slicing training set all meet data slicing requirements. In particular, for latency-sensitive scenarios, data slicing may introduce additional processing delays, so you can choose not to slice and transmit the entire data packet directly. Alternatively, if the data packet explicitly prohibits slicing, data slicing is not required.

[0083] Furthermore, it's important to note that each data slice within the data segmentation rules for the same data sample to be transmitted must have the same identifier to ensure data uniqueness. Loss or inconsistency of identifiers can lead to confusion or data errors. Preserving identifiers also helps maintain the relevance and interpretability of data slices during the model output phase, facilitating subsequent analysis and verification.

[0084] Step S12: Use the data slice training set to train the data slice large model to obtain a trained data slice large model.

[0085] Preferably, the data slice large model can be implemented using the Transform model. The specific implementation process of this step is described below.

[0086] Step S121: In each training process, each data slicing training sample in the data slicing training set is used as input, and the data slicing large model outputs the matching data slicing rules (also called "data splitting rules").

[0087] The data slicing rule is used to guide the segmentation and organization of data during subsequent data transmission to adapt to different transmission conditions and requirements.

[0088] Step S122: performing data slicing processing on the data sample to be transmitted according to the generated data slicing rule to obtain a plurality of data slices; and performing data transmission on each data slice according to the transmission parameters of the data sample to be transmitted.

[0089] Step S123: Calculate a comprehensive loss function based on the transmission performance index and confidentiality level during the data transmission process, and optimize the model parameters of the data slicing model based on the comprehensive loss function.

[0090] Step S124: Jump to the next training until the preset training end condition is met, and finally a large data slice model that has passed the training is obtained.

[0091] That is, after obtaining the optimized model parameters in step S123, the optimized model parameters are reapplied to the data slicing rule generation and data transmission process, and the loss function is continued to be calculated and the model parameters are optimized until the preset training end conditions are met (such as the loss function converges to the specified threshold, the model parameter changes tend to be stable, the maximum number of iterations is reached, etc.), and finally a trained data slicing large model is obtained.

[0092] In step S123, the transmission performance indicators during the data transmission process include the actual transmission volume, the actual average delay, and the number of packet losses. Preferably, in the process of calculating the loss function based on the transmission performance indicators during the data transmission process, in order to comprehensively measure the quality of the data slicing rules, this embodiment can define multiple loss functions to evaluate different performance objectives, such as a resource utilization loss function (measuring bandwidth utilization efficiency), a data transmission delay loss function, and a packet loss rate loss function.

[0093] 1) Resource Utilization Loss Function ,

[0094] (1)

[0095] in, 、 The actual transmission volume and theoretical maximum transmission volume of all data slices of the data sample to be transmitted, respectively. The actual transmission volume of all data slices refers to the amount of data that can actually be transmitted to the receiving end, which can be monitored by network equipment. The theoretical maximum transmission volume is the data volume of all data slices, which can be measured using network performance testing tools (such as iPerf).

[0096] Compute resource utilization loss function , designed to maximize broadband usage efficiency. The smaller the value (closer to 0), the higher the bandwidth usage efficiency.

[0097] 2) Data transmission delay loss function ,

[0098] (2)

[0099] in, 、 The actual average delay and theoretical average delay of all data slices of the data sample to be transmitted are respectively represented. The actual average delay of all data slices is the average transmission delay experienced by all data slices in actual transmission. The theoretical average delay of all data slices is the average transmission delay expected under ideal conditions (no congestion, queuing delay, etc.).

[0100] Assuming data slice The sending time and receiving time are expressed as 、 , then, data slice The actual delay Expressed as:

[0101] (3)

[0102] The actual average latency of all data slices Expressed as:

[0103] (4)

[0104] Indicates the total number of data slices.

[0105] In addition, you can use network simulation tools (such as NS2 and OMNeT++) to simulate the network environment and measure the theoretical average delay.

[0106] Calculate the data transmission delay loss function , aims to minimize data transmission time. The actual average delay is always greater than or equal to the theoretical average delay because there are various additional delay factors in the actual network, while the theoretical delay only considers the basic transmission time under ideal conditions. The smaller it is (closer to 1), the closer the transmission time is to the ideal state.

[0107] 3) Packet loss rate loss function ,

[0108] (5)

[0109] in, 、 They represent the number of lost packets and the total number of transmitted packets of the data slice of the data sample to be transmitted respectively.

[0110] Calculate packet loss rate loss function , designed to reduce the probability of data loss. The smaller the value (closer to 0), the more reliable the data transmission.

[0111] Preferably, in this embodiment, the large data slice model tends to output more data slices in the case of a high confidentiality level. This ensures that in the event of a malicious attack or data interception, even if the attacker or interceptor obtains a portion of the data slices, they can only obtain a small portion of the data slices, and it is difficult to restore the complete data content, thereby achieving the purpose of ensuring data security. Based on this, the following confidentiality level loss function is set.

[0112] 4) Confidentiality Level Loss Function ,

[0113] (6)

[0114] in, Indicates the actual number of data slices, Indicates the minimum number of data slices corresponding to the confidentiality level of the currently transmitted data sample.

[0115] During implementation, different minimum data slice counts can be set for different confidentiality levels based on the actual network environment. For example, for a low confidentiality level, the minimum number of data slices is 2; for a medium confidentiality level, the minimum number of data slices is 5; and for a high confidentiality level, the minimum number of data slices is 15. Therefore, when the actual number of data slices is less than the corresponding minimum number of data slices, the loss value of the confidentiality level loss function increases proportionally; when the minimum number of data slices is met or exceeded, the loss value is 0.

[0116] The above loss functions reflect the degree of deviation between the data slicing rules and the ideal transmission effect from different dimensions. Preferably, a comprehensive loss function can be constructed and the relationship between the above different loss functions can be balanced by weights. Comprehensive loss function Expressed as:

[0117] (7)

[0118] in, 、 、 、 Represent resource utilization loss functions respectively , data transmission delay loss function , packet loss rate loss function , confidentiality level loss function In the specific implementation process, the weight parameters can be pre-set according to the actual situation or expert experience.

[0119] Step S2: training a multi-path routing planning large model based on the multi-path routing training set to obtain a trained multi-path routing planning large model.

[0120] During the route discovery process, multipath routing protocols form multiple paths to the destination node. Multipath routing has the following advantages: (1) Load balancing: Traffic is distributed across multiple paths to avoid overloading a single path, thereby improving overall bandwidth utilization; (2) Fault tolerance: When a path fails, data can continue to be transmitted through other paths, enhancing system reliability; (3) Fast transmission: By transmitting data in parallel across multiple paths, the total delay of data transmission can be reduced.

[0121] Preferably, the multipath routing training set provided in this embodiment includes the following: transmission parameters for each routing path (such as bandwidth, latency, and packet loss rate), the volume and confidentiality level of the sample data to be transmitted, and the label of the optimal multipath routing algorithm. During the labeling process, when the network is in good condition and traffic is stable, the appropriate algorithm is labeled; when the network is congested and the data is of high importance, the algorithm that provides high availability and fast response is labeled.

[0122] During the training process of the large multipath routing planning model, the network status of each routing path, the amount of sample data to be transmitted, and the confidentiality level are used as input, and the corresponding optimal multipath routing algorithm label is used as output. By optimizing loss functions (such as cross entropy loss and mean squared error), the model learns to select the appropriate routing algorithm under different network conditions.

[0123] Common multipath routing algorithm labels include: Equal Cost Multipath Routing (ECMP), Weighted Multipath Routing (WCMP), Global Awareness Network Traffic Distribution Optimization Algorithm based on Multipath Routing, Multipath Transmission Control Protocol (MPTCP), Greedy Algorithm, and Multipath Routing based on Genetic Algorithm. They are respectively suitable for the following situations:

[0124] (1) Equal-cost multi-path routing (ECMP): This is a scenario where there are multiple paths with the same cost in the network and simple load balancing is required, but there are no complex requirements for traffic distribution.

[0125] (2) Weighted Multi-Path Routing (WCMP): Networks with unbalanced resources, scenarios requiring high availability and stability, multi-link environments, and dynamic network environments;

[0126] (3) Globally aware network traffic distribution optimization algorithm based on multi-path routing: In scenarios requiring high availability and stability, such as large-scale data processing, real-time data analysis, and microservice architecture, network slicing technology can create independent virtual networks for different types of services. Globally aware algorithms can optimize traffic distribution for each slice to ensure efficient resource utilization.

[0127] (4) Multipath Transmission Control Protocol (MPTCP): It is suitable for mobile devices that need to transmit data in parallel on multiple network interfaces (such as WiFi and mobile networks), applications with high bandwidth and fault tolerance requirements, and scenarios that require high availability and fast response;

[0128] (5) Greedy algorithm: The network structure is simple, the path selection is not complicated, and the computing resource requirements are low. It is suitable for preliminary path selection strategies or small networks.

[0129] (6) Multi-path routing based on genetic algorithms: In complex network environments, there is a need to find solutions close to the global optimal solution. It is suitable for situations where the optimization problem is more complex, the network topology changes frequently, and flexible adaptation is required.

[0130] Step S3: Utilize the trained data slicing model and the multi-path routing planning model to transmit the real-time data to be transmitted.

[0131] The specific implementation process of step S3 is described as follows.

[0132] Step S31: The data sender receives and determines whether data slicing can be performed on the real-time data to be transmitted. If so, the real-time data to be transmitted, its confidentiality level, and transmission parameters are input into the trained data slicing model to predict and output data slicing rules.

[0133] Step S32: The data sender performs data slicing processing on the real-time data to be transmitted according to the data slicing rules to obtain a number of data slices.

[0134] Step S33: The transmission parameters of each routing path from the data sender to the data receiver, the data volume of each data slice to be transmitted, and the confidentiality level are input into the trained multi-path routing planning model to predict and output the optimal multi-path routing algorithm.

[0135] Step S34: obtaining a multi-path route for the real-time data to be transmitted according to the predicted optimal multi-path routing algorithm, and transmitting each data slice of the data to be transmitted to the data receiver according to the multi-path route.

[0136] Step S35: The data receiver reassembles the received data slices according to the data slicing rules to complete the data transmission.

[0137] During the data transmission process of this embodiment, a multipath routing algorithm selects a large model that chooses an appropriate transmission path based on the transmission environment and data level. Data slices (including the IP address of the access slices, slice identifiers, and other information) are sent simultaneously over multiple paths. The data sender and receiver use a consensus algorithm to determine the data slicing rules. The receiver then reassembles the data slices according to the corresponding rules, completing the data transmission. The consensus algorithm for data slicing rules involves the sender obtaining the data slicing rules through the consensus algorithm, and the receiver reassembles the data using the rules. The consensus algorithm can employ Federated Byzantine Agreement (FBA), which can reach consensus quickly and achieve efficient and secure information confirmation in large-scale networks.

[0138] In summary, the method provided in this embodiment has the following beneficial effects: Appropriate slicing rules are generated from the trained large model, and these rules are used to slice the data, forming multiple, mutually isolated, independent slices of different parts of the data to be transmitted, thereby improving data confidentiality and transmission efficiency, and enhancing the user experience. The sliced ​​data is transmitted using multiple paths. In the face of various complex network environments, the large model is used to dynamically select the appropriate multi-path routing algorithm in each complex environment to enhance bandwidth capacity and fault tolerance, achieve load balancing, and reduce latency.

[0139] Based on data slicing to improve data confidentiality, a large model is used to intelligently generate data splitting rules, making them more adaptable to data of varying sizes and confidentiality levels. Multi-path routing is planned using a large model. Trained on large datasets, the large model can learn from a wealth of knowledge, reducing reliance on large amounts of training data in specific environments. The large model also has better generalization capabilities and can adapt to different network environments and traffic patterns.

[0140] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0141] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A data transmission optimization method that comprehensively considers data slicing and multi-path routing, characterized in that: The method comprises: The data slicing training set is used to train the data slicing model to generate data slicing rules. The data transmission results based on the data slicing rules are used to calculate the comprehensive loss function and optimize the model parameters of the data slicing model. After multiple iterations of training, a trained data slicing model is obtained. Training a multi-path routing planning model based on a multi-path routing training set to obtain a trained multi-path routing planning model; The real-time data to be transmitted is processed by slicing data using the data slicing rules predicted by the trained data slicing model. The optimal multi-path routing algorithm predicted by the trained multi-path routing planning model is used to perform multi-path routing transmission on each data slice of the real-time data to be transmitted. The receiver reassembles each data slice according to the predicted data slicing rules to complete the transmission. Each data slice training sample in the data slice training set includes a data sample to be transmitted, its confidentiality level, and transmission parameters; the data slice large model is trained by performing the following operations: In each training process, each data slice training sample in the data slice training set is used as input, and the data slice big model outputs the matching data slice rules; Performing data slicing processing on the data sample to be transmitted according to the generated data slicing rules to obtain a number of data slices; and transmitting data to each data slice according to the transmission parameters of the data sample to be transmitted; Calculating a comprehensive loss function based on the transmission performance indicators and confidentiality level during the data transmission process, and optimizing the model parameters of the data slicing model based on the comprehensive loss function; the transmission performance indicators include actual transmission volume, actual average delay, and number of packet losses; Jump to the next training until the preset training end conditions are met, and finally obtain a large data slice model that has passed the training; The comprehensive loss function Expressed as: (1) in, 、 、 、 They represent the resource utilization loss function, data transmission delay loss function, packet loss rate loss function, and confidentiality level loss function respectively; 、 、 、 They represent the weight parameters of resource utilization loss function, data transmission delay loss function, packet loss rate loss function, and confidentiality level loss function respectively; Resource utilization loss function Expressed as: (2) in, 、 They represent the actual transmission volume and theoretical maximum transmission volume of all data slices of the data sample to be transmitted respectively; Data transmission delay loss function Expressed as: (3) in, 、 They represent the actual average delay and theoretical average delay of all data slices of the data samples to be transmitted respectively; Packet loss rate loss function Expressed as: (4) in, 、 Respectively represent the number of lost packets and the total number of transmitted packets of the data slice of the data sample to be transmitted; Confidentiality level loss function representation for: (5) in, Indicates the actual number of data slices, Indicates the minimum number of data slices corresponding to the confidentiality level of the currently transmitted data sample.

2. The data transmission optimization method according to claim 1, wherein: In the multi-path routing training set, the multi-path routing samples include: transmission parameters of each routing path, the data volume and confidentiality level of the sample data to be transmitted, and the optimal multi-path routing algorithm label.

3. The data transmission optimization method according to claim 2, wherein: Train the multipath routing model by performing the following operations: During the training process of the multi-path routing planning large model, the network status of each routing path, the amount of sample data to be transmitted and the confidentiality level are used as input, and the corresponding optimal multi-path routing algorithm label is used as output to train the multi-path routing planning large model.

4. The data transmission optimization method according to claim 3, wherein: To transfer real-time data, perform the following operations: The data sender receives and determines whether data slicing can be performed on the real-time data to be transmitted. If so, the real-time data to be transmitted, its confidentiality level, and transmission parameters are input into the trained data slicing model to predict and output data slicing rules. The data sender performs data slicing on the real-time data to be transmitted according to the data slicing rules to obtain several data slices; Input the transmission parameters of each routing path from the data sender to the data receiver, the data volume of each data slice to be transmitted, and the confidentiality level into the trained multi-path routing planning model, and predict and output the optimal multi-path routing algorithm; Obtain a multipath route for the data to be transmitted according to the predicted optimal multipath routing algorithm, and transmit each data slice of the real-time data to be transmitted to the data receiver according to the multipath route; The data receiver reassembles the received data slices according to the data slicing rules to complete the data transmission.

Images