A path and bandwidth coordination allocation method and system for data task migration

Abstract

The application discloses a path and bandwidth coordination allocation method and system for data task migration, and belongs to the technical field of network resource allocation, and specifically comprises the following steps: extracting migration data volume, a deadline and a service priority parameter of an input end data migration task, and establishing an initial bandwidth demand feature vector of the data migration task; fusing the initial bandwidth demand feature vector and an existing service occupation state of a network link, eliminating a limited link, and generating a candidate transmission path set satisfying a protection constraint; constructing a joint coordination mapping matrix of the path and the bandwidth; solving the joint coordination mapping matrix to generate a global coordination allocation result, analyzing a target migration path and a transmission bandwidth allocation quota of the global coordination allocation result; injecting the data migration task into the target migration path according to the transmission bandwidth allocation quota for transmission operation, and continuously collecting network state variation parameters of the target migration path in real time.

Description

Technical Field

[0001] This invention relates to the field of network resource allocation technology, specifically to a method and system for collaborative allocation of paths and bandwidth for data task migration. Background Technology

[0002] In an architecture deeply integrating distributed computing and wide area networks, network-computer collaborative scheduling technology has become a core foundation supporting cross-data center computing power services and business interconnection. To address the data flow needs between multi-source heterogeneous computing power nodes, existing network scheduling systems generally deploy data task migration mechanisms to achieve coordinated utilization of cross-domain resources. When executing such data migration tasks, the core management operations of the scheduling system mainly rely on path selection and bandwidth configuration to achieve resource allocation on the network side. Conventional scheduling strategies are usually based on known static network topology, using shortest path first or basic traffic engineering algorithms to calculate the data transmission route from the source node to the destination node, and then issuing bandwidth reservation instructions to the underlying network devices according to service tags or task description files. This basic resource management mechanism, in a conventional network environment, establishes end-to-end connections for data flow at the physical device level, maintaining basic data forwarding and network-computer interoperability.

[0003] However, existing technologies typically treat path selection and bandwidth allocation as two separate decision-making stages when handling data migration tasks, lacking a deep collaborative mapping mechanism. Traditional resource allocation often involves statically planning the optimal path before blindly requesting underlying bandwidth. This fragmented decision-making model ignores the dynamic distribution characteristics of available bandwidth in the network. When a large number of sudden migration tasks are routed to local paths, it can easily lead to link overload and severe congestion, and can also forcibly seize underlying resources, crowding out the guaranteed bandwidth of existing core services and triggering service level agreement (SLA) breaches. Furthermore, due to the lack of coordination between routing and bandwidth adjustment, the system cannot flexibly allocate remaining global resources when available bandwidth decreases, directly leading to a deterioration in the actual data transmission rate and frequent migration task timeouts and failures. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for collaborative allocation of path and bandwidth for data task migration, thereby solving the problems in the background art:

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for collaborative allocation of path and bandwidth for data task migration, comprising:

[0007] S1: Extract the migration data volume, deadline, and business priority parameters of the input data migration task, and establish the initial bandwidth requirement feature vector of the data migration task.

[0008] S2: Integrate the initial bandwidth requirement feature vector with the existing service occupancy status of network links, eliminate restricted links, and generate a set of candidate transmission paths that meet protection constraints;

[0009] S3: Extract the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and construct a joint collaborative mapping matrix of path and bandwidth;

[0010] S4: Solve the joint cooperative mapping matrix to generate the global cooperative allocation result, and analyze the target migration path and transmission bandwidth allocation quota of the global cooperative allocation result;

[0011] S5: Inject the data migration task into the target migration path according to the transmission bandwidth quota for transmission operation, and continuously collect the network status variation parameters of the target migration path in real time.

[0012] S6: Trigger incremental replanning instructions based on network state variation parameters, and iteratively correct the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data.

[0013] As a further aspect of the present invention: in step S1, the process of extracting the migration data volume, deadline, and business priority parameters of the input data migration task, and establishing the initial bandwidth requirement feature vector of the data migration task, is as follows:

[0014] Intercept the data migration task request from the input end, parse the description file of the data migration task request, and extract the initial values ​​of the migration data volume, deadline, and business priority parameters.

[0015] The initial values ​​of the migration data volume, deadline, and business priority parameters are input into the dimension elimination function to generate standardized values ​​within the same metric space.

[0016] The standardized values ​​are linearly concatenated and spliced ​​according to the preset dimensional order to establish the initial bandwidth requirement feature vector for the data migration task.

[0017] As a further aspect of the present invention: In step S2, the process of fusing the initial bandwidth demand feature vector with the existing service occupancy status of network links, eliminating restricted links, and generating a set of candidate transmission paths that meet protection constraints is as follows:

[0018] The expected bandwidth requirement of the data migration task is parsed from the initial bandwidth requirement feature vector; the existing service occupancy status of the network links is extracted, and the existing service occupancy status is added to the expected bandwidth requirement value to generate the predicted total load value of each network link.

[0019] The predicted total load value is compared with the protection constraint parameter. Network links whose predicted total load value is greater than the protection constraint parameter are marked as restricted links and removed.

[0020] Search for the remaining network links after removing restricted links, extract all connected branches between nodes, and merge all connected branches to generate a set of candidate transmission paths that satisfy the protection constraints.

[0021] As a further aspect of the present invention: in step S3, the process of extracting the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and constructing a joint cooperative mapping matrix of paths and bandwidth, is as follows:

[0022] Traverse the topology of the candidate transmission path set, collect the remaining capacity, latency, congestion risk and service pricing characteristics of the corresponding path, and generate the original feature dataset.

[0023] The original feature dataset is input into the normalization model, which performs a dimensionless operation to transform and generate a standard path feature parameter sequence with a uniform metric.

[0024] Extract the discrete bandwidth allocation vector, set the candidate transmission path set as the matrix row dimension, set the discrete bandwidth allocation vector as the column dimension, and establish the skeleton of the cooperative mapping matrix;

[0025] The standard path feature parameter sequence is written into the coordinate nodes of the collaborative mapping matrix skeleton according to the dimensional relationship, and a joint collaborative mapping matrix of path and bandwidth is constructed.

[0026] As a further aspect of the present invention: in step S4, the process of solving the joint cooperative mapping matrix to generate the global cooperative allocation result, and parsing the target migration path and transmission bandwidth allocation quota of the global cooperative allocation result is as follows:

[0027] Establish an objective function with the minimum migration cost as the convergence constraint, and input the objective function and the joint cooperative mapping matrix into the optimization algorithm model to perform the solution process;

[0028] The optimization algorithm model is triggered to perform iterative convergence operations in the feasible solution space, extract the discrete optimal solutions that satisfy the objective function, and combine the discrete optimal solutions to generate a global collaborative allocation result;

[0029] The parsing component is invoked to read the global collaborative allocation result, and the data structure of the global collaborative allocation result is reverse-decoded to separate the position index vector representing the route direction;

[0030] Map the location index vector to the candidate transmission path set, lock the corresponding target migration path, and extract the transmission bandwidth allocation quota carried by the global collaborative allocation result.

[0031] As a further aspect of the present invention: in step S5, the process of injecting the data migration task into the target migration path according to the transmission bandwidth allocation quota for transmission operation, and continuously collecting the network status variation parameters of the target migration path in real time, is as follows:

[0032] Set the port rate according to the transmission bandwidth allocation quota, divide the data migration task into packets, and inject the packets into the target migration path according to the set port rate to perform the transmission operation.

[0033] Based on the target migration path for performing transmission operations, a telemetry probe is activated to continuously capture operational snapshots of nodes along the route in real time and extract the basic link features carried by the operational snapshots.

[0034] By comparing the basic link characteristics of adjacent time series, the bandwidth fluctuation value and latency spike value of the target migration path are analyzed, and the fluctuation value and spike value are concatenated to generate network state variation parameters.

[0035] As a further aspect of the present invention: in step S6, the process of triggering incremental replanning instructions based on network state variation parameters and iteratively correcting the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data is as follows:

[0036] The network state variation parameter is input into the comparator. When the network state variation parameter exceeds the preset boundary, an incremental replanning instruction is triggered, and the latest network snapshot data of the entire network is extracted at the same time.

[0037] According to the incremental replanning instruction, the latest network snapshot data is injected into the joint collaborative mapping matrix to update the link parameters, and the target migration path and transmission bandwidth allocation quota are recalculated and iteratively corrected.

[0038] A path and bandwidth collaborative allocation system for data task migration, comprising:

[0039] The requirement building module is used to extract the migration data volume, deadline, and business priority parameters of the input data migration task, and to establish the initial bandwidth requirement feature vector of the data migration task.

[0040] The path filtering module is used to integrate the initial bandwidth requirement feature vector with the existing service occupancy status of network links, eliminate restricted links, and generate a set of candidate transmission paths that meet protection constraints.

[0041] The matrix construction module is used to extract the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and construct a joint cooperative mapping matrix of paths and bandwidth.

[0042] The collaborative allocation module is used to solve the joint collaborative mapping matrix to generate global collaborative allocation results, and to parse the target migration path and transmission bandwidth allocation quota of the global collaborative allocation results.

[0043] The transmission acquisition module is used to inject the data migration task into the target migration path according to the transmission bandwidth allocation quota for transmission operation, and to continuously collect the network status variation parameters of the target migration path in real time.

[0044] The incremental planning module is used to trigger incremental replanning instructions based on network state variation parameters, and iteratively correct the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data.

[0045] An electronic device includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the execution of the program on the processor implements the steps of a path and bandwidth collaborative allocation method for data task migration as described above.

[0046] A storage medium storing a computer program, which, when executed by a processor, implements the steps of a path and bandwidth collaborative allocation method for data task migration as described above.

[0047] The beneficial effects of this invention are:

[0048] First, this invention breaks away from the traditional decision-making model that separates routing and bandwidth allocation by constructing a joint collaborative mapping matrix of paths and bandwidth, achieving deep collaboration and globally optimal scheduling of underlying network resources. Specifically, in the initial allocation phase, this scheme integrates the initial bandwidth demand feature vector with the existing service occupancy status of links and eliminates restricted links, establishing strict existing service protection constraints. This fundamentally avoids service level agreement (SLA) breaches triggered by a large number of sudden tasks forcibly seizing resources. Subsequently, a joint matrix is ​​solved by comprehensively considering features such as remaining capacity, latency, and congestion risk, outputting the target migration path and transmission bandwidth allocation quota in one step. This "joint debugging and allocation" mechanism abandons the blind bandwidth requests after static routing and effectively avoids link overload and severe congestion caused by sudden tasks being routed to local paths.

[0049] Secondly, this invention establishes a dynamic incremental replanning mechanism based on closed-loop feedback of network state variations, significantly improving transmission stability and task success rate under scenarios of diminishing available bandwidth. After task injection and transmission, the system continuously collects network state variation parameters in real time, endowing routing and bandwidth adjustment with a high degree of dynamic linkage capability. When the network environment deteriorates or available bandwidth diminishes, this scheme can immediately trigger incremental replanning instructions and iteratively correct paths and quotas using the latest network snapshot data. This microsecond or millisecond-level adaptive rebalancing operation ensures that the system can flexibly schedule global remaining resources for compensation, effectively curbing the degradation trend of data transmission rates and solving the problem of migration task timeout failures caused by frequent congestion in traditional mechanisms. Attached Figure Description

[0050] The invention will now be further described with reference to the accompanying drawings.

[0051] Figure 1 This is a flowchart illustrating a method for collaborative allocation of path and bandwidth for data task migration according to the present invention.

[0052] Figure 2 This is a schematic diagram of a path and bandwidth collaborative allocation method for data task migration according to the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Please see Figure 1 As shown, this invention provides a method for collaborative allocation of path and bandwidth for data task migration, comprising the following steps:

[0055] S1: Extract the migration data volume, deadline, and business priority parameters of the input data migration task, and establish the initial bandwidth requirement feature vector of the data migration task.

[0056] S2: Integrate the initial bandwidth requirement feature vector with the existing service occupancy status of network links, eliminate restricted links, and generate a set of candidate transmission paths that meet protection constraints;

[0057] S3: Extract the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and construct a joint collaborative mapping matrix of path and bandwidth;

[0058] S4: Solve the joint cooperative mapping matrix to generate the global cooperative allocation result, and analyze the target migration path and transmission bandwidth allocation quota of the global cooperative allocation result;

[0059] S5: Inject the data migration task into the target migration path according to the transmission bandwidth quota for transmission operation, and continuously collect the network status variation parameters of the target migration path in real time.

[0060] S6: Trigger incremental replanning instructions based on network state variation parameters, and iteratively correct the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data.

[0061] In one embodiment of the present invention, the process of extracting the migration data volume, deadline, and service priority parameters of the input data migration task and establishing the initial bandwidth requirement feature vector of the data migration task in step S1 is as follows:

[0062] In this embodiment, the specific implementation process of step S1 begins from the input end. Specifically, the network scheduling controller deployed in the core computer room continuously listens for task request signals in the network. When a user or upper-layer application system issues a data migration task request through a terminal device, the network scheduling controller intercepts the data migration task request through its inbound port. After intercepting the data migration task request, the network scheduling controller calls its internal parsing engine to read the description file carried in the data migration task request. It should be noted that the description file is a data packet pre-packaged according to a specific format, containing key indicators required to perform the migration. The parsing engine unpacks the data layer by layer according to the encapsulation format, accurately extracting the initial values ​​of three key indicators: the amount of data to be migrated, the deadline, and the business priority parameter. For example, the initial value of the extracted migration data amount may be 500 gigabytes, the initial value of the deadline may be 3600 seconds, and the initial value of the business priority parameter may be 3. These initial values ​​will be temporarily stored in the network scheduling controller's cache as the raw input material for subsequent data processing, ensuring the complete capture of the core attributes of the task.

[0063] After obtaining the initial values ​​of the three parameters mentioned above, due to the significant differences in their physical units and orders of magnitude, they cannot be directly used for unified mathematical calculations and comparisons. Therefore, these initial values ​​need to be processed by a dimensionless function. Specifically, the microprocessor inside the network scheduling controller extracts the initial values ​​of the migration data volume, deadline, and service priority parameters from the buffer and inputs them sequentially into the pre-configured dimensionless function. It should be noted that the dimensionless function is a mathematical mapping algorithm that scales values ​​with different physical meanings according to a certain ratio, projecting them into a unified numerical range, such as 0 to 1. Through the operation of the dimensionless function, the initial values, originally with different physical units, are stripped of their dimensional attributes, transforming them into standardized values ​​within the same metric space. It should be noted that the same metric space means that the three standardized values ​​after transformation no longer have specific physical labels such as gigabytes or seconds, but become pure, relative proportional values ​​of the same order of magnitude. This allows subsequent matrix operations and feature comparisons to fairly measure the weight of each parameter, avoiding calculation biases caused by significant differences in absolute values.

[0064] After dimension elimination, the network scheduling controller further performs vector construction. The controller's internal logic unit linearly concatenates the three standardized values ​​according to a preset dimensional order. It's important to note that this preset dimensional order is a fixed arrangement rule set by the administrator during system initialization. For example, the first dimension might correspond to the amount of data to be migrated, the second to the deadline, and the third to the service priority parameter. Linear concatenation means combining these three independent scalar values ​​into an ordered one-dimensional array, following the fixed arrangement rule. For instance, if the standardized value for the amount of data to be migrated is 0.8, the standardized value for the deadline is 0.5, and the standardized value for the service priority parameter is 0.6, and the preset dimensional order is data amount, time, priority, then the resulting one-dimensional array after linear concatenation is [0.8, 0.5, 0.6]. This one-dimensional array, formed by sequentially combining standardized values, constitutes the initial bandwidth requirement feature vector for the completed data migration task. Thus, the network scheduling controller successfully transformed the originally scattered and heterogeneous task description information into a structured and computable mathematical vector, laying the data foundation for subsequent fusion calculations with network link states.

[0065] In one embodiment of the present invention, step S2, the process of fusing the initial bandwidth demand feature vector with the existing service occupancy status of network links, eliminating restricted links, and generating a set of candidate transmission paths that meet protection constraints, is as follows:

[0066] In this embodiment, the specific implementation process of step S2 begins with the calculation of the superposition of task requirements and network load. After obtaining the initial bandwidth requirement feature vector generated in the previous step, the network scheduling controller calls the internal resource parsing module to parse the expected bandwidth requirement value of the data migration task based on the initial bandwidth requirement feature vector. It should be noted that the initial bandwidth requirement feature vector includes the standardized migration data volume and deadline. The network scheduling controller remaps the standardized values ​​of these two dimensions to the actual physical bandwidth rate through a preset reverse conversion rule, thereby calculating the expected bandwidth requirement value necessary to complete the data migration task. Subsequently, the network scheduling controller extracts the existing service occupancy status of each network link in real time through the southbound interface protocol of the software-defined network. It should be noted that the existing service occupancy status refers to the physical bandwidth capacity currently occupied by existing core service data flows in the network link. The arithmetic logic unit inside the network scheduling controller adds the existing service occupancy status of each network link to the calculated expected bandwidth requirement value, that is, adds the bandwidth used by the current link to the bandwidth required by the task, to generate the predicted total load value of each network link. The predicted total load figure intuitively reflects the expected level of total traffic load that the link will bear after the current data migration task is injected into the corresponding link.

[0067] After generating the predicted total load values ​​for each network link, the policy comparison module in the network scheduling controller begins to perform security constraint determination and restricted link removal operations. Specifically, the network scheduling controller's storage medium is pre-configured with protection constraint parameters. It should be noted that these protection constraint parameters are link bandwidth safety thresholds set by network administrators to prevent link overload and ensure the quality of existing core services. They are typically set as a specific percentage of the total physical bandwidth of the link, such as 80% of the total bandwidth. The policy comparison module compares the predicted total load value calculated for each network link with the corresponding protection constraint parameter one by one. When the predicted total load value of a network link exceeds the protection constraint parameter, it indicates that adding this data migration task on top of the existing services will exceed the safety bandwidth threshold, easily causing network congestion and crowding out the guaranteed bandwidth for core services. Therefore, the network scheduling controller marks network links with predicted total load values ​​exceeding the protection constraint parameters as restricted links and removes them from the logical topology view, rendering them unavailable in subsequent routing logic, thereby physically preventing the risk of excessive traffic concentration.

[0068] After removing restricted links, the network scheduling controller transfers the routing calculation task to the internal topology calculation engine to generate a set of secure candidate paths. Based on the logical topology view after removing restricted links, the topology calculation engine searches for the remaining network links. Specifically, the engine uses a depth-first search or breadth-first search algorithm from graph theory to traverse the connections of the remaining network links, finding reachable routes between the source and destination nodes of the data migration task, thus extracting all connected branches between nodes. It should be noted that a connected branch refers to an independent path topology structure in the network after removing restricted links, composed of interconnected remaining links, that enables end-to-end data communication between the source and destination nodes. After extracting all possible connected branches, the topology calculation engine logically summarizes these independent connected branches, merging them to generate a set of candidate transmission paths that satisfy protection constraints. Each path in this set of candidate transmission paths consists of secure remaining network links, ensuring the underlying bandwidth availability and security of existing services from the source for subsequent path selection.

[0069] In one embodiment of the present invention, step S3, which involves extracting the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set and constructing a joint cooperative mapping matrix of paths and bandwidth, is as follows:

[0070] In this embodiment, the specific implementation process of step S3 begins with the feature acquisition of candidate paths. After generating a set of candidate transmission paths, the topology calculation engine inside the network scheduling controller starts the feature acquisition module to traverse the topology of the candidate transmission path set. During the traversal, the feature acquisition module collects four key feature data of each candidate path in real time through the southbound interface protocol of the software-defined network and telemetry probes. It should be noted that the remaining capacity refers to the maximum available bandwidth on the candidate path, in megabits per second; latency refers to the time required for a data packet to be transmitted from the source node to the destination node via the candidate path, in milliseconds; congestion risk refers to the probability that the current load of the candidate path is close to saturation, causing packet loss or queuing, usually expressed as a percentage; service pricing feature refers to the rental cost set by the network operation policy for the unit bandwidth of the candidate path, in yuan per megabit per second. The feature acquisition module stores the collected raw data of these four different physical meanings and numerical ranges according to the path number, generating a feature raw dataset. This feature raw dataset constitutes the data base for subsequent collaborative computing, ensuring that the underlying network state information is fully captured.

[0071] After generating the original feature dataset, the data processing module within the network scheduling controller begins to unify the heterogeneous features. Specifically, the data processing module inputs the original feature dataset into a pre-configured data normalization model to perform dimensionless transformation. It's important to note that dimensionless transformation is an algorithmic process that eliminates the influence of different physical units through mathematical transformations. Its purpose is to project feature data of different units and orders of magnitude onto the same numerical interval, such as a closed interval between 0 and 1. Since remaining capacity prioritizes maximum values, latency and service pricing features prioritize minimum values, and congestion risk prioritizes minimum values, the data normalization model integrates range transformation or standard deviation transformation methods to perform corresponding positive normalization and scaling processing for indicators in different directions. After the data normalization model's operation, all data in the original feature dataset is stripped of specific physical units such as megabits per second, milliseconds, and yuan, transforming it into a standard path feature parameter sequence with a unified metric scale. It should be noted that each value in the standard path feature parameter sequence is under the same metric scale, which allows different features to be fairly weighted and compared in subsequent matrix operations, avoiding the effect of features with larger absolute values ​​masking the effect of features with smaller values.

[0072] After completing the feature standardization process, the matrix construction module in the network scheduling controller begins to build a multi-dimensional mapping space. The matrix construction module first extracts discrete bandwidth allocation vectors. It should be noted that the discrete bandwidth allocation vector is a pre-defined one-dimensional array composed of multiple discrete bandwidth allocation gradient values, for example, [10Mbps, 20Mbps, 50Mbps, 100Mbps], representing the specific bandwidth quota gradients available for data migration tasks. Subsequently, the matrix construction module sets the candidate transmission path set as the matrix row dimension and the discrete bandwidth allocation vector as the column dimension, establishing a collaborative mapping matrix skeleton. It should be noted that the collaborative mapping matrix skeleton is a two-dimensional table structure, with the number of rows equal to the total number of paths in the candidate transmission path set, and the number of columns equal to the total number of bandwidth gradients in the discrete bandwidth allocation vector. This cross-row and column design logically constructs a joint decision space for path selection and bandwidth allocation, ensuring that each candidate path and each bandwidth quota has a unique correspondence.

[0073] After establishing the collaborative mapping matrix skeleton, the matrix operation unit within the network scheduling controller performs the final data filling operation. The matrix operation unit writes the previously generated standard path feature parameter sequences into the coordinate nodes of the collaborative mapping matrix skeleton according to dimensional relationships. It should be noted that the dimensional relationship refers to the correspondence between the standard path feature parameter sequences and candidate paths. Specifically, the four-dimensional feature parameters (remaining capacity, latency, congestion risk, and service pricing characteristics) belonging to the same candidate path are written into the corresponding coordinate nodes where the rows and columns intersect, based on the path's position in the matrix row dimension and the discrete bandwidth allocation vector's position in the column dimension. It should be noted that the coordinate nodes are the basic storage units in the collaborative mapping matrix skeleton. Since each node corresponds to a combination of "specific path + specific bandwidth," the standard path feature parameter sequence written into that coordinate node actually represents the comprehensive state attributes that the network will exhibit when allocating that specific bandwidth on that specific path. Once all standard path feature parameter sequences have been written into their corresponding coordinate nodes, the joint collaborative mapping matrix of paths and bandwidths is constructed. This matrix completely breaks through the traditional separation of route selection and bandwidth allocation, realizing a deep collaborative mapping between path status and bandwidth quota, and providing a complete mathematical model input for subsequent global optimization solutions.

[0074] In one embodiment of the present invention, the process of solving the joint cooperative mapping matrix to generate the global cooperative allocation result and parsing the target migration path and transmission bandwidth allocation quota of the global cooperative allocation result in step S4 is as follows:

[0075] In this embodiment, the specific implementation process of step S4 first involves the optimization calculation module deployed in the network scheduling controller establishing an objective function with the minimum migration cost as the convergence constraint. Specifically, the network scheduling controller can be a high-performance software-defined network (SDN) controller server based on the x86 architecture, internally configured with a central processing unit and memory for executing the optimization calculation task. Since the row dimension of the joint cooperative mapping matrix corresponds to the set of candidate transmission paths, the column dimension corresponds to the discrete bandwidth allocation vector, and the matrix coordinate nodes store a sequence of standard path feature parameters, the establishment of the objective function depends on the specific feature parameters in this sequence. Let the i-th element in the joint cooperative mapping matrix... Line number The standard path feature parameter sequence corresponding to the coordinate nodes of the column is as follows ,in Indicates time delay characteristics, Indicates congestion risk characteristics. Indicates service pricing characteristics, This represents the characteristics of the remaining capacity. The established objective function is specifically expressed as: It should be noted that, , These are preset weighting coefficients for latency characteristics, congestion risk characteristics, service pricing characteristics, and remaining capacity characteristics, respectively, and satisfy the following conditions: Since increased latency, congestion risk, and service pricing lead to higher migration costs, these three factors are weighted positively. Conversely, increased remaining capacity implies more abundant network resources and lower migration costs, hence they are weighted negatively. The minimum migration cost is used as the convergence constraint, requiring the optimization algorithm to find the minimum cost during the solution process. The coordinate node that obtains the minimum value .

[0076] Subsequently, the network scheduling controller inputs the established objective function and joint cooperative mapping matrix into the optimization algorithm model for solution processing. In this embodiment, the optimization algorithm model specifically adopts the Discrete Particle Swarm Optimization (DPSO) algorithm model, which is deployed in the memory of the network scheduling controller and executed by the central processing unit. The DPSO algorithm model first delineates the feasible solution space in the joint cooperative mapping matrix. It should be noted that the feasible solution space is a set of coordinate nodes that meet the basic transmission requirements of the data migration task; if the discrete bandwidth allocation value corresponding to a coordinate node is less than the minimum tolerable bandwidth of the data migration task, then the coordinate node is removed from the feasible solution space. Within the feasible solution space, the DPSO algorithm model initializes multiple particles, each particle using a two-dimensional position index vector. Encoding is performed to represent the coordinate node position currently explored by the particle. During the iterative convergence operation, each particle is based on the objective function. Calculate the migration cost at its current position, and update the individual's historical best position and the population's global best position; the particle's velocity update formula is: The position update formula is: ,in For inertial weights, and As a learning factor, and for Random numbers between For the individual's historical best position, The optimal position for the entire population. This is the current position. It should be noted that, due to the position index vector... The result must be an integer. Therefore, after each velocity and position update, the calculation result needs to be rounded to ensure that the particle is always mapped to a valid coordinate node of the joint cooperative mapping matrix.

[0077] The network scheduler controller triggers the optimization algorithm model to continuously perform iterative convergence calculations within the feasible solution space. In each iteration, the discrete particle swarm optimization algorithm model recalculates the objective function values ​​of all particles under the new position index vector and determines whether the termination condition of minimizing the migration cost is met. It should be noted that the specific condition of the convergence constraint is: when the migration cost corresponding to the global optimal position of the population is minimized... In each iteration, the change was less than the preset convergence threshold. When, or when the number of iterations reaches the maximum number of iterations. When the algorithm converges, it is determined that the algorithm has converged. For example, a preset convergence threshold is used. It can be set to 0.001, the number of consecutive iterations. Set to 10 iterations, maximum number of iterations The iteration count is set to 200. When the above convergence constraints are met, the iterative convergence operation terminates. The network scheduler extracts the coordinate node corresponding to the current global optimal position of the population and uses this coordinate node as the discrete optimal solution that satisfies the objective function. The network scheduler encapsulates the position index vector, discrete bandwidth allocation value, candidate transmission path identifier, and corresponding minimum migration cost corresponding to this discrete optimal solution, and combines them to generate a global collaborative allocation result. The global collaborative allocation result is temporarily stored in the network scheduler's memory cache in the form of structured data, awaiting subsequent calls.

[0078] After generating the global coordinated allocation result, the network scheduler controller calls its built-in parsing component to read the result. This parsing component is specifically a data decoding program running in the network scheduler controller's central processing unit. Since the core data in the global coordinated allocation result is a position index vector encoded by the optimization algorithm model, the parsing component performs reverse decoding on the data structure of the result, splitting the encapsulated data fields to extract the position index vector representing the route direction. It should be noted that the position index vector is a two-dimensional vector. The first dimension index The first dimension represents the row dimension, i.e., the sorting number of the candidate transmission path within the candidate transmission path set; the second dimension is the index. This represents the column dimension, i.e., the sorting number of the discrete bandwidth allocation value in the discrete bandwidth allocation vector. The reverse decoding process is the process of restoring the compressed encoded integer index to an independent dimension, thereby decoupling the routing direction from the physical meaning of the bandwidth quota.

[0079] Finally, the network scheduling controller maps the location index vectors representing the route direction, separated by the parsing component, to the candidate transmission path set. Since the candidate transmission path set is stored sequentially in the controller as an array or linked list, the first dimension of the location index vector... Directly corresponds to the first in the set Given several candidate transmission paths, the network scheduling controller locks the corresponding target migration path through direct addressing. It should be noted that the target migration path is the final determined physical link sequence through which the data packets are actually forwarded, containing information about the source node, destination node, and all switching nodes along the way. Simultaneously, the network scheduling controller extracts the transmission bandwidth allocation quota carried in the global collaborative allocation result, which is indexed by the second dimension of the bit index vector. This is obtained by mapping to the discrete bandwidth allocation vector. It should be noted that the transmission bandwidth allocation quota is the [number]th [unit / vector]. Each discrete bandwidth allocation value represents the allowed data migration path for the monthly target migration task. The exact bandwidth ratio used. The network scheduling controller completed the entire process from solving the joint cooperative mapping matrix to extracting physical network resource parameters, obtaining the target migration path and transmission bandwidth allocation quota that can be directly used to issue forwarding flow tables.

[0080] In one embodiment of the present invention, step S5, which involves injecting the data migration task into the target migration path according to the transmission bandwidth allocation quota for transmission operation, and continuously collecting the network state variation parameters of the target migration path in real time, is as follows:

[0081] In this embodiment, the specific implementation process of step S5 begins with the injection and transmission of the data migration task. After parsing the target migration path and transmission bandwidth allocation quota, the network scheduling controller sends flow table entries and rate limiting configuration commands to the source edge switch through the software-defined network southbound interface. Specifically, the source edge switch can be a programmable white-box switch that supports the OpenFlow protocol. After receiving the command, it sets its own outgoing port rate according to the transmission bandwidth allocation quota. It should be noted that setting the port rate according to the transmission bandwidth allocation quota means using the token bucket algorithm built into the switch chip to precisely lock the upper limit of the outgoing port's sending rate to the bandwidth value specified by the transmission bandwidth allocation quota, thereby ensuring that the data migration task does not over-occupy network link resources. Subsequently, the data plane processing logic of the source edge switch divides the overall data stream of the data migration task received at the input end into fixed-length packets. It should be noted that dividing the data migration task into packets is a basic process of network transmission. Typically, large data blocks are divided into standard Ethernet data frames containing the source Internet Protocol address, the destination Internet Protocol address, and the payload according to the network's maximum transmission unit. Finally, the source edge switch injects these segmented packets into the target migration path at the set port rate to perform transmission operations, so that the data flow strictly follows the predetermined route and bandwidth quota to advance to the destination.

[0082] When a packet undergoes transmission on the target migration path, the network scheduling controller activates telemetry probes based on the target migration path. It's important to note that telemetry probes are network status monitoring mechanisms based on in-band network telemetry technology. They instruct network devices on the target migration path to record their own status information by inserting specific command headers into normally transmitted data packets. Specifically, the network scheduling controller sends telemetry acquisition strategies to each hop node deployed along the target migration path. These nodes can be backbone routers or core switches that support in-band network telemetry. When probe packets or service packets carrying command headers pass through these nodes, the switching chips at these nodes automatically and continuously capture real-time snapshots of the current node's operation. It's worth noting that an operation snapshot is an instantaneous record of the network device's operational status within a very short time slice, containing underlying data such as port queue depth, link transient utilization, and outgoing port timestamps. After receiving the returned packet, the network scheduling controller strips and parses the command header data to extract the basic link characteristics carried by the operation snapshot. It should be noted that basic link characteristics refer to raw state indicators that are read directly from the runtime snapshot without secondary calculations, such as the instantaneous available bandwidth of the current link and the queuing delay experienced by the current packet. These basic link characteristics provide the most realistic underlying data support for subsequent dynamic assessment of network status.

[0083] After extracting the basic link features, the timing analysis module inside the network scheduling controller begins dynamic variation analysis. The network scheduling controller compares the basic link features of adjacent timing sequences. It should be noted that adjacent timing sequences refer to two consecutive telemetry acquisition cycles, such as the previous acquisition cycle and the current acquisition cycle. The comparison process involves performing a difference calculation on the basic link features of the same link segment extracted in these two cycles. Through the difference calculation, the network scheduling controller analyzes the bandwidth fluctuation value and latency spike increase value of the target migration path. It should be noted that the bandwidth fluctuation value refers to the decrease in available bandwidth in the current timing sequence compared to the previous timing sequence. It is calculated by subtracting the available bandwidth in the previous timing sequence from the available bandwidth in the basic link features. The bandwidth fluctuation value reflects the degree of attenuation of the remaining link capacity. The latency spike increase value refers to the increase in total latency in the current timing sequence compared to the previous timing sequence. It is calculated by subtracting the latency value in the previous timing sequence from the latency value in the basic link features of the current timing sequence. The latency spike increase value reflects the deterioration of link congestion. Finally, the network scheduling controller concatenates the calculated fluctuation value and the sudden increase value according to a preset data structure to generate a network state variation parameter. It should be noted that concatenating the fluctuation value and the sudden increase value to generate the network state variation parameter means combining these two independent scalar values ​​into a two-dimensional vector. This network state variation parameter, as a holistic indicator, intuitively and comprehensively quantifies the degree of state deviation of the target migration path at a micro-time scale, providing a precise triggering basis for incremental replanning in subsequent steps.

[0084] In one embodiment of the present invention, step S6, the process of triggering incremental replanning instructions based on network state variation parameters and iteratively correcting the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data, is as follows:

[0085] In this embodiment, the specific implementation process of step S6 begins with the determination of network state anomalies and the triggering of replanning. After generating network state variation parameters in step S5, the network scheduling controller inputs them to a comparator deployed inside the network scheduling controller. It should be noted that the comparator is a logic judgment circuit or software judgment module running in the central processing unit of the network scheduling controller. It has preset boundaries, which are set by the network administrator according to the network's service level agreement and tolerance, specifically including bandwidth fluctuation threshold and latency spike increase threshold. The comparator compares the bandwidth fluctuation value and latency spike increase value in the received network state variation parameters with the corresponding thresholds. When the value of either dimension exceeds the corresponding preset boundary, it is determined that the transmission quality of the target migration path has deteriorated. At this time, the network scheduling controller immediately triggers an incremental replanning instruction. It should be noted that the incremental replanning instruction is an internal system event interruption signal used to notify the optimization engine to suspend regular monitoring and start the dynamic adjustment process. At the same time as triggering the incremental replanning instruction, the network scheduling controller sends a status query request to all programmable switches and routers in the network through the software-defined network southbound interface to extract the latest network snapshot data of the entire network. It should be noted that the latest network snapshot data refers to the collection of real-time status information such as load, available bandwidth, and queuing latency of all network nodes and links collected at the current moment, which reflects the latest global resource distribution of the network.

[0086] After triggering the incremental replanning instruction and obtaining the latest network snapshot data for the entire network, the network scheduling controller begins to execute iterative correction operations based on real-time data. The optimization engine within the network scheduling controller injects the latest network snapshot data into the joint cooperative mapping matrix constructed in step S3, according to the incremental replanning instruction, to update the link parameters. It should be noted that updating the link parameters means using information such as real-time remaining capacity and real-time latency contained in the latest network snapshot data to replace the original standard path feature parameter sequences of the corresponding coordinate nodes in the joint cooperative mapping matrix, ensuring that the network state described by the joint cooperative mapping matrix is ​​completely synchronized with the current real state of the physical network. Subsequently, the network scheduling controller triggers the optimization algorithm model to recalculate based on the updated joint cooperative mapping matrix. It should be noted that, unlike the full initialization calculation in step S4, the recalculation adopts an incremental local optimization strategy, that is, using the currently executing global cooperative allocation result as the initial feasible solution of the optimization algorithm model, rather than randomly generating a new initial population. This can significantly shorten the convergence time of the algorithm and meet the real-time response requirements of data migration tasks to network mutations. The optimization algorithm model performs fast iterative convergence operations within the feasible solution space of the updated joint cooperative mapping matrix, searching for the discrete optimal solution that minimizes the migration cost under the new network state, thereby iteratively correcting the target migration path and transmission bandwidth allocation quota. Finally, the network scheduling controller reissues the corrected target migration path and transmission bandwidth allocation quota to the underlying network devices, enabling the data migration task to smoothly switch to the new path and bandwidth quota and continue execution, avoiding migration task timeout failures due to network state deterioration.

[0087] Please see Figure 2 As shown, the present invention also provides a path and bandwidth collaborative allocation system for data task migration, comprising:

[0088] The requirement building module is used to extract the migration data volume, deadline, and business priority parameters of the input data migration task, and to establish the initial bandwidth requirement feature vector of the data migration task.

[0089] The path filtering module is used to integrate the initial bandwidth requirement feature vector with the existing service occupancy status of network links, eliminate restricted links, and generate a set of candidate transmission paths that meet protection constraints.

[0090] The matrix construction module is used to extract the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and construct a joint cooperative mapping matrix of paths and bandwidth.

[0091] The collaborative allocation module is used to solve the joint collaborative mapping matrix to generate global collaborative allocation results, and to parse the target migration path and transmission bandwidth allocation quota of the global collaborative allocation results.

[0092] The transmission acquisition module is used to inject the data migration task into the target migration path according to the transmission bandwidth allocation quota for transmission operation, and to continuously collect the network status variation parameters of the target migration path in real time.

[0093] The incremental planning module is used to trigger incremental replanning instructions based on network state variation parameters, and iteratively correct the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data.

[0094] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the execution of the program on the processor implements the steps in the path and bandwidth collaborative allocation method for data task migration as described above.

[0095] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps in the path and bandwidth collaborative allocation method for data task migration as described above.

[0096] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the modules described above can be referred to the corresponding process in the aforementioned method implementation, and will not be repeated here.

[0097] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0098] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0099] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer system (which may be a personal computer, server, or network system, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A method for path and bandwidth co-allocation for data task migration, characterized in that, Includes the following steps: S1: Extract the migration data volume, deadline, and business priority parameters of the input data migration task, and establish the initial bandwidth requirement feature vector of the data migration task. S2: Integrate the initial bandwidth requirement feature vector with the existing service occupancy status of network links, eliminate restricted links, and generate a set of candidate transmission paths that meet protection constraints; S3: Extract the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and construct a joint collaborative mapping matrix of path and bandwidth; S4: Solve the joint cooperative mapping matrix to generate the global cooperative allocation result, and analyze the target migration path and transmission bandwidth allocation quota of the global cooperative allocation result; S5: Inject the data migration task into the target migration path according to the transmission bandwidth quota for transmission operation, and continuously collect the network status variation parameters of the target migration path in real time. S6: Trigger incremental replanning instructions based on network state variation parameters, and iteratively correct the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data. 2.The method of claim 1, wherein, In step S1, the process of extracting the migration data volume, deadline, and business priority parameters of the input data migration task, and establishing the initial bandwidth requirement feature vector of the data migration task, is as follows: Intercept the data migration task request from the input end, parse the description file of the data migration task request, and extract the initial values ​​of the migration data volume, deadline, and business priority parameters. The initial values ​​of the migration data volume, deadline, and business priority parameters are input into the dimension elimination function to generate standardized values ​​within the same metric space. The standardized values ​​are linearly concatenated and spliced ​​according to the preset dimensional order to establish the initial bandwidth requirement feature vector for the data migration task.

3. The method of claim 1, wherein, In step S2, the process of fusing the initial bandwidth demand feature vector with the existing service occupancy status of network links, eliminating restricted links, and generating a set of candidate transmission paths that meet protection constraints is as follows: The expected bandwidth requirement of the data migration task is parsed from the initial bandwidth requirement feature vector; the existing service occupancy status of the network links is extracted, and the existing service occupancy status is added to the expected bandwidth requirement value to generate the predicted total load value of each network link. The predicted total load value is compared with the protection constraint parameter. Network links whose predicted total load value is greater than the protection constraint parameter are marked as restricted links and removed. Search for the remaining network links after removing restricted links, extract all connected branches between nodes, and merge all connected branches to generate a set of candidate transmission paths that satisfy the protection constraints.

4. The method of claim 1, wherein, In step S3, the process of extracting the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and constructing a joint cooperative mapping matrix of paths and bandwidth, is as follows: Traverse the topology of the candidate transmission path set, collect the remaining capacity, latency, congestion risk and service pricing characteristics of the corresponding path, and generate the original feature dataset. The original feature dataset is input into the normalization model, which performs a dimensionless operation to transform and generate a standard path feature parameter sequence with a uniform metric. Extract the discrete bandwidth allocation vector, set the candidate transmission path set as the matrix row dimension, set the discrete bandwidth allocation vector as the column dimension, and establish the skeleton of the cooperative mapping matrix; The standard path feature parameter sequence is written into the coordinate nodes of the collaborative mapping matrix skeleton according to the dimensional relationship, and a joint collaborative mapping matrix of path and bandwidth is constructed.

5. The method for collaborative allocation of path and bandwidth for data task migration according to claim 1, characterized in that, In step S4, the process of solving the joint cooperative mapping matrix to generate the global cooperative allocation result, and parsing the target migration path and transmission bandwidth allocation quota of the global cooperative allocation result is as follows: Establish an objective function with the minimum migration cost as the convergence constraint, and input the objective function and the joint cooperative mapping matrix into the optimization algorithm model to perform the solution process; The optimization algorithm model is triggered to perform iterative convergence operations in the feasible solution space, extract the discrete optimal solutions that satisfy the objective function, and combine the discrete optimal solutions to generate a global collaborative allocation result; The parsing component is invoked to read the global collaborative allocation result, and the data structure of the global collaborative allocation result is reverse-decoded to separate the position index vector representing the route direction; Map the location index vector to the candidate transmission path set, lock the corresponding target migration path, and extract the transmission bandwidth allocation quota carried by the global collaborative allocation result.

6. The method for collaborative allocation of path and bandwidth for data task migration according to claim 1, characterized in that, In step S5, the process of injecting the data migration task into the target migration path according to the transmission bandwidth quota for transmission operation, and continuously collecting the network status variation parameters of the target migration path in real time, is as follows: Set the port rate according to the transmission bandwidth allocation quota, divide the data migration task into packets, and inject the packets into the target migration path according to the set port rate to perform the transmission operation. Based on the target migration path for performing transmission operations, a telemetry probe is activated to continuously capture operational snapshots of nodes along the route in real time and extract the basic link features carried by the operational snapshots. By comparing the basic link characteristics of adjacent time series, the bandwidth fluctuation value and latency spike value of the target migration path are analyzed, and the fluctuation value and spike value are concatenated to generate network state variation parameters.

7. The method for collaborative allocation of path and bandwidth for data task migration according to claim 1, characterized in that, In step S6, the process of triggering incremental replanning instructions based on network state variation parameters and iteratively correcting the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data is as follows: The network state variation parameter is input into the comparator. When the network state variation parameter exceeds the preset boundary, an incremental replanning instruction is triggered, and the latest network snapshot data of the entire network is extracted at the same time. According to the incremental replanning instruction, the latest network snapshot data is injected into the joint collaborative mapping matrix to update the link parameters, and the target migration path and transmission bandwidth allocation quota are recalculated and iteratively corrected.

8. A path and bandwidth collaborative allocation system for data task migration, implemented in any one of claims 1-7, characterized in that, include: The requirement building module is used to extract the migration data volume, deadline, and business priority parameters of the input data migration task, and to establish the initial bandwidth requirement feature vector of the data migration task. The path filtering module is used to integrate the initial bandwidth requirement feature vector with the existing service occupancy status of network links, eliminate restricted links, and generate a set of candidate transmission paths that meet protection constraints. The matrix construction module is used to extract the remaining capacity, latency, congestion risk, and service pricing characteristics of the candidate transmission path set, and construct a joint cooperative mapping matrix of paths and bandwidth. The collaborative allocation module is used to solve the joint collaborative mapping matrix to generate global collaborative allocation results, and to parse the target migration path and transmission bandwidth allocation quota of the global collaborative allocation results. The transmission acquisition module is used to inject the data migration task into the target migration path according to the transmission bandwidth allocation quota for transmission operation, and to continuously collect the network status variation parameters of the target migration path in real time. The incremental planning module is used to trigger incremental replanning instructions based on network state variation parameters, and iteratively correct the target migration path and transmission bandwidth allocation quota based on the latest network snapshot data.

9. An electronic device, comprising a memory and a processor, characterized in that, The memory stores a computer program that can run on the processor, and when the program is executed on the processor, it implements the steps in the path and bandwidth collaborative allocation method for data task migration as described in any one of claims 1-7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the path and bandwidth collaborative allocation method for data task migration as described in any one of claims 1-7.

Images