Virtual node dynamic resource allocation method and system based on edge cloud collaboration

Abstract

The invention relates to a virtual node dynamic resource allocation method and system based on edge cloud collaboration, and the method comprises the following steps: collecting historical resource use data of all virtual nodes, and carrying out the preprocessing of the historical resource use data, and constructing a virtual node historical resource use data set; constructing a virtual node load prediction model, training the model through the virtual node historical resource use data set, and predicting the load of the virtual node through the trained virtual node load prediction model; and allocating virtual nodes to the calculation task based on the virtual node prediction load and the calculation task characteristics.

Description

Technical Field

[0001] The present invention relates to a method and system for dynamic resource allocation of virtual nodes based on edge-cloud collaboration, and belongs to the field of edge computing technology. Background Art

[0002] In the current computing and communication environment, the combination of edge computing and cloud computing has become the main architecture in many application scenarios. This edge-cloud collaborative architecture enables computing resources to be closer to the data source, improving the efficiency and real-time performance of data processing. However, existing edge-cloud collaborative technologies have some shortcomings in terms of dynamic resource allocation, task scheduling, and load prediction.

[0003] Static resource allocation: Most existing resource allocation strategies are static, based on preset rules and strategies, and fail to reflect changes in task requirements and node status in real time, which leads to waste of resources and reduced task execution efficiency.

[0004] Lack of adaptive capabilities: Existing methods often fail to adjust their strategies based on real-time data and lack flexibility. When load increases or network status changes, the system's response speed and adaptability are limited.

[0005] Ignoring the collaboration between nodes: In the process of resource allocation and task scheduling between edge nodes, existing solutions often ignore the collaboration between nodes, resulting in the overall system performance not being optimal. Summary of the invention

[0006] In order to solve the above-mentioned problems existing in the prior art, the present invention proposes a method and system for dynamic resource allocation of virtual nodes based on edge-cloud collaboration.

[0007] The technical solution of the present invention is as follows:

[0008] On the one hand, the present invention provides a method for dynamic resource allocation of virtual nodes based on edge-cloud collaboration, comprising the following steps:

[0009] Collect the historical resource usage data of all virtual nodes, and construct it into a virtual node historical resource usage dataset after preprocessing;

[0010] Build a virtual node load prediction model, train the model using the virtual node historical resource usage data set, and predict the load of the virtual node using the trained virtual node load prediction model;

[0011] Based on the predicted load of the virtual nodes and the characteristics of the computing tasks, virtual nodes are allocated to the computing tasks.

[0012] As a preferred implementation of the present invention, the virtual node load prediction model is constructed based on the LSTM model.

[0013] As a preferred implementation manner of the present invention, the computing task characteristics include computing complexity, delay tolerance, data size and priority.

[0014] As a preferred implementation of the present invention, the steps of allocating computing task virtual nodes are as follows:

[0015] For any computing task, calculate its execution cost and future resource usage on each virtual node;

[0016] Construct an optimization objective function with the goal of minimizing delay and load, as shown in the following formula:

[0017]

[0018] Where: M represents the number of virtual nodes; N represents the number of computing tasks; C m represents the execution overhead when the nth computing task is assigned to the mth virtual node; R mn represents the future resource usage rate when the nth computing task is assigned to the mth virtual node; λ represents the empirical parameter;

[0019] Set constraints based on the predicted load of the virtual node, as shown in the following formula:

[0020] Subject to:L m (t)+R mn <L max

[0021] Where: L m (t) represents the predicted load of virtual node m at time t; L max Indicates the preset load threshold of the virtual node;

[0022] The best virtual node for each computing task is calculated based on the above optimization objective function and constraints.

[0023] As a preferred embodiment of the present invention, the execution cost C of any computing task in each virtual node is m The calculation formula is:

[0024]

[0025] Among them: J n represents the computational complexity of computing task n; k represents the processing time of the current computing task; L m (t+k) represents the predicted load of virtual node m at time t+k; R m represents the remaining available resources of virtual node m; γ represents the empirical parameter; P n Indicates the priority of computing task n; D nRepresents the delay tolerance of computing task n.

[0026] As a preferred embodiment of the present invention, the future resource usage rate R of any computing task in each virtual node mn The calculation formula is:

[0027]

[0028] Among them: U m U (t) represents the resource utilization rate of virtual node m at time t; task (t) represents the number of executable tasks of virtual node m at time t; T n Indicates the data size of computing task n; H m Represents the historical resource usage pattern of virtual node m.

[0029] On the other hand, the present invention also provides a virtual node dynamic resource allocation system based on edge-cloud collaboration, including a data acquisition module, a load prediction module and a resource allocation module;

[0030] The data collection module is used to collect the historical resource usage data of all virtual nodes, and construct it into a virtual node historical resource usage data set after preprocessing;

[0031] The load prediction module is used to construct a virtual node load prediction model, train the model through a virtual node historical resource usage data set, and predict the load of the virtual node through the trained virtual node load prediction model;

[0032] The resource allocation module is used to allocate virtual nodes to computing tasks based on the predicted load of the virtual nodes and the computing task characteristics.

[0033] On the other hand, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0034] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0035] The present invention has the following beneficial effects:

[0036] 1. The present invention introduces multimodal data and deep learning technology, comprehensively considers multi-dimensional factors such as task history, node status, network delay, etc., which can greatly improve the accuracy of load prediction and thus optimize task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0040] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0041] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0042] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0043] Embodiment 1:

[0044] See also Figure 1 , a virtual node dynamic resource allocation method based on edge-cloud collaboration, comprising the following steps:

[0045] Collect the historical resource usage data of all virtual nodes, and construct it into a virtual node historical resource usage dataset after preprocessing;

[0046] Build a virtual node load prediction model, train the model using the virtual node historical resource usage data set, and predict the load of the virtual node using the trained virtual node load prediction model;

[0047] Based on the predicted load of the virtual nodes and the characteristics of the computing tasks, virtual nodes are allocated to the computing tasks.

[0048] As a preferred implementation of this embodiment, the virtual node load prediction model is constructed based on the LSTM model.

[0049] As a preferred implementation of this embodiment, the computing task characteristics include computing complexity (describing the computing amount of the task, the unit can be computing cycle or FLOPS), delay tolerance (describing the maximum delay that the task can tolerate), data size and priority.

[0050] As a preferred implementation of this embodiment, the steps of allocating computing tasks to virtual nodes are as follows:

[0051] For any computing task, calculate its execution cost and future resource usage on each virtual node;

[0052] Construct an optimization objective function with the goal of minimizing delay and load, as shown in the following formula:

[0053]

[0054] Where: M represents the number of virtual nodes; N represents the number of computing tasks; C mn represents the execution overhead when the nth computing task is assigned to the mth virtual node; R mn represents the future resource usage rate when the nth computing task is assigned to the mth virtual node; λ represents the empirical parameter;

[0055] Set constraints based on the predicted load of the virtual node, as shown in the following formula:

[0056] Subject to:L m (t)+R mn <L max

[0057] Where: L m (t) represents the predicted load of virtual node m at time t; L max Indicates the preset load threshold of the virtual node;

[0058] The best virtual node for each computing task is calculated based on the above optimization objective function and constraints.

[0059] As a preferred implementation of this embodiment, the execution cost C of any computing task on each virtual node is mn The calculation formula is:

[0060]

[0061] Among them: J n represents the computational complexity of computing task n; k represents the processing time of the current computing task; L m (t+k) represents the predicted load of virtual node m at time t+k; R m represents the remaining available resources of virtual node m; γ represents the empirical parameter; Pn Indicates the priority of computing task n; D n Represents the delay tolerance of computing task n.

[0062] As a preferred implementation of this embodiment, the future resource usage rate R of any computing task in each virtual node mn The calculation formula is:

[0063]

[0064] Among them: U m (t) represents the resource utilization rate of virtual node m at time t; U task (t) represents the number of executable tasks of virtual node m at time t; T n Indicates the data size of computing task n; H m Represents the historical resource usage pattern of virtual node m.

[0065] Embodiment 2:

[0066] A virtual node dynamic resource allocation system based on edge-cloud collaboration, including a data acquisition module, a load prediction module and a resource allocation module;

[0067] The data collection module is used to collect the historical resource usage data of all virtual nodes, and construct it into a virtual node historical resource usage data set after preprocessing;

[0068] The load prediction module is used to construct a virtual node load prediction model, train the model through a virtual node historical resource usage data set, and predict the load of the virtual node through the trained virtual node load prediction model;

[0069] The resource allocation module is used to allocate virtual nodes to computing tasks based on the predicted load of the virtual nodes and the computing task characteristics.

[0070] The system is used to implement the method in Example 1, which will not be described in detail here.

[0071] Embodiment three:

[0072] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0073] Embodiment 4:

[0074] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present invention is implemented.

[0075] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0076] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0077] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0078] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0079] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for dynamic resource allocation of virtual nodes based on edge-cloud collaboration, characterized in that: The following steps are involved: Collect the historical resource usage data of all virtual nodes, and construct it into a virtual node historical resource usage dataset after preprocessing; Build a virtual node load prediction model, train the model using the virtual node historical resource usage data set, and predict the load of the virtual node using the trained virtual node load prediction model; Based on the predicted load of the virtual nodes and the characteristics of the computing tasks, virtual nodes are allocated to the computing tasks.

2. According to the method of claim 1, a virtual node dynamic resource allocation method based on edge-cloud collaboration is characterized in that: The virtual node load prediction model is built based on the LSTM model.

3. According to the method of claim 1, a virtual node dynamic resource allocation method based on edge-cloud collaboration is characterized in that: The computing task characteristics include computing complexity, delay tolerance, data size, and priority.

4. According to the method of claim 3, a method for dynamic resource allocation of virtual nodes based on edge-cloud collaboration is characterized in that: The steps of allocating computing tasks to virtual nodes are as follows: For any computing task, calculate its execution cost and future resource usage on each virtual node; Construct an optimization objective function with the goal of minimizing delay and load, as shown in the following formula: Where: M represents the number of virtual nodes; N represents the number of computing tasks; C m represents the execution overhead when the nth computing task is assigned to the mth virtual node; R mn represents the future resource usage rate when the nth computing task is assigned to the mth virtual node; λ represents the empirical parameter; Set constraints based on the predicted load of the virtual node, as shown in the following formula: Subject to:L m (t)+R mn <L max Where: L m (t) represents the predicted load of virtual node m at time t; L max Indicates the preset load threshold of the virtual node; The best virtual node for each computing task is calculated based on the above optimization objective function and constraints.

5. According to the method of claim 4, a method for dynamic resource allocation of virtual nodes based on edge-cloud collaboration is characterized in that: The execution cost C of any computing task on each virtual node mn The calculation formula is: Among them: J n represents the computational complexity of computing task n; k represents the processing time of the current computing task; L m (t+k) represents the predicted load of virtual node m at time t+k; R m represents the remaining available resources of virtual node m; γ represents the empirical parameter; P n Indicates the priority of computing task n; D n Represents the delay tolerance of computing task n.

6. According to the method of claim 4, a method for dynamic resource allocation of virtual nodes based on edge-cloud collaboration is characterized in that: The future resource usage rate R of any computing task on each virtual node mn The calculation formula is: Among them: U m (t) represents the resource utilization rate of virtual node m at time t; U task (t) represents the number of executable tasks of virtual node m at time t; T n Indicates the data size of computing task n; H m Represents the historical resource usage pattern of virtual node m.

7. A virtual node dynamic resource allocation system based on edge-cloud collaboration, characterized in that: It includes data collection module, load prediction module and resource allocation module; The data collection module is used to collect the historical resource usage data of all virtual nodes, and construct it into a virtual node historical resource usage data set after preprocessing; The load prediction module is used to construct a virtual node load prediction model, train the model through a virtual node historical resource usage data set, and predict the load of the virtual node through the trained virtual node load prediction model; The resource allocation module is used to allocate virtual nodes to computing tasks based on the predicted load of the virtual nodes and the computing task characteristics.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images