A CPU resource allocation optimization method within the knowledge distillation cycle based on edge computing

Abstract

A method for optimizing CPU resource allocation within a knowledge distillation cycle based on edge computing includes the following steps: establishing a digital model of the time dependency between communication and computation in the edge environment KD, constructing a parallel scenario for communication and computation tasks throughout the KD cycle, proposing a strategy for optimizing time management, and evaluating the effectiveness of the optimization method in KD based on an actual edge environment. This method deeply studies the complex challenges of the KD process in the context of edge computing, and provides unique insights into the impact of CPU core allocation on the speed of communication and computation on edge devices with limited resources. It emphasizes the ideal dependencies and constraints in edge-based KD, and introduces a dynamic resource allocation solution, changing key parameters such as the number of CPU cores and the amount of data through simulation experiments. Compared with traditional strategies, the method of the present invention has good robustness, and the constructed framework can provide further exploration and innovation for optimizing edge computing operations.

Description

Technical Field

[0001] The present invention relates to the field of knowledge distillation based on edge computing. Specifically, it introduces a dynamic resource management method to perform an approximately optimal resource scheduling method for the limited communication and computing resources in the knowledge distillation process of the edge environment, thereby improving the operating efficiency of the edge model. Background Art

[0002] Model personalization in edge computing is crucial for edge devices to solve practical local tasks. However, it also introduces its own drawbacks, such as model drift, where model performance degrades due to changes in the local environment. Although large-scale models are better able to adapt to environmental changes, their computational requirements often exceed the available resources on edge devices. Federated learning, as a collaborative approach, mitigates model drift by training on local data and aggregating updates across multiple devices. However, it requires extensive communication resources and consistent client participation, which can be problematic in resource-constrained edge environments and pose challenges to model quality and security.

[0003] In response to the limitations of the two aforementioned approaches to addressing model drift on edge devices, knowledge distillation (KD), as an effective model compression method, provides a viable approach to address these issues. KD involves transferring knowledge from comprehensive, complex models to streamlined, edge-adapted models, thereby preserving complex model performance within the capabilities of edge devices. This provides an excellent solution for model personalization and model drift.

[0004] While addressing the aforementioned limitations, the KD process also presents its own set of challenges, particularly in resource-constrained edge environments, where the competition between computation and communication becomes more pronounced. In resource-constrained edge environments, this competition becomes more pronounced as edge devices struggle to balance the ingestion of distilled knowledge from larger, more complex models with the computational demands of local training programs. This balancing act can lead to resource bottlenecks that potentially undermine the benefits of KD: slower communication due to contention for data transfer, or delayed computation due to overscaling processing power.

[0005] In the context of knowledge distillation in edge computing, communication tasks involve transferring data between teacher and student models, while computation tasks include model training and inference. Competition for CPU resources during these tasks can lead to inefficient student model training performance. Therefore, optimizing CPU resource scheduling in edge knowledge distillation is crucial for improving communication and computational efficiency in edge devices. Summary of the Invention

[0006] In order to overcome the deficiencies in the above-mentioned background technology, the present invention proposes a CPU resource allocation optimization method within the knowledge distillation cycle based on edge computing.

[0007] This paper explores the resource contention problem in edge computing, focusing on the complex balance between communication and computing in resource-limited devices. The goal is to optimally configure communication resources and computing resources in edge computing, maximize the time savings of the knowledge distillation process, and improve system efficiency.

[0008] In order to achieve the above technical objectives, the present invention provides the following technical solutions:

[0009] A method for optimizing CPU resource allocation within a knowledge distillation cycle based on edge computing includes the following steps:

[0010] (1) Establish a digital model for the temporal dependency between communication and computation by knowledge distillation in edge environments;

[0011] (2) Construct a parallel scenario of communication and computing tasks in the iterative process of knowledge distillation in edge computing, and propose a time optimization task within the entire KD cycle;

[0012] (3) Propose a strategy to optimize time management: adopt a framework based on dynamic resource allocation to achieve a near-optimal solution;

[0013] (4) Simulation experiments are conducted to evaluate the effectiveness of the optimization method in KD based on actual edge environments.

[0014] Furthermore, in step (1), considering the impact of CPU cores on the total time scheduling required for knowledge distillation, a series of control experiments were conducted on Raspberry Pi and Jetson Xavier NX platforms to quantitatively evaluate the changes in the time required for communication and computing tasks under different CPU core allocations. Based on the experimental results, the following function fitting representing the relationship between communication time and computing time and CPU core allocation was proposed:

[0015] ① Computation: The time required for computing tasks in edge devices depends on the allocation of CPU cores. The relationship between the required computing time and the number of CPU cores allocated to communication can be approximated by the following function:

[0016]

[0017] where f(x,c) is the computation time required to compute workload x on the c CPU cores allocated to the communication task. The constants a1 and a2 reflect the effect of CPU allocation on computation time. In this equation, computation time increases as the number of CPU cores available for communication tasks increases. This highlights the linear reduction in computational resources as more CPU cores are allocated away from computation tasks.

[0018] ② Communication: Communication tasks in edge devices have diminishing marginal returns in terms of CPU core allocation. The time required for communication tasks can be approximated by the following function:

[0019]

[0020] Here, g(y,c) represents the time required for a communication task with data size y and c CPU cores allocated for communication. b1·c indicates that communication speed increases linearly with the number of cores, while b2·log(b3·c) indicates that the improvement decreases logarithmically with the number of allocated cores. The constants b1, b2, and b3 define the specific characteristics of this relationship. This formula shows that initially, increasing the number of cores c dedicated to communication significantly reduces communication time, but after a certain point, the effect becomes less pronounced.

[0021] Furthermore, in step (2), it is assumed that the iterative process of knowledge distillation in the edge computing environment involves N cycles of Comm and Comp tasks, each consisting of a Forward and a Backward task. The parallel execution of Comm and Comp tasks highlights the inherent resource contention and its impact on the speed of the two processes. Therefore, a time optimization task around N cycles is proposed, specifically including:

[0022] ① Propose constraints: Consider that Comm is a continuous process. The distillation temperature coefficient T transmits logit data from the cloud, and the calculation involves forward and backward propagation, where backward depends on logits. The following constraints are used to construct a time optimization problem:

[0023] 1) Comm is uninterrupted, and logits are transmitted from the teacher model in the cloud, which requires continuous allocation of resources for data transmission until the communication task is completed;

[0024] 2) Each Backward in the Comp sequence depends on the logits of the corresponding Comm cycle. If Comm has not yet completed, the Backward must wait, resulting in a delay, such as t″;

[0025] 3) When time is greater than t end When , it means that all Comm tasks have been completed. At this time, all computing resources will be allocated to Comp tasks, that is, there are only Comm tasks in stage t″;

[0026] ② Time calculation: In the complex coordination of the system Comm and Comp, time calculation is a key factor, especially when considering the interdependencies between various calculation processes. To facilitate a thorough analysis, a stage-based approach is adopted, dividing the timeline before each reverse calculation. Assuming there are N reverse calculation instances, N+1 different time periods are determined for analysis:

[0027] 1) Figure 1 The first time period in is denoted as t1. Indicates the computational load of forwarding 1, It represents the communication load of the first communication cycle Comm. The duration of t1 can be expressed as:

[0028]

[0029] In this formula, and They represent the functions of quantizing the forward propagation completion time and the communication cycle, respectively. It means in After completion, reallocate the number of cores Complete the remaining calculations The time required. The criterion for determining the duration of t1 depends on whether the forward propagation time is less than the communication time. It is worth noting that if If it is smaller, it means that the subsequent communication cycle (for i>1) will overlap with Forward1;

[0030] 2) For t2 to t N Consider three cases. The first case, shown as t2, consists of overlapping time t′ and computation time t″. The second case, shown as t3, indicates that communication and computation are completed in parallel. The third case, shown as t N As shown, this occurs when there is no communication task, resulting in the reallocation of kernel resources to favor computing tasks. The time periods for the three cases are shown below:

[0031] 1. The first scenario: Communication and computing tasks overlap. For a general time period t i , which is expressed as follows:

[0032]

[0033]

[0034] 2. The second scenario: Communication and computation are completed in parallel. For a general time period t i , which is expressed as follows:

[0035]

[0036]

[0037] 3. The third scenario: All communication tasks have been completed, and only the computing task remains. In this case, all resources can be redistributed to Comp. Then, for time period t i , which is expressed as follows:

[0038]

[0039] 3) For the last time period t N+1 , since there is no additional Comm task, the required time is dedicated to BackwardN. It can be expressed mathematically as follows:

[0040]

[0041] Total computing time T total , which represents the total duration of the KD process across N segments of Comm and Comp tasks. total The optimization of T depends on an effective CPU core resource allocation strategy, the main goal of which is to minimize T by optimizing the core utilization of each segment. total , ensuring that from t1 to t N The sum of the time is as short as possible. This concept can be expressed mathematically as:

[0042]

[0043] In this formula, t i (c i ) indicates that on a given CPU core c i The time spent in segment i under the allocation condition.

[0044] In the step (3), for the three communication tasks and computing task scenarios within the knowledge distillation cycle constructed in step (2), dynamic resource allocation is used to optimize the time of each segment of the entire cycle.

[0045] Furthermore, in step (3), the time optimization solution includes the following steps:

[0046] ①Introduce state representation: define S i，j，k To indicate the state of completing the i, j, k rounds of Comm, Forward and Backward loops, state S i，j，k The time can be expressed as T(S i，j，k );

[0047] ② Propose the optimization target minT(S N，N，N ): Aims to find the final value T(S) that is minimized N，N，N), thereby optimizing the entire process in N cycles of Comm, Forward, and Backward;

[0048] ③ Conduct policy management on the core allocation of the CPU: Propose two constraint problems of the core allocation model, which are defined by the function Φ(S i，j，k ) and related conditions as:

[0049]

[0050]

[0051] In this function, in any state S i，j，k The total number of allocated cores does not exceed the total number of available cores c total . Among them, c comm (S) and c comp (S) represent the number of cores allocated to Comm and Comp in state S i，j，k . The function Alloc(S) dynamically determines the allocation according to the current state S i，j，k . Strategically manage the CPU allocation through three specific conditions to optimize the system performance: If i < j, that is, when Comm has not ended after the Backward of the corresponding previous Comp task ends, it is necessary to ensure that all available cores are dedicated to Comm, corresponding to the first scenario proposed in step (2); If i = N, that is, when all Comm tasks are completed, redirect all resources to Comp, corresponding to the third scenario proposed in step (2); In other cases, it is necessary to dynamically adjust the core allocation to adapt to the real-time changes of task requirements and system performance, corresponding to the second scenario proposed in step (2);

[0052] Among them, the specific steps of dynamically adjusting the core allocation are as follows:

[0053] 1) Allocate all the total number of CPU cores to Comp;

[0054] [[ID=3�]]2) Obtain the communication calculation and time under the given corresponding cores from the f and g formulas, and take the maximum value as the current required time;

[0055] 3) Compare the current time with the minimum time, take the smaller value and update it as the minimum time, and record the number of current CPU cores allocated to Comm and Comp respectively;

[0056] 4) Change the CPU core allocation method, gradually increase the number of cores allocated to Comm, and loop through the above steps 2) and 3) until all allocation methods are iterated, and finally obtain the core allocation method with the lowest time consumption;

[0057] ④ Propose the state transition equation: Si，j，k The state transition at depends on the core allocation, and its mathematical expression is as follows:

[0058]

[0059] Where Δ represents the time transfer, which is defined as the shortest interval between three different time periods: the minimum completion time of the Comm, Forward, and Backward phases within a given cycle. Variables i′, j′, and k′ represent the time intervals in S i，j，k +Δ time is the number of times the Comm, Forward, and Backward loops are completed respectively;

[0060] ⑤Time calculation, find the minimum value T(S N，N，N ), the steps are as follows:

[0061] 1) Time is initialized to T(S 0，0，0 )=0;

[0062] 2) According to Φ(S i，j，k ) to allocate the kernel, and c total Assigned to C Comm and c Comp ;

[0063] 3) Determine the time transfer amount Δ using the formulas f and g;

[0064] 4) Update the current state to S i，j，k +Δ, that is, S i′，j′k′ ;

[0065] 5) Based on Δ, c Comm and c Comp Update communication and computational task workloads;

[0066] 6) Repeat steps 2), 3), 4), and 5) until the KD cycle ends.

[0067] Furthermore, in step (4), a series of experiments are conducted to simulate the actual KD process based on the edge environment through a large amount of data fitting results. Compared with the common method, the performance of the optimization scheme in different simulation scenarios is evaluated:

[0068] ① Simulation of the KD process in an actual edge environment: Using datasets of different batch sizes as input for teacher model inference and student model training, we obtain logits of different sizes and the corresponding number of communications and computations. These empirical sampling data are used as input for the simulation framework and function (f and g) fitting (Comm and Comp workloads).

[0069] ② Set different Comm to Comp workload ratios, setting them to 1:1 and 1:2 respectively to closely simulate actual operations; and configure Comm and Comp workloads to reflect real operating data;

[0070] ③Change key parameters, such as the total number of CPU cores c total :In the c total Experiments were conducted with the number of CPU cores set to 6, 8, 10, and 12 to examine how changes in CPU core availability affect the effectiveness of resource allocation strategies;

[0071] ④ Conduct 1000 experiments on each configuration random allocation algorithm, and compare the optimization scheme of the present invention with the commonly used allocation scheme:

[0072] 1) Random allocation: Considering the dependency of KD based on edge environment, CPU resources are randomly allocated in the case of parallel Comm and Comp tasks;

[0073] 2) Priority-based allocation: CPU resources are exclusively allocated to Comm and Comp tasks at any given time, regardless of parallel contention;

[0074] Excellent experimental results are obtained: under different workload distributions and different total number of available CPUs, the time spent by the optimization scheme can achieve near-optimal performance.

[0075] This paper delves into the complex challenges of the KD process in the context of edge computing, providing unique insights into the impact of CPU core allocation on communication and computation speed on resource-constrained edge devices. It emphasizes the ideal dependencies and constraints in edge-based KD and introduces a dynamic resource allocation solution, using simulations to vary key parameters such as the number of CPU cores and data volume.

[0076] The beneficial effects of the present invention are: constructing a communication and computing task time model within the knowledge distillation cycle in a limited resource edge environment, and proposing a corresponding dynamic resource optimization method with good robustness. The constructed framework can provide further exploration and innovation for optimizing edge computing operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 (a) and Figure 1 (b) is a trend diagram of the communication and computing time under different CPU core allocations of the present invention and the change with the number of cores allocated to communication, where Figure 1 (a) shows the trend of computing time during training with bathsize = 16 and communication time during data transmission with 1MB when 1, 2, or 3 CPU cores are allocated for communication tasks in Pi. Figure 1(b) shows the trends of computation time during training with bathsize = 128 and communication time during data transmission with 5MB when 1, 2, or 3 CPU cores are allocated for communication tasks in NX.

[0078] Figure 2 A diagram of the computing and communication task scenarios within the edge computing-based knowledge distillation cycle of the present invention;

[0079] Figure 3 A trend chart showing how the time required for communication and calculation of the KD in the simulated actual edge environment of the present invention changes with the number of cores allocated to the calculation;

[0080] Figure 4 (a) and Figure 4 (b) is a diagram showing the comparison of the optimization scheme proposed by the present invention with the kernel effects of priority allocation and random allocation, where Figure 4 (a) is the effect diagram when the workload ratio of Comm to Comp is set to 1:1. Figure 4 (b) shows the effect when the workload ratio of Comm to Comp is set to 2:1;

[0081] Figure 5 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0082] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific examples.

[0083] Reference Figures 1 to 5 ,A CPU resource allocation optimization method within the knowledge distillation cycle based on edge computing, comprising the following steps:

[0084] (1) Establish a digital model for the temporal dependency between communication and computation by knowledge distillation in edge environments;

[0085] (2) Construct a parallel scenario of communication and computing tasks in the iterative process of knowledge distillation in edge computing, and propose a time optimization task within the entire KD cycle;

[0086] (3) Propose a strategy to optimize time management: adopt a framework based on dynamic resource allocation to achieve a near-optimal solution;

[0087] (4) Simulation experiments are conducted to evaluate the effectiveness of the optimization method in KD based on actual edge environments.

[0088] Furthermore, in step (1), considering the impact of CPU cores on the total time required for knowledge distillation, a series of control experiments were conducted on Raspberry Pi and Jetson Xavier NX platforms to quantitatively evaluate the changes in the time required for communication and computing tasks under different CPU core allocations, as shown in Figure 2. Figure 1 (a) and Figure 1 (b) Based on the experimental results, the following function fitting is proposed to represent the relationship between communication time and computing time and CPU core allocation:

[0089] ① Computation: The time required for computing tasks in edge devices depends on the allocation of CPU cores. The relationship between the required computing time and the number of CPU cores allocated to communication can be approximated by the following function:

[0090]

[0091] where f(x,c) is the computation time required to compute workload x on the c CPU cores allocated to the communication task. The constants a1 and a2 reflect the effect of CPU allocation on computation time. In this equation, computation time increases as the number of CPU cores available for communication tasks increases. This highlights the linear reduction in computational resources as more CPU cores are allocated away from computation tasks.

[0092] ② Communication: Communication tasks in edge devices have diminishing marginal returns in terms of CPU core allocation. The time required for communication tasks can be approximated by the following function:

[0093]

[0094] Here, g(y,c) represents the time required for a communication task with data size y and c CPU cores allocated for communication. b1·c indicates that communication speed increases linearly with the number of cores, while b2·log(b3·c) indicates that the improvement decreases logarithmically with the number of allocated cores. The constants b1, b2, and b3 define the specific characteristics of this relationship. This formula shows that initially, increasing the number of cores c dedicated to communication significantly reduces communication time, but after a certain point, the effect becomes less pronounced.

[0095] Furthermore, in step (2), it is assumed that the iterative process of knowledge distillation in the edge computing environment involves N cycles of Comm and Comp tasks, each consisting of a Forward and a Backward task. The parallel execution of Comm and Comp tasks highlights the inherent resource contention and its impact on the speed of the two processes. Therefore, a time optimization task around N cycles is proposed, specifically including:

[0096] ① Propose constraints: Consider that Comm is a continuous process. The distillation temperature coefficient T transmits logit data from the cloud, and the calculation involves forward and backward propagation, where backward depends on logits. The following constraints are used to construct a time optimization problem:

[0097] 1) Comm is uninterrupted, and logits are transmitted from the teacher model in the cloud, which requires continuous allocation of resources for data transmission until the communication task is completed;

[0098] 2) Each Backward in the Comp sequence depends on the logits of the corresponding Comm cycle. If Comm has not yet completed, the Backward must wait, resulting in a delay, such as t″;

[0099] 3) When time is greater than t end When , it means that all Comm tasks have been completed. At this time, all computing resources will be allocated to Comp tasks, that is, there are only Comm tasks in stage t″;

[0100] ② Time calculation: In the complex coordination of the system Comm and Comp, time calculation is a key factor, especially when considering the interdependence between various calculation processes. In order to facilitate a thorough analysis, a stage-based approach is adopted to split the timeline before each reverse calculation. Assuming there are N reverse calculation instances, N+1 different time periods are determined for analysis, such as Figure 2 As shown:

[0101] 1) Figure 1 The first time period in is denoted as t1. Indicates the computational load of forwarding 1, It represents the communication load of the first communication cycle Comm. The duration of t1 can be expressed as:

[0102]

[0103] In this formula, and They represent the functions of quantizing the forward propagation completion time and the communication cycle, respectively. It means in After completion, reallocate the number of cores Complete the remaining calculations The time required. The criterion for determining the duration of t1 depends on whether the forward propagation time is less than the communication time. It is worth noting that if If it is smaller, it means that the subsequent communication cycle (for i>1) will overlap with Forward1;

[0104] 3) For t2 to tN Consider three cases. The first case, shown as t2, consists of overlapping time t′ and computation time t″. The second case, shown as t3, indicates that communication and computation are completed in parallel. The third case, shown as t N As shown, this occurs when there is no communication task, resulting in the reallocation of kernel resources to favor computing tasks. The time periods for the three cases are shown below:

[0105] 1. The first scenario: Communication and computing tasks overlap. For a general time period t i , which is expressed as follows:

[0106]

[0107]

[0108] 2. The second scenario: Communication and computation are completed in parallel. For a general time period t i , which is expressed as follows:

[0109]

[0110]

[0111] 3. The third scenario: All communication tasks have been completed, and only the computing task remains. In this case, all resources can be redistributed to Comp. Then, for time period t i , which is expressed as follows:

[0112]

[0113] 3) For the last time period t N+1 , since there is no additional Comm task, the required time is dedicated to BackwardN. It can be expressed mathematically as follows:

[0114]

[0115] Total computing time T total , which represents the total duration of the KD process across N segments of Comm and Comp tasks. total The optimization of T depends on an effective CPU core resource allocation strategy, the main goal of which is to minimize T by optimizing the core utilization of each segment. total , ensuring that from t1 to t N The sum of the time is as short as possible. This concept can be expressed mathematically as:

[0116]

[0117] In this formula, t i (ci ) indicates that on a given CPU core c i The time spent in segment i under the allocation condition.

[0118] In the step (3), for the three communication tasks and computing task scenarios within the knowledge distillation cycle constructed in step (2), dynamic resource allocation is used to optimize the time of each segment of the entire cycle.

[0119] Furthermore, in step (3), the time optimization solution includes the following steps:

[0120] ①Introduce state representation: define S i，j，k To indicate the state of completing the i, j, k rounds of Comm, Forward and Backward loops, state S i，j，k The time can be expressed as T(S i，j，k );

[0121] ② Propose the optimization target minT(S N，N，N ): Aims to find the final value T(S) that is minimized N，N，N ), thereby optimizing the entire process in N cycles of Comm, Forward, and Backward;

[0122] ③ Strategic management of CPU core allocation: Two constraints of the core allocation model are proposed, and the function Φ(S i，j，k ) and related conditions are defined as:

[0123]

[0124]

[0125] In this function, at any state S i，j，k The total number of cores allocated does not exceed the total number of available cores c total Among them, c comm (S) and c comp (S) indicates state S i，j，k The number of cores allocated to Comm and Comp. Function Alloc(S) is based on the current state S i，j，kDynamic determination of allocation. The CPU allocation is strategically managed through three specific conditions to optimize system performance: If i < j, that is, when Comm has not ended after the Backward of the corresponding previous Comp task has ended, all available cores should be dedicated to Comm, corresponding to the first scenario proposed in step (2); If i = N, that is, when all Comm tasks are completed, all resources are redirected to Comp, corresponding to the third scenario proposed in step (2); In other cases, the core allocation needs to be dynamically adjusted to adapt to the real-time changes in task requirements and system performance, corresponding to the second scenario proposed in step (2);

[0126] Among them, the specific steps for dynamically adjusting the core allocation are as follows:

[0127] 1) Allocate all the total number of CPU cores to Comp;

[0128] 2) Obtain the communication calculation and time under the given corresponding cores from the f and g formulas, and take the maximum value as the current required time;

[0129] 3) Compare the current time with the minimum time, take the smaller value and update it as the minimum time, and record the current number of CPU cores allocated to Comm and Comp respectively;

[0130] 4) Change the CPU core allocation method, gradually increase the number of cores allocated to Comm, and loop through the above steps 2) and 3) until all allocation methods have been iterated, and finally obtain the core allocation method with the lowest time consumption;

[0131] ④ Propose the state transition equation: The state transition at s i,j,k depends on the core allocation, and its mathematical expression is as follows:

[0132]

[0133] Among them, Δ represents the time transfer amount, which is defined as the shortest interval between three different time periods: the minimum completion time of the Comm, Forward, and Backward phases within a given period. The variable i ′ , j ′ , k ′ represent the respective completion times of the Comm, Forward, and Backward loops at the S i,j,k +Δ moment;

[0134] ⑤ Time calculation, find the minimum value T(S N,N,N ), the steps are as follows:

[0135] 1) Initialize the time as T(S 0，0，0 ) = 0;

[0136] 2) According to Φ(S i，j，k ) to allocate the kernel, and c total Assigned to c Comm and c Comp ;

[0137] 3) Determine the time transfer amount Δ using the formulas f and g;

[0138] 4) Update the current state to S i，j，k +Δ, that is, S i′，j′，k′ ;

[0139] 5) Based on Δ, c Comm and c Comp Update communication and computational task workloads;

[0140] 6) Repeat steps 2), 3), 4), and 5) until the KD cycle ends.

[0141] Furthermore, in step (4), a series of experiments are conducted to simulate the actual KD process based on the edge environment through a large amount of data fitting results. Compared with the common method, the performance of the optimization scheme in different simulation scenarios is evaluated:

[0142] ① Simulation of the KD process in an actual edge environment: Using datasets of different batch sizes as input for teacher model inference and student model training, we obtain logits of different sizes and the corresponding number of communications and computations. These empirical sampling data are used as input for the simulation framework and function (f and g) fitting (Comm and Comp workloads), as shown in Figure 2. Figure 3 As shown;

[0143] ② Set different Comm to Comp workload ratios, setting them to 1:1 and 1:2 respectively to closely simulate actual operations; and configure Comm and Comp workloads to reflect real operating data;

[0144] ③Change key parameters, such as the total number of CPU cores c total :In the c total Experiments were conducted with the number of CPU cores set to 6, 8, 10, and 12 to examine how changes in CPU core availability affect the effectiveness of resource allocation strategies;

[0145] ④ Conduct 1000 experiments on each configuration random allocation algorithm, and compare the optimization scheme of the present invention with the commonly used allocation scheme:

[0146] 1) Random allocation: Considering the dependency of KD based on edge environment, CPU resources are randomly allocated in the case of parallel Comm and Comp tasks;

[0147] 2) Priority-based allocation: CPU resources are exclusively allocated to Comm and Comp tasks at any given time, regardless of parallel contention;

[0148] Excellent experimental results were obtained, such as Figure 4 (a) and Figure 4 (b) shows that under different workload distributions and different total number of available CPUs, the time taken by the optimization scheme can achieve near-optimal performance.

[0149] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The scope of protection of the present invention should not be regarded as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for optimizing CPU resource allocation within a knowledge distillation cycle based on edge computing, comprising the following steps: (1) Establish a digital model for the temporal dependency between communication and computation by knowledge distillation in edge environments; (2) Construct a parallel scenario of communication and computing tasks in the iterative process of knowledge distillation in edge computing, and propose a time optimization task within the entire KD cycle; (3) Propose a strategy to optimize time management: adopt a framework based on dynamic resource allocation to achieve a near-optimal solution; (4) Conduct simulation experiments to evaluate the effectiveness of the optimization method in KD based on real edge environments; In step (1), considering the impact of CPU cores on the total time required for knowledge distillation, a series of controlled experiments were conducted on the Raspberry Pi and Jetson Xavier NX platforms to quantitatively evaluate the changes in the time required for communication and computing tasks under different CPU core allocations. Based on the experimental results, the following function fitting is proposed to represent the relationship between communication time and computing time and CPU core allocation: ① Computation: The time required for computing tasks in edge devices depends on the allocation of CPU cores; the relationship between the required computing time and the number of CPU cores allocated to communication is approximated by the following function: where f(x,c) is the computation time required to compute workload x on c CPU cores allocated to the communication task. The constants a1 and a2 reflect the effect of CPU allocation on computation time. In this equation, computation time increases with the number of CPU cores available for communication tasks. This highlights the linear reduction in computational resources as more CPU cores are allocated from computation tasks. ② Communication: Communication tasks in edge devices have diminishing marginal returns in terms of CPU core allocation. The time required for communication tasks is approximated by the following function: where g(y,c) represents the time required for a communication task with data size y and c CPU cores allocated for communication; b1·c indicates that the communication speed increases linearly with the number of cores; and b2·log(b3·c) indicates that the improvement decreases logarithmically with the number of allocated cores; constants b1, b2, and b3 define specific characteristics of this relationship; the formula shows that initially, increasing the number of cores c dedicated to communication can significantly reduce communication time, but after a certain point, the effect becomes less significant.

2. The CPU resource allocation optimization method within the knowledge distillation cycle based on edge computing according to claim 1, characterized in that: In step (2), it is assumed that the iterative process of knowledge distillation in the edge computing environment involves N cycles of Comm and Comp tasks, each cycle consisting of a Forward and a Backward task. The parallel execution of Comm and Comp tasks highlights the inherent resource contention and its impact on the speed of the two processes. Therefore, a time optimization task around N cycles is proposed, specifically including: ① Propose constraints: Consider that Comm is a continuous process. The distillation temperature coefficient T transmits logit data from the cloud, and the calculation involves forward and backward propagation, where backward depends on logits. The following constraints are used to construct a time optimization problem: 1) Comm is uninterrupted, and logits are transmitted from the teacher model in the cloud, which requires continuous allocation of resources for data transmission until the communication task is completed; 2) Each Backward in the Comp sequence depends on the logits of the corresponding Comm cycle; if Comm has not yet completed, the Backward must wait, resulting in a delay; 3) When time is greater than t end When , it means that all Comm tasks have been completed. At this time, all computing resources will be allocated to Comp tasks, that is, there are only Comm tasks in stage t″; ② Time calculation: In the complex coordination of the system Comm and Comp, time calculation is a key factor, especially when considering the interdependence between various calculation processes. To facilitate a thorough analysis, a stage-based approach is adopted, dividing the timeline before each reverse calculation. Assuming there are N reverse calculation instances, N+1 different time periods are determined for analysis: 1) The first time period is denoted as t1; let Indicates the computational load of forwarding 1, It represents the communication load of the first communication cycle Comm; the duration of t1 is expressed as: In this formula, and Respectively represent the functions of quantizing the forward propagation completion time and the communication cycle; It means in After completion, reallocate the number of cores Complete the remaining calculations The time required; the criterion for determining the duration of t1 depends on whether the forward propagation time is less than the communication time; it is worth noting that if If it is smaller, it means that the subsequent communication cycle will overlap with Forward1; 2) For t2 to t N We consider three cases: the first case, as shown in t2, consists of overlapping time t′ and computation time t″; the second case, as shown in t3, indicates that communication and computation are completed in parallel; the third case, as shown in t N As shown, it occurs when there is no communication task, resulting in the reallocation of kernel resources to facilitate computing tasks; the time periods in the three cases are shown as follows:

1. The first scenario: communication and computing tasks overlap; for time period t i , which is expressed as follows:

2. The second scenario: communication and computation are completed in parallel; for time period t i , which is expressed as follows:

3. The third scenario: All communication tasks have been completed, and only the computing task remains. At this time, all resources can be redistributed to Comp. Then, for time period t i , which is expressed as follows: 3) For the last time period t N+1 , since there is no additional Comm task, the required time is dedicated to BackwardN; it can be expressed mathematically as follows: Total computing time T total , which represents the total duration of the KD process across N segments of Comm and Comp tasks; T total The optimization of T depends on an effective CPU core resource allocation strategy, the main goal of which is to minimize T by optimizing the core utilization of each segment. total , ensuring that from t1 to t N The sum of the time is as short as possible; mathematically expressed as: In this formula, t i (c i ) indicates that on a given CPU core c i The time spent in segment i under the allocation condition.

3. The method for optimizing CPU resource allocation within a knowledge distillation cycle based on edge computing according to claim 2, wherein, in step (3), dynamic resource allocation is used to optimize the time of each segment of the entire cycle under the three communication tasks and computing task scenarios within the knowledge distillation cycle constructed in step (2); the content of the time optimization scheme includes the following steps: ①Introduce state representation: define S i,j,k To indicate the completion of i, j, j rounds of Comm, Forward and Backward loops, state S i,j,k The time is expressed as T(S i,j,k ); ② Propose the optimization target min T(S N,N,N ): Aims to find the final value T(S) that is minimized N,N,N ), thereby optimizing the entire process in N cycles of Comm, Forward, and Backward; ③ Strategic management of CPU core allocation: Two constraints of the core allocation model are proposed, and the function Φ(S i,j,k ) and related conditions are defined as: In this function, at any state S i,j,k The total number of allocated cores does not exceed the total number of available cores c total ; where c comm (S) and c comp (S) represent the number of cores allocated to Comm and Comp at state S i,j,k ; the function Alloc(S) dynamically determines the allocation according to the current state S i,j,k Strategically manage the CPU allocation through three specific conditions to optimize system performance: If i < j, that is, when Comm has not ended after the Backward of the corresponding previous Comp task has ended, ensure that all available cores are dedicated to Comm, corresponding to the first scenario proposed in step (2); If i = N, that is, when all Comm tasks are completed, redirect all resources to Comp, corresponding to the third scenario proposed in step (2); In other cases, the core allocation needs to be dynamically adjusted to adapt to the real-time changes in task requirements and system performance, corresponding to the second scenario proposed in step (2); The specific steps for dynamically adjusting core allocation are as follows: 1) Allocate all CPU cores to Comp; 2) Calculate the communication calculation and time under the given corresponding kernel using the formulas f and g, and take the maximum value as the current required time; 3) Compare the current time with the minimum time, take the smaller value and update it to the minimum time, and record the number of CPU cores currently allocated to Comm and Comp respectively; 4) Change the CPU core allocation method, gradually increase the number of cores allocated to Comm, and repeat steps 2) and 3) above until all allocation methods have been iterated and the core allocation method with the lowest time cost is finally obtained; ④ Propose the state transfer equation: S i,j,k The state transition at depends on the core allocation, and its mathematical expression is as follows: Where Δ represents the time transfer amount, which is defined as the shortest interval between three different time periods: the minimum completion time of the Comm, Forward, and Backward stages within a given cycle, and the variables i′, j′, and k′ represent the time transfer amount in S i,j,k +Δ time is the number of times Comm, Forward, and Backward loops are completed respectively; ⑤Time calculation, find the minimum value T(S N,N,N ), the steps are as follows: 1) Time is initialized to T(S 0,0,0 )=0; 2) According to Φ(S i,j,k ) to allocate the kernel, and c total Assigned to c Comm and c Comp ; 3) Determine the time transfer amount Δ using the formulas f and g; 4) Update the current state to S i,j,k +Δ, that is, S i′,j′,k′ ; 5) Based on Δ, c Comm and c Comp Update communication and computation task workloads; 6) Repeat steps 2), 3), 4), and 5) until the KD cycle ends.

4. The CPU resource allocation optimization method within the knowledge distillation cycle based on edge computing according to claim 3, characterized in that: In step (4), a series of experiments are conducted to simulate the actual KD process based on the edge environment through a large amount of data fitting results. The performance of the optimization scheme in different simulation scenarios is evaluated by comparing it with the conventional method: ① Simulation of the KD process in an actual edge environment: Using datasets of different batch sizes as input for teacher model inference and student model training, we obtain logits of different sizes and the corresponding communication and computation times. These empirically sampled data are used as input for the simulation framework and function fitting, i.e., the workloads of Comm and Comp; where the functions include f and g; ② Set different Comm to Comp workload ratios, setting them to 1:1 and 1:2 respectively to closely simulate actual operations; and configure Comm and Comp workloads to reflect real operating data; ③Change key parameters, such as the total number of CPU cores c total :In the c total Experiments were conducted with the number of CPU cores set to 6, 8, 10, and 12 to examine how changes in CPU core availability affect the effectiveness of resource allocation strategies; ④ Conduct 1000 experiments on each configuration random allocation algorithm, and compare the optimization scheme of the optimization method with the commonly used allocation scheme: 1) Random allocation: Considering the dependency of KD based on edge environment, CPU resources are randomly allocated in the case of parallel Comm and Comp tasks; 2) Priority-based allocation: CPU resources are exclusively allocated to Comm and Comp tasks at any given time, regardless of parallel contention; Excellent experimental results are obtained: under different workload distributions and different total number of available CPUs, the time spent by the optimization scheme can achieve near-optimal performance.

Images