Method and device for optimizing end-side energy consumption for model segmentation

Abstract

The application discloses a model segmentation-oriented end-side energy consumption optimization method and device, and relates to the technical field of device energy consumption control. The method is applied to a model collaborative training system, the system comprises at least one collaborative scheduling device and multiple computing devices, the method is executed by the collaborative scheduling device, and the method comprises the following steps: dividing a target large model to be trained into a plurality of different model segments according to a preset model segmentation scheme, and sending the plurality of different model segments to the multiple computing devices respectively; acquiring running temperature information and a current batch size when each computing device executes a corresponding target model segment; dynamically adjusting the current batch size according to a temperature warning value corresponding to each computing device and the running temperature information, so as to obtain an optimized batch size of each computing device for executing the target model segment in a current task cycle; and based on the optimized batch size, controlling each computing device to execute iterative training of the corresponding target model segment.

Description

Technical Field

[0001] This application relates to the field of equipment energy consumption control technology, and in particular to an end-side energy consumption optimization method and apparatus oriented towards model segmentation. Background Technology

[0002] In recent years, large-scale artificial intelligence models have been widely applied in scenarios such as natural language processing, computer vision, intelligent recommendation, industrial inspection, autonomous driving, medical assistance, and intelligent interaction. As the scale of models and input data continues to increase, single terminal devices, edge devices, or servers often struggle to stably handle the computation of the entire model over long periods in terms of memory capacity, computing power, heat dissipation, and power supply. To address this, the industry has proposed model segmentation techniques, which divide a complete neural network model into multiple computational segments according to layers, operators, subgraphs, expert modules, or pipeline stages, and deploy these segments on one or more computing devices for collaborative execution.

[0003] However, while model segmentation technology can alleviate the pressure on single device memory, reduce the computational burden on the edge, or achieve edge-cloud collaboration, the load on computing devices that allocate different model segments varies significantly. Some segments may become computational hotspots during the continuous computation of training, causing the computing device carrying the hotspot segment to accumulate temperature rapidly and enter a passive thermal protection state. This causes the pre-configured model segmentation deployment scheme in the computing device corresponding to the hotspot segment to fail, resulting in problems such as increased segment latency, pipeline blockage, and decreased throughput during model training, leading to increased energy consumption per unit task for each computing device.

[0004] Therefore, ensuring the stability of model segment training services on various computing devices in model segmentation scenarios, in order to avoid increased energy consumption per unit task on the computing devices, has become an urgent problem to be solved. Summary of the Invention

[0005] The main objective of this application is to provide an edge-side energy consumption optimization method and apparatus for model segmentation, aiming to solve the technical problem of how to ensure the service stability of each computing device in the model segmentation scenario, so as to avoid the increase of unit task energy consumption of the computing device.

[0006] To achieve the above objectives, this application proposes an edge-side energy consumption optimization method for model segmentation. This method is applied to a model co-training system, which includes at least one co-scheduling device and multiple computing devices. The method is executed by the co-scheduling device and includes: The target large model to be trained is divided into several different model segments according to a preset model segmentation scheme, and the several different model segments are sent to the multiple computing devices respectively; Obtain the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; The current batch size is dynamically adjusted based on the temperature warning value corresponding to each computing device and the operating temperature information to obtain the optimized batch size for each computing device to execute the target model fragment in the current task cycle; Based on the optimized batch size, each computing device is controlled to perform iterative training of the corresponding target model segment.

[0007] Furthermore, to achieve the above objectives, this application also proposes an end-side energy consumption optimization device for model segmentation, which includes: The model segmentation module is used to divide the target large model to be trained into several different model segments according to a preset model segmentation scheme, and send the several different model segments to multiple computing devices respectively. The device behavior monitoring module is used to obtain the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; The batch adjustment module is used to dynamically adjust the current batch size according to the temperature warning value corresponding to each computing device and the operating temperature information, so as to obtain the optimized batch size of each computing device executing the target model segment in the current task cycle; The energy consumption optimization module is used to control each of the computing devices to perform iterative training of the corresponding target model segment based on the optimized batch size.

[0008] In addition, to achieve the above objectives, this application also proposes a cooperative scheduling device, which includes: a memory, a processor, and a model-segmentation-oriented end-side energy consumption optimization program stored in the memory and capable of running on the processor. The model-segmentation-oriented end-side energy consumption optimization program is configured to implement the steps of the model-segmentation-oriented end-side energy consumption optimization method as described above.

[0009] In addition, to achieve the above objectives, this application also proposes a model collaborative training system, which includes at least one collaborative scheduling device mentioned above and several computing devices.

[0010] In addition, to achieve the above objectives, this application also provides a storage medium storing a program for implementing an edge-side energy consumption optimization method for model segmentation. The program for implementing the edge-side energy consumption optimization method for model segmentation is executed by a processor to implement the steps of the edge-side energy consumption optimization method for model segmentation as described above.

[0011] This application provides an edge-side energy consumption optimization method and apparatus for model segmentation. The method is applied to a model collaborative training system, which includes at least one collaborative scheduling device and multiple computing devices. The method is executed by the collaborative scheduling device and includes: dividing the target large model to be trained into several different model segments according to a preset model segmentation scheme, and sending the several different model segments to the multiple computing devices respectively; obtaining the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; dynamically adjusting the current batch size according to the temperature warning value and operating temperature information of each computing device to obtain the optimized batch size of each computing device executing the target model segment in the current task cycle; and controlling the iterative training of the corresponding target model segment by each computing device based on the optimized batch size.

[0012] This application proactively drives the batch size dynamic adjustment of the device side by sensing the device operating temperature information, thereby enabling each computing device to actively reduce the computing load before the operating temperature exceeds the temperature warning value, reducing the number of computing devices entering the hardware passive thermal protection state, ensuring the stability of the device side in executing model segment training, and thus avoiding the increase of unit task energy consumption of each computing device. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the first embodiment of the end-side energy consumption optimization method for model segmentation in this application. Figure 2 This is a flowchart illustrating the second embodiment of the end-side energy consumption optimization method for model segmentation in this application. Figure 3 This is a simplified schematic diagram of the end-side energy consumption optimization method for model segmentation in this application; Figure 4 This is a schematic diagram of the module structure of the end-side energy consumption optimization device for model segmentation according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the end-side energy consumption optimization method for model segmentation in the embodiments of this application.

[0016] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application. Furthermore, all actions involving the acquisition of signals, information, or data in this application are performed in accordance with the relevant data protection regulations of the country where the application is located and with authorization from the owner of the corresponding device.

[0018] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0019] The main solution of this application is to propose an edge-side energy consumption optimization method for model segmentation in a model co-training system. The system includes at least one co-scheduling device and multiple computing devices. The method is executed by the co-scheduling device and includes: dividing the target large model to be trained into several different model segments according to a preset model segmentation scheme, and sending the several different model segments to the multiple computing devices respectively; obtaining the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; dynamically adjusting the current batch size according to the temperature warning value and operating temperature information of each computing device to obtain the optimized batch size of each computing device executing the target model segment in the current task cycle; and controlling the iterative training of the corresponding target model segment by each computing device based on the optimized batch size. Currently, in model segmentation training scenarios, each model segment typically has different computational loads, memory usage, input / output tensor sizes, and execution latency. The front-end segments may run near the data acquisition end, while the back-end segments may run on edge servers or cloud GPUs (Graphics Processing Units); they can also be pipelined across multiple accelerators on the same device. When the segmented tasks are run on various computing devices in batch inference, micro-batch training, pipelined parallelism, or continuous service modes, some segments may become computational hotspots, causing rapid temperature accumulation in the GPUs, NPUs (Neural Network Processing Units), or other accelerators of the computing devices hosting these hotspot segments. If the computing device temperature approaches or exceeds its own operational safety threshold, the hardware typically enters a passive thermal protection state, causing the pre-configured model segmentation deployment scheme in the computing device corresponding to the hotspot segment to fail, resulting in increased segment latency, pipeline congestion, decreased throughput, and increased energy consumption per unit task during model training. Therefore, ensuring the service stability of model segment training on various computing devices in model segmentation scenarios, and avoiding increased energy consumption per unit task for the corresponding computing devices, has become an urgent problem to be solved.

[0020] To address this issue, this application proactively drives dynamic batch adjustment on the device side by sensing the device's operating temperature. This allows each computing device to proactively reduce its computational load before its operating temperature exceeds a warning threshold, reducing the likelihood of the device entering passive thermal protection mode and ensuring the stability of model segment training on the device side. This, in turn, avoids increased energy consumption per unit task for each computing device. Therefore, this application, through temperature-sensing batch adjustment, proactively adjusts the computational load of computing devices before hardware thermal protection is triggered, shifting the timing of temperature control from "passive frequency reduction after overheating" to "proactive load adjustment before thermal risk occurs." This reduces passive thermal frequency reduction events in model segmentation training scenarios, ensuring stable operation of the training pipeline and preventing increased energy consumption per unit task for each computing device.

[0021] It should be noted that the execution entity in this embodiment can be a model co-training system in a model segmentation scenario. This model co-training system can be a distributed training system composed of at least one co-scheduling device and multiple computing devices connected through a high-speed training network. In this case, the model co-training system can carry out distributed training tasks after model segmentation and supports a hybrid deployment of pipeline parallelism, data parallelism, and model parallelism. The aforementioned computing devices can refer to hardware nodes that carry out model segment training tasks. Each device is configured with at least one AI (Artificial Intelligence) acceleration chip such as a GPU / NPU, and has temperature sampling, power consumption monitoring, and frequency adjustment capabilities to perform forward computation, backpropagation, and gradient updates of the allocated model segments.

[0022] It is readily understood that the execution entity in this embodiment can also be a device in the model collaborative training system that acts as a global control node, possessing the capabilities of task orchestration, status acquisition, strategy decision-making, and instruction issuance, i.e., a collaborative scheduling device. The following description uses a collaborative scheduling device (hereinafter referred to as a scheduling device) as the execution entity to illustrate this embodiment and the subsequent embodiments.

[0023] Based on this, embodiments of this application provide an end-side energy consumption optimization method oriented towards model segmentation, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the end-side energy consumption optimization method for model segmentation in this application.

[0024] In this embodiment, the edge-side energy consumption optimization method for model segmentation is applied to a model collaborative training system. The system includes at least one collaborative scheduling device and multiple computing devices. The method is executed by the collaborative scheduling device and includes steps S10 to S40: Step S10: Divide the target large model to be trained into several different model segments according to a preset model segmentation scheme, and send the several different model segments to the multiple computing devices respectively; Understandably, the aforementioned target large model can refer to a large-scale deep neural network model that cannot be fully trained by a single computing device, such as a large language model, a multimodal large model, a visual backbone model, etc., which usually contain multi-layer network structures and massive parameters, serving as the segmentation object for the model segmentation scenario.

[0025] At this point, the aforementioned preset model segmentation scheme can refer to the pre-determined rules for partitioning and deploying the target large model. Specifically, it may include the total number of model segments, the set of network layers / operators covered by each segment, the deployment devices corresponding to the segments, the input / output tensor dimensions and transitive relationships between segments, the latency budget for each segment, and the pipeline stage division. This scheme can be pre-generated by a graph partitioning algorithm, a pipeline parallel strategy, or manual configuration. In this embodiment, it is only used as the scheduling input for the energy optimization scheme to be performed later, and no restrictions are placed on the model segmentation algorithm itself. That is, the edge-side energy consumption optimization scheme proposed in this embodiment is applicable to any model segmentation training algorithm.

[0026] Therefore, the aforementioned model fragments can refer to computational subgraphs formed after the target large model is segmented, which can be executed independently on a single computing device. In this embodiment, after generating several model fragments, the scheduling device can send different model fragments to multiple computing devices included in the system, and each fragment can be executed sequentially according to the training pipeline order. The output tensor / gradient of the previous fragment becomes the input of the next fragment, and each fragment has a globally unique identifier.

[0027] For example, in this embodiment, the collaborative scheduling device loads the complete computation graph of the target large model, and splits the continuous network layers / operators into multiple independent subgraphs according to the segmentation points defined by the preset model segmentation scheme. Each subgraph is assigned a unique identifier and its input / output interfaces are labeled. Then, the collaborative scheduling device can package the model segment weights, operator structures, input / output formats, and dependencies corresponding to the several model segments generated after splitting into a deployment package, and distribute it to the corresponding computing devices through a high-speed training network, enabling each computing device to complete the model loading and memory initialization of its corresponding target model segment.

[0028] Step S20: Obtain the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; Understandably, the aforementioned target model fragment can refer to the model fragment received and iteratively trained by each computing device. In this case, the operating temperature information can refer to a multi-dimensional data set reflecting the thermal state of the accelerator chip when the computing device executes the target model fragment. This data may include at least the real-time operating temperature and temperature rise rate of the accelerator chip, and may also include the temperature sliding average. In this embodiment, this operating temperature information can serve as the core basis for subsequent batch adjustments.

[0029] At this point, the current batch size mentioned above can also be called the batch size / micro-batch size, which can refer to the number of training samples that the computing device processes in a single training iteration. It can be a core parameter that determines the instantaneous computing load and thermal power consumption of the device.

[0030] For example, the collaborative scheduling device can collect data such as temperature and batch size from each computing device through the device behavior monitoring module at a fixed sampling period (e.g., 100ms / time). At the same time, each computing device in the system can read the real-time core temperature and the current configured batch size through the chip driver interface and actively report it to the collaborative scheduling device, which can then perform preprocessing such as moving average and trend calculation on the raw data.

[0031] Step S30: Dynamically adjust the current batch size according to the temperature warning value corresponding to each computing device and the operating temperature information to obtain the optimized batch size of each computing device executing the target model fragment in the current task cycle; It is easy to understand that since some segments after model segmentation may become hot segments in a short period of time, waiting for the computing device to actively adjust will cause the segment latency to suddenly increase. This embodiment can reduce the instantaneous computing load and heat accumulation rate of the current segment by reducing the micro-batch size in advance, so that the device enters the passive thermal protection state less often.

[0032] At this point, the aforementioned temperature warning value can refer to the batch adjustment trigger threshold preset by each computing device. In this embodiment, the function of this temperature warning value is to initiate proactive intervention before the formation of thermal risks. Therefore, the specific value of this temperature warning value can be equal to the device's safe temperature threshold (T_safe, such as 85℃) minus the temperature warning margin (ΔT, such as 5℃), and this value is lower than the hardware thermal protection trigger temperature.

[0033] It should be understood that the task cycle can refer to the smallest control granularity of batch adjustment, such as a training round, a micro-batch pipeline stage, or a fixed-duration scheduling window (such as 1 second). That is, in this embodiment, each computing device can perform a batch size update once in each task cycle.

[0034] Therefore, the optimized batch size can be dynamically adjusted based on the temperature warning value to configure a new batch size for the computing device in the current task cycle. Simultaneously, this optimized batch size needs to be constrained between a preset minimum batch size (b_min) and a preset maximum batch size (b_max) to balance thermal safety and training efficiency.

[0035] For example, in this embodiment, the collaborative scheduling device can use the temperature warning value as a benchmark, combine the operating temperature information to determine the thermal risk level of each computing device, and then determine the adjustment direction and magnitude of the batch corresponding to each computing device, thereby generating an optimized batch size that meets the boundary constraints.

[0036] It is easy to understand that existing batch adjustment schemes rely solely on instantaneous temperature as the basis for judgment, resulting in a significant lag: when the temperature rises rapidly, even if the instantaneous temperature has not yet reached the warning value, it may exceed the safety threshold within a short period of time, leading to untimely adjustment and ultimately triggering hardware thermal throttling. In a feasible implementation, the operating temperature information includes real-time operating temperature and the rate of temperature increase; in this embodiment, step S30 may include steps A1~A3: Step A1: Obtain the temperature comparison result generated after comparing the temperature warning value corresponding to each of the computing devices with the real-time operating temperature, and obtain the temperature exceeding warning result generated based on the real-time operating temperature and the temperature rise rate. Understandably, the aforementioned real-time operating temperature can refer to the current instantaneous core temperature of the computing device's accelerator chip. As a direct representation of the computing device's thermal state, it can be directly read by the computing device through the chip's built-in temperature sensor. Therefore, the aforementioned temperature comparison result can refer to the state conclusion obtained after comparing the real-time operating temperature with the temperature warning value. In this embodiment, the temperature comparison result can be divided into three states: "below the warning value," "in the warning threshold range," and "above the warning value." For example, in this embodiment, during each task cycle decision, the collaborative scheduling device can first subtract the collected real-time operating temperature value from the preset temperature warning value, and then classify the comparison result and generate corresponding state labels based on the sign and magnitude of the difference. For example, assuming the current real-time operating temperature of the computing device is T_now and the temperature warning value is T_warn, then the temperature difference ΔT = T_now - T_warn. At this point, if ΔT < -2℃, the temperature comparison result can be determined as "below the warning value", which is within the safe range; if -2℃ ≤ ΔT ≤ 0℃, the temperature comparison result can be determined as "within the warning critical range", which is within the warning edge; if ΔT > 0℃, the temperature comparison result can be determined as "above the warning value", which is within the warning range.

[0037] It should be noted that the aforementioned temperature rise rate refers to the increase in chip temperature per unit time (unit: °C / s), which can be calculated by the collaborative computing device based on the temperature values ​​uploaded by the computing device at multiple consecutive sampling points. In this embodiment, the temperature rise rate can be used to predict future temperature trends, thereby identifying the hidden risk of the computing device potentially overheating in a short period of time. In this case, the aforementioned temperature exceedance warning result can refer to a prediction based on the current real-time temperature and the temperature rise rate, using trend extrapolation to determine whether the temperature will exceed the device's safe temperature threshold within a preset timeframe. In this embodiment, the temperature exceedance warning result can be divided into three levels: "no risk of exceeding the limit," "risk of exceeding the limit exists," and "imminent exceedance."

[0038] Exemplarily, the cooperative scheduling device may take the current real-time operating temperature as the starting point and the measured temperature rise rate as the slope, perform linear extrapolation to predict the temperature value 3 to 5 seconds in the future, compare the estimated predicted temperature value with the device safety temperature threshold, and then generate a corresponding risk level according to the temperature comparison result and the preset temperature warning margin, so as to identify implicit thermal risks in a rapid temperature rise scenario in advance.

[0039] For example, assume that the temperature rise rate measured by the scheduling device is k (°C / s), the device safety temperature threshold is T_safe, and the temperature warning margin is 3°C. Then, the predicted temperature value T_pred 3 seconds later predicted by the linear extrapolation method is T_pred=T_now+k×3. At this time, T_pred is compared with the device safety temperature threshold T_safe, and the correspondingly generated temperature exceeding warning result is: if T_pred<T_safe-3°C, the result may be "no over-limit risk"; if T_safe-3°C≤T_pred<T_safe, the result may be "over-limit risk exists"; if T_pred≥T_safe, the result may be "about to exceed the limit". At this time, the scheduling device can predict future thermal risks in combination with the temperature change trend, identify implicit risks in the rapid temperature rise scenario 3 to 5 seconds in advance, achieve more accurate adjustment timing, and further reduce the probability of thermal frequency reduction.

[0040] Step A2: Determine a target adjustment amplitude and a target adjustment direction according to the temperature comparison result and / or the temperature exceeding warning result. It should be understood that, in this embodiment, the scheduling device may determine the thermal risk level corresponding to each computing device according to the temperature comparison result and / or the temperature exceeding warning result, and determine the change direction of each batch size according to the determined thermal risk level, that is, the aforementioned target adjustment direction. In this embodiment, the target adjustment direction may include three types: "increase", "decrease", and "remain unchanged".

[0041] At this time, the aforementioned target adjustment amplitude may refer to the change amount or change proportion of a single adjustment of the batch size. In this embodiment, the target adjustment amplitude may be set in two manners: fixed step increment / decrement or proportional scaling, and the specific amplitude value may also be positively correlated with the aforementioned thermal risk degree.

[0042] It is understandable that in a small interval centered on the temperature warning value (e.g., warning value ±2°C), tiny temperature fluctuations are likely to trigger repeated switching of batch adjustment, and when the load size of the computing device frequently switches near the temperature threshold, it easily causes waiting for upstream and downstream segments of supply and cache jitter of the switching device. Therefore, further, In this embodiment, when the real-time operating temperature falls within the threshold critical range centered on the temperature warning value (e.g., T_warn-2℃ to T_warn+2℃), the scheduling device first generates the original adjustment direction and amplitude according to the temperature comparison result and the over-limit warning result according to the conventional logic, but it does not take effect for the time being, and serves as the initial value to be verified; if it does not fall within the critical range, the initial adjustment direction is directly used as the final target adjustment direction without additional verification.

[0043] Then, the scheduling device can perform jitter verification on the initial value through a preset jitter suppression strategy. If the verification fails, the adjustment direction will be corrected to "remain unchanged". If the verification passes, the initial direction will be retained as the final target adjustment direction.

[0044] Among them, the preset jitter suppression strategy can refer to a set of mechanisms used to avoid the batch size from frequently switching back and forth near the threshold. In this embodiment, one or more combinations of temperature hysteresis, moving average temperature verification, continuous multiple sampling confirmation, minimum hold period, and segment-level cooling window can be used to perform jitter verification on the initial adjustment direction.

[0045] For example, when using temperature hysteresis for jitter suppression verification, the scheduling device can set two different thresholds: an upward trigger threshold and a downward recovery threshold. Load reduction is only triggered when the real-time operating temperature of the computing device rises and reaches the upward trigger threshold, and recovery is only allowed when the temperature falls and reaches the downward recovery threshold, forming a hysteresis interval and eliminating oscillations at single-point thresholds. If a minimum hold period is used, the scheduling device can maintain a minimum number of task cycles that must remain unchanged after each batch size adjustment, prohibiting repeated adjustments within a short period. In this case, jitter verification is achieved by detecting the interval between the change in the previous adjustment direction.

[0046] Furthermore, this embodiment can also employ a triple combination strategy of "temperature hysteresis + minimum hold period + continuous sampling confirmation" for verification. In this case, if the scheduling device detects that the initial adjustment direction is "decreasing", it can first verify whether the temperature has reached the rising trigger threshold (T_warn+1℃). If it has, the temperature hysteresis verification passes; otherwise, it fails. If the initial adjustment direction is "increasing", it verifies whether the temperature has fallen back to the falling recovery threshold (T_warn-2℃). If it has, the temperature hysteresis verification passes; otherwise, it fails.

[0047] Next, after the temperature hysteresis passes, the scheduling equipment can perform continuous sampling confirmation verification. At this time, the scheduling equipment can read the temperature data of the three most recent consecutive sampling cycles, and the verification item will only pass if the trigger conditions of the corresponding adjustment direction are met in all three cycles; if any cycle does not meet the conditions, the continuous sampling confirmation verification will fail.

[0048] Finally, after the continuous sampling confirmation verification is also passed, the scheduling equipment can first query the cycle position where the last batch adjustment occurred during the minimum hold period verification process, and calculate the interval number from the current cycle; if the interval number is less than the preset minimum hold period (default 2 cycles), the jitter verification will still fail; if it is greater than or equal to the minimum hold period, the jitter verification will pass.

[0049] Therefore, to address the issue of frequent batch size switching caused by traditional single-point threshold adjustment logic when temperature fluctuates slightly around the warning value, and to avoid fluctuations in model segmentation pipeline throughput, upstream and downstream segment waiting, and buffering due to frequent batch changes, this embodiment uses jitter verification to avoid batch switching caused by slight temperature fluctuations, reducing the frequency of batch adjustment and improving the smoothness of training load and overall system stability. Simultaneously, this embodiment only enables strong jitter suppression in the critical range, allowing the edge-side energy consumption optimization method to maintain its original response speed in both safe and high-risk ranges, thus achieving a balance between stability and timely temperature control.

[0050] Step A3: Dynamically adjust the current batch size based on the target adjustment range and the target adjustment direction to obtain the optimized batch size for each computing device to execute the target model fragment in the current task cycle.

[0051] For example, the collaborative scheduling device can query a preset adjustment strategy table based on a combination of temperature comparison results and temperature exceedance warning results to determine the target adjustment direction and target adjustment magnitude. It is understandable that because a single-dimensional judgment cannot distinguish between "slow rise in temperature" and "rapid rise in temperature" scenarios, it is prone to over-adjustment or under-adjustment, failing to balance training efficiency and thermal safety. To solve this problem, in this embodiment, the batch adjustment strategy set based on temperature comparison results and / or temperature exceedance warning results can match differentiated adjustment magnitudes for different risk levels, following the principle of "slow start growth in the safe zone, rapid load reduction in the warning zone, and gradual recovery in the recovery zone." In the safe zone, the batch size is fully increased to improve throughput, while in the risk zone, the load is rapidly reduced to suppress temperature rise, achieving the optimal balance between training efficiency and thermal stability.

[0052] For example, when the temperature comparison result is "below the warning value" and the exceedance warning is "no risk of exceeding the limit", the target adjustment direction can be "increase". The target adjustment range can follow the slow start rule, that is, when the current batch size is ≤ b_max / 2, a large step size can be used to increase the current batch size by 25% each time; if the current batch size is > b_max / 2, a small step size can be used to increase the current batch size by 10% each time. The purpose of this step is to quickly increase the throughput when the batch size is small, and to increase it gradually when the batch size is large to avoid temperature spikes.

[0053] When the temperature comparison result is "within the early warning critical interval" or the excess warning is "there is an excess risk", the target adjustment direction may be "decrease", and the target adjustment amplitude is the current batch multiplied by 0.75, that is, a 25% reduction.

[0054] When the temperature comparison result is "higher than the early warning value" or the excess warning is "about to exceed the standard", the target adjustment direction may be "decrease", and the adjustment amplitude may be the current batch multiplied by 0.5, that is, a 50% reduction; In other boundary scenarios, there is no need to adjust the batch size of the computing device. Correspondingly, the target adjustment direction of the current task cycle can remain the same as that of the previous cycle, and the amplitude is set to zero.

[0055] Therefore, the scheduling device can update the value of the current batch size according to the adjustment direction and amplitude determined based on the aforementioned temperature monitoring, so as to generate an optimized batch size. Further, the scheduling device can also perform batch size boundary constraint and output, that is, the scheduling device can calculate a temporary batch value according to the target adjustment direction and amplitude. It is easy to understand that when an increase is required, temporary batch = current batch × (1 + growth ratio); when a decrease is required, temporary batch = current batch × reduction ratio. At this time, if the temporary batch is less than b_min, b_min is taken; if the temporary batch is greater than b_max, b_max is taken. The verified value is finally output as the optimized batch size of the current task cycle.

[0056] In this embodiment, before each inference batch starts, the temperature-aware batch size control module determines the temperature state of the currently executed hot spot segment: if the GPU temperature is within the safe interval, the micro-batch size is gradually increased to improve throughput; if the temperature is close to the early warning interval, the micro-batch size is reduced to lower the computational load of the segment. Therefore, this embodiment can perform a comprehensive temperature judgment of the computing device through real-time static temperature comparison and dynamic prediction of temperature trends, improve the forward-looking nature and accuracy of thermal risk identification, solve the problem of adjustment lag caused by the existing method that only relies on instantaneous temperature judgment, realize more precise hierarchical batch adjustment, and achieve the effect of balancing training throughput and thermal stability.

[0057] Step S40: Based on the optimized batch size, control each of the computing devices to perform iterative training on the corresponding target model segment.

[0058] It is understandable that the above iterative training may refer to multiple rounds of training corresponding to the complete parameter update process of model training in the model segmentation scenario, and each round of training may include four links: forward propagation, loss calculation, back propagation, and gradient update, and a single round of training requires all model segments to be completed collaboratively in a pipeline sequence.

[0059] This embodiment provides an edge-side energy consumption optimization method for model segmentation. The method is executed by a collaborative scheduling device and includes: dividing the target large model to be trained into several different model segments according to a preset model segmentation scheme, and sending the several different model segments to multiple computing devices; obtaining the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; dynamically adjusting the current batch size according to the temperature warning value and operating temperature information of each computing device to obtain the optimized batch size of each computing device executing the target model segment in the current task cycle; and controlling the iterative training of the corresponding target model segment by each computing device based on the optimized batch size.

[0060] This embodiment actively drives the batch size dynamic adjustment of the device side by sensing the device operating temperature information, so that each computing device can actively reduce the computing load before the operating temperature exceeds the temperature warning value, so that each computing device enters the passive thermal protection state less often, ensuring the stability of the device side in executing model segment training, and thus avoiding the increase of unit task energy consumption of each computing device.

[0061] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment described above can be referred to the above description, and will not be repeated hereafter.

[0062] Understandably, different model segments of a large target model have different operator types, computational densities, and memory access characteristics, resulting in significant differences in energy consumption and latency performance at the same operating frequency across different segments. Existing fixed-frequency or default frequency tuning schemes cannot match the optimal energy efficiency point for different model segments and batch sizes, leading to low energy efficiency gains. Therefore, optimizing energy consumption solely through batch size adjustment as proposed in Example 1 has limited optimization depth and cannot fully tap the energy efficiency potential at different frequencies. Furthermore, if batch adjustment and frequency scheduling are independent during training without a coordinated mechanism, the frequency may be forced to be reduced by hardware due to overheating after configuration, causing the optimization strategy to fail.

[0063] Therefore, to address the shortcomings of insufficient energy consumption optimization depth relying solely on batch adjustment, please refer to the following based on the first embodiment. Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the end-side energy consumption optimization method for model segmentation in this application. In this embodiment, step S40 includes steps B1 to B3: Step B1: Obtain the model fragment identifier when each computing device executes the corresponding target model fragment; Understandably, the aforementioned model fragment identifier can refer to a globally unique model fragment identity (such as a numeric ID or string name), which can be used to distinguish different model fragments. The aforementioned target operating frequency can refer to the core operating frequency that the computing device's acceleration chip should be configured with in the current task cycle. In this case, the target operating frequency directly determines the chip's computing speed and power consumption level, and therefore can also serve as a core control parameter for edge-side power consumption optimization in this embodiment.

[0064] It is important to understand that existing solutions cannot model each model segment individually. However, different model segments have different operator types, memory access characteristics, and computational densities. The energy consumption benefits of the same GPU frequency on different segments are not the same. Therefore, if lightweight and heavy computational segments are mixed together, the accuracy of frequency selection will be low. Therefore, in this embodiment, the collaborative scheduling device can read the unique identifier corresponding to the model segment currently being carried by each computing device when scheduling each task cycle, and use it as a dimension index for subsequent frequency scheduling.

[0065] Step B2: Based on the optimized batch size and the model fragment identifier, generate the target operating frequency for each computing device; Understandably, in this embodiment, the collaborative scheduling device can use model fragment identifiers and optimized batch size as a joint index to select the frequency that meets the corresponding fragment latency constraints and has the lowest energy consumption from all configurable frequency levels supported by each computing device as the target operating frequency of each computing device.

[0066] It is important to understand that learning a separate model for each possible batch size would incur excessive exploration overhead. Therefore, this embodiment attempts to divide the batch size range into several representative intervals and maintain the energy consumption prediction model and latency prediction model online for "model segment identifier + batch interval". Since similar batch sizes typically have similar frequency-energy consumption characteristics, this embodiment can reduce the number of models and online explorations through intervalization; maintaining the model by segment avoids mixing lightweight segments and computationally intensive segments, improving the accuracy of frequency selection. Therefore, in a feasible implementation, step B2 in this embodiment may include steps B21-B24: Step B21: Select a matching batch range from the preset representative batch range according to the optimized batch size; It is understandable that the aforementioned preset representative batch interval can refer to several consecutive representative intervals pre-divided according to the value range of the batch size. It is also understandable that batch sizes within the same interval typically have similar frequency-energy consumption and frequency-latency characteristics. In this case, the aforementioned matching batch interval can refer to the preset representative batch interval into which the value corresponding to the current optimized batch size of each computing device falls, which can serve as a secondary dimension for model indexing in this embodiment. Specifically, the scheduling device can pre-divide 3 to 5 representative batch intervals according to the batch value range of the training scenario. For example, when b_min=1 and b_max=32, it can be divided into three preset batch intervals: [1,8], [9,16], and [17,32]. Then, the scheduling device can obtain the value of the current optimized batch size, compare it with the interval boundaries one by one, and determine the interval into which the value of the current optimized batch size falls, which is the matching batch interval.

[0067] Step B22: Based on the matching batch interval and the model segment identifier, determine the target energy consumption prediction model and target latency prediction model corresponding to each computing device from the pre-configured energy consumption prediction model and latency prediction model; Understandably, the aforementioned pre-configured energy consumption prediction model can refer to a mathematical model describing the mapping relationship between GPU operating frequency and fragment execution energy consumption. Its input can be the operating frequency, and its output can be the predicted energy consumption and confidence interval. Similarly, the aforementioned latency prediction model can refer to a mathematical model describing the mapping relationship between GPU operating frequency and fragment execution latency. Its input can be the operating frequency, and its output can be the predicted latency and confidence interval. Preferably, the aforementioned pre-configured energy consumption prediction model and latency prediction model can use a Gaussian process model to describe the relationship between GPU frequency and unit sample energy consumption and unit sample fragment latency, or they can use a Bayesian optimization model, multi-armed slot machine, random forest, gradient boosting tree, neural network regression, or other online learning models.

[0068] At this point, the aforementioned target energy consumption prediction model can refer to the energy consumption prediction model corresponding to the current model segment and the current matching batch interval. It is a dedicated model for predicting the energy consumption of the target model segment corresponding to each computing device under the specific scenario of the currently selected optimized batch size. Similarly, the aforementioned target latency prediction model can also be a latency prediction model corresponding to the target model segment and the current matching batch interval. It is a dedicated model for predicting the latency of the target model segment under the aforementioned specific scenario.

[0069] It should be noted that in this embodiment, the system model library corresponding to the pre-configured energy consumption prediction model and latency prediction model can adopt a two-level index structure. For example, the first-level index is the model fragment identifier, and the second-level index is the batch interval. Each index node is configured with an independent energy consumption prediction model and latency prediction model. Then, the scheduling device can use "model fragment identifier + matching batch interval" as the key to retrieve the corresponding prediction model from the energy consumption prediction model and latency prediction model configured in the system model library as the target energy consumption prediction model and target latency prediction model for each computing device. If no matching model is found in the system model library, the model parameters of adjacent intervals can be used as the initial values ​​of the model, and the model can be updated while running.

[0070] It is important to understand that the existing scheduling schemes use offline energy consumption modeling methods, which require pre-measuring all model segments, all batch sizes, and all combinations of frequencies and hardware platforms. This results in an extremely large sampling workload and very high costs. Furthermore, offline models cannot adapt to dynamic changes. When the model segmentation scheme is adjusted, equipment ages, ambient temperature changes, or heat dissipation conditions change, the prediction accuracy of the offline model will drop significantly, requiring resampling and calibration, which is inflexible.

[0071] To address this issue, in this embodiment, energy consumption prediction models and latency prediction models can be pre-maintained online for each model segment and each batch interval, respectively. That is, upon obtaining a new task cycle observation, only the two prediction models corresponding to the current model segment and the current batch interval need to be updated, allowing the scheduling strategy to adapt to changes in equipment aging, ambient temperature, load, or model partitioning scheme. Therefore, as one possible implementation, in this embodiment, step B22 may include steps B221-B223: Step B221: Based on the matching batch interval and the model segment identifier, determine the online energy consumption prediction model and online latency prediction model corresponding to each computing device from the pre-configured energy consumption prediction model and latency prediction model; Step B222: Obtain historical running data of each computing device when executing the target model fragment in a historical task cycle. The historical running data includes energy consumption data and latency data corresponding to the target model fragment under different historical running frequencies and different batch sizes. Understandably, the aforementioned historical operational data can refer to the operational records of target model segments collected and archived in past task cycles. It can at least include the energy consumption data and latency data corresponding to each target model segment at different historical operational frequencies, and may also include the segment identifier, batch size, operational frequency, temperature status, etc. of each target model segment.

[0072] Energy consumption data can refer to the total energy consumption of the target model segment during a single iteration in history. This can be calculated by integrating the device input power with the execution time, or directly read from the device power management interface, the chip's built-in power sensor, or an external power meter. Latency data, on the other hand, refers to the actual execution time of the target model segment during a single iteration in history, i.e., the time difference between loading the input tensor and generating the output tensor. This can be obtained from training framework events, inference service timers, system timers, or device monitoring programs.

[0073] In this embodiment, the historical operation data can be stored according to the "model fragment". For example, the scheduling device can archive the historical operation data to the historical operation database by using the "model fragment identifier" as the first-level index and the "batch interval" as the second-level index.

[0074] Step B223: Based on the historical operating data, update the online energy consumption prediction model and the online latency prediction model in real time to obtain the target energy consumption prediction model and the target latency prediction model corresponding to each computing device.

[0075] It should be understood that, in this context, the aforementioned real-time update can refer to the scheduling device incrementally updating the model parameters of the pre-configured energy consumption prediction model and latency prediction model after obtaining each new observation data.

[0076] In this embodiment, the scheduling device can incrementally update the matched online energy consumption prediction model and online latency prediction model in real time each time it obtains new observation data for a model segment based on historical running data. That is, when the sample points (frequency, energy consumption) and (frequency, latency) of the current model segment and the batch interval are obtained, the scheduling device can use an online learning algorithm to obtain the latest target energy consumption prediction model and target latency prediction model adapted to each computing device. During this incremental update process, the scheduling device only needs to add sample points to update the kernel function parameters, without the need for full retraining, resulting in low computational overhead, which can be completed within each task cycle.

[0077] In this embodiment, data can be automatically collected and online modeled during system training, avoiding the high-cost offline batch training required during initial deployment. Furthermore, because this embodiment can maintain separate prediction models for different model segments according to the model segmentation scheme, it can distinguish the energy efficiency characteristics of computationally intensive and memory-intensive segments, thereby improving the accuracy of frequency selection. Moreover, the energy consumption prediction and latency prediction models can dynamically adapt to changes in the device status and ambient temperature of different computing devices. Therefore, this embodiment can effectively ensure the accuracy of energy consumption prediction and latency prediction results, further improving the accuracy of frequency selection.

[0078] Step B23: Using the target latency prediction model, predict the segment execution latency of the target model segment corresponding to each computing device under multiple supported configuration running frequencies, and filter the multiple configuration running frequencies according to the segment execution latency to obtain at least one candidate running frequency corresponding to each computing device. Step B24: Predict the execution energy consumption of the target model segment at the candidate running frequency using the target energy consumption prediction model, and take the candidate running frequency with the minimum execution energy consumption as the target running frequency.

[0079] Understandably, the aforementioned operating frequencies can refer to all configurable core frequency levels supported by the GPU hardware of each computing device. These frequencies are determined by the hardware of each computing device itself and typically include dozens of levels. In this case, the aforementioned candidate operating frequencies can refer to the set of frequencies that meet the segment latency requirements after being filtered for latency constraints, representing the optimal range for energy efficiency.

[0080] It is easy to understand that when the scheduling device performs frequency scheduling, it can traverse all the configuration operating frequencies supported by each computing device based on the target energy consumption prediction model and target latency prediction model updated online for each computing device, and then input the two types of prediction models to obtain the predicted latency and predicted energy consumption corresponding to each configuration operating frequency, and select the target operating frequency that best meets the requirements based on the predicted latency and predicted energy consumption.

[0081] At this point, the scheduling device can first input each configured frequency into the target delay prediction model to obtain the predicted delay, and then retain the frequencies that meet the delay constraints based on the predicted delay to obtain at least one candidate operating frequency, forming a candidate frequency set. Next, the scheduling device can input the obtained at least one candidate frequency into the target energy consumption prediction model to obtain the predicted energy consumption of each candidate frequency, i.e., the aforementioned execution energy consumption, and then select the candidate operating frequency corresponding to the minimum execution energy as the final target operating frequency.

[0082] In this embodiment, firstly, two types of prediction models can be maintained online at a secondary modeling granularity based on model fragment identification and matching batch intervals, balancing model maintenance overhead and prediction accuracy. Simultaneously, this embodiment can ensure that the selected target operating frequency can minimize energy consumption while meeting pipeline delay requirements through a two-stage process of delay constraint screening and energy consumption optimization. This solves the contradiction between high single-batch modeling overhead and low full-batch modeling accuracy, as well as the target balance problem of delay and energy consumption in frequency optimization.

[0083] It is important to understand that the segment execution latency output by the target latency prediction model is the average predicted value for the ideal situation where the latency "might just meet the standard." This predicted average value has uncertainty (variance). If the scheduling device selects a frequency solely based on the predicted average latency, a timeout violation will occur if the actual execution latency exceeds the predicted average value, potentially blocking the entire pipeline. Since online prediction models may have errors, directly selecting based on the average value may lead to actual latency exceeding the standard. Therefore, to improve safety, this embodiment uses a conservative secondary screening method to ensure that the actual latency of each candidate operating frequency meets the constraints with a high probability, improving the robustness of frequency scheduling and avoiding pipeline blockage due to optimistic predictions. Therefore, in a feasible implementation, step B23 may include steps C1~C2: Step C1: Filter the multiple configured operating frequencies according to the preset delay constraints and the segment execution delay to obtain the initial frequency set corresponding to each computing device; Understandably, the aforementioned preset latency constraint can refer to the pre-set maximum allowable execution latency of a single segment, which is generated by the scheduling device according to the end-to-end training latency requirements of the target large model, divided into pipeline stages. This preset latency constraint can be a core constraint indicator to ensure that the pipeline does not block, and can be sent simultaneously by the scheduling device when distributing model segments to each computing device. It corresponds to the segment latency budget mentioned above.

[0084] At this point, the aforementioned initial frequency set can refer to the set of frequencies that meet the latency constraints, obtained solely based on the mean of the latency predictions. To ensure the model segmentation pipeline doesn't become blocked due to a single slow segment, frequencies whose predicted segment latency meets the segment latency budget or end-to-end latency constraints can be prioritized. Therefore, the scheduling device can obtain the segment execution latency μ_i predicted by all supported configuration running frequencies through the target latency prediction model of each computing device, compare it with the preset latency constraint T_limit, and select configuration running frequencies whose segment execution latency is less than or equal to the preset latency constraint (i.e., μ_i ≤ T_limit) to form the initial frequency set.

[0085] Step C2: Based on a pre-set confidence interval, the initial frequency set is conservatively screened to obtain at least one candidate operating frequency corresponding to each computing device.

[0086] It is easy to understand that the aforementioned preset confidence interval can refer to a pre-set confidence level (such as a 95% or 99% confidence interval), and the upper bound of the interval corresponding to the preset confidence interval can represent the worst-case time delay conservatively estimated. In this case, this embodiment can not only use the expected value (average value) of the prediction for screening, but further use the high quantile of the prediction distribution as the conservative estimate value, and make time delay constraint determination based on the conservative estimate value.

[0087] Therefore, to improve security, this embodiment can use a conservative estimate of delay prediction to further screen the initial frequency set. That is, only frequencies in the initial frequency set that still meet the delay requirements at a high confidence level are considered as feasible candidate operating frequencies. Therefore, this embodiment can ensure scheduling reliability based on this preset confidence interval.

[0088] For example, in this embodiment, it can be assumed that the preset confidence interval is 95%, corresponding to a quantile of 1.96 for the standard normal distribution (if the preset confidence interval is 99%, the corresponding quantile of the standard normal distribution is 2.326). Then, for each frequency fi in the initial frequency set F_initial, the standard deviation σ_i of its time delay prediction can be read from the target time delay prediction model, and the upper bound of the 95% confidence interval for that frequency's time delay can be calculated based on the mean and standard deviation of the time delay prediction initially output by the model: T_upper_i = μ_i + 1.96 × σ_i.

[0089] Next, the scheduling device can use the upper bound of the confidence interval T_upper_i for each frequency as a conservative estimate of the delay, compare it with the preset delay constraint T_limit, and retain only the frequencies in the initial frequency set that satisfy T_upper_i≤T_limit as the final candidate operating frequencies, forming a candidate frequency set Fcandidate containing at least one candidate operating frequency. Subsequent energy consumption optimization can be performed based on the final candidate set Fcandidate. At this time, this embodiment can use the upper bound of the 95% confidence interval for conservative screening, controlling the probability of actual delay exceeding the standard to within 5%, ensuring that the entire model segmentation pipeline will not be blocked due to single-point sudden delay.

[0090] Furthermore, if the candidate frequency set Fcandidate is empty after the second screening, the confidence level can be gradually reduced (e.g., from 95% to 90%, 80%) for re-screening until a non-empty candidate set is obtained. If it is still empty after reducing to the lowest confidence level, the frequency with the shortest prediction latency corresponding to the target model segment or the highest safe frequency of the computing device can be directly selected as a fallback scheme to ensure that the model segmentation task continues to execute. Therefore, the confidence level for the second conservative screening in this embodiment can be configured according to specific circumstances. A high confidence level can be set for scenarios with strict latency requirements, while the confidence level can be appropriately reduced for scenarios with energy consumption as a priority, flexibly adapting to different business needs.

[0091] In this embodiment, the target operating frequency is generated based on an online learning prediction model. At this time, the scheduling device can update the energy consumption and latency model online through historical operating data. It can achieve accurate frequency selection without having to complete offline sampling of the entire scenario in advance. At the same time, it has the ability to adapt the prediction model to changes in equipment aging, ambient temperature, and segmentation scheme. Therefore, this embodiment can effectively solve the problems of high sampling cost and inability to adapt to dynamic changes in existing offline energy consumption modeling.

[0092] Step B3: Based on the optimized batch size and the target running frequency, control each computing device to perform iterative training of the corresponding target model segment.

[0093] It is easy to understand that during the iterative training process of controlling each computing device to execute the corresponding target model segment, the scheduling device can jointly issue the optimized batch size and the target running frequency, so that each computing device first completes the frequency configuration and then executes the training iteration according to the specified batch.

[0094] For example, the collaborative scheduling device can encapsulate the generated optimized batch size and target operating frequency into a scheduling instruction and send it to the corresponding computing device. Upon receiving the instruction, the computing device can first set the GPU core frequency to the target operating frequency through the GPU driver interface and confirm that the frequency configuration is effective. Then, it loads training samples according to the optimized batch size, initiates the forward and backward propagation calculations of the corresponding target model segment, and completes this round of iterative training. After this round of training, it reports the actual execution latency, actual energy consumption, and final temperature data to the collaborative scheduling device so that the scheduling device can perform subsequent model updates and decisions for the next round. At this point, this embodiment can form a closed loop of "temperature monitoring—batch adjustment—frequency selection—segment execution—energy consumption and latency feedback," and repeat the closed loop process until the target large model is trained.

[0095] This embodiment discloses the following steps: obtaining the model fragment identifier when each computing device executes the corresponding target model fragment; selecting a matching batch interval from a preset representative batch interval based on the optimized batch size; determining the online energy consumption prediction model and online latency prediction model corresponding to each computing device from pre-configured energy consumption prediction models and latency prediction models based on the matching batch interval and the model fragment identifier; obtaining historical operating data of each computing device when executing the target model fragment in historical task cycles, including energy consumption data and latency data corresponding to the target model fragment under different historical operating frequencies and different batch sizes; and updating the online energy consumption prediction model and online latency prediction model in real time based on the historical operating data to obtain the corresponding data for each computing device. The system employs target energy consumption prediction models and target latency prediction models for each computing device. Using the target latency prediction model, it predicts the execution latency of target model segments on each computing device at multiple supported configuration operating frequencies. Based on preset latency constraints and segment execution latency, it filters these multiple configuration operating frequencies to obtain an initial frequency set for each computing device. It then obtains the confidence level corresponding to the segment execution latency and filters the initial frequency set based on preset confidence intervals and confidence levels to obtain at least one candidate operating frequency for each computing device. Finally, it predicts the execution energy consumption of the target model segment at the candidate operating frequencies using the target energy consumption prediction model and selects the candidate operating frequency with the lowest execution energy consumption as the target operating frequency. Based on the optimized batch size and target operating frequency, it controls each computing device to perform iterative training of the corresponding target model segment.

[0096] Therefore, this embodiment adds a GPU frequency scheduling stage to the batch adjustment strategy proposed in Embodiment 1. The purpose is to construct a two-layer closed-loop collaborative optimization method that combines batch scale adjustment and GPU frequency scheduling. In this case, the scheduling device can first suppress thermal risks and stabilize the temperature range through batch adjustment, and then select the optimal energy-efficient frequency within the safe temperature range. The two form a closed-loop collaboration. Compared with the single adjustment method, this embodiment can not only further reduce the unit task energy consumption of the computing device, but also suppress thermal risks in advance through batch adjustment, so that the target frequency generated by scheduling can be executed stably, avoiding interruption by hardware thermal throttling, and improving the effectiveness of the optimization strategy.

[0097] Furthermore, this embodiment can schedule the operating frequency of each computing device separately according to model segments, thereby effectively adapting to the computing characteristics of different segments. Compared with the existing global unified frequency adjustment method, it can effectively improve the optimization accuracy.

[0098] For example, to help understand the technical concept or principle of the model-oriented end-side energy consumption optimization method after combining this embodiment with the above-described Embodiments 1 and 2, please refer to Figure 3 , Figure 3The following is a simplified schematic diagram of the end-side energy consumption optimization method for model segmentation in this application: It is important to understand that, in order to address the problems of hotspot segments causing device overheating, deployment failure, latency jitter, pipeline blockage, and increased energy consumption in existing model segmentation training scenarios, this application can construct a basic framework for edge-side energy consumption optimization for model segmentation training scenarios in the central node of the system, i.e., the collaborative scheduling device. For example... Figure 3 As shown, the framework may include a device behavior monitoring module, a temperature sensing batch size control module, an online GPU frequency scheduling module, a frequency configuration execution module, and a task execution module.

[0099] It is easy to understand that the core objective of this framework is to proactively drive the dynamic adjustment of batch scale by sensing the real-time operating temperature of computing devices, thereby reducing the computing load in advance before the device temperature triggers the hardware passive thermal protection, reducing the probability of computing devices entering a thermal throttling state, and thus reducing overall energy consumption while ensuring training stability.

[0100] The device behavior monitoring module can be used to collect GPU temperature, current GPU frequency, input power, fragment execution latency, batch size, video memory usage, fragment identification, and task status. The temperature-sensing batch size control module can be used to determine the current batch size or micro-batch size based on the temperature status. The online GPU frequency scheduling module can be used to determine the target GPU frequency based on the current segment, batch interval, and historical observation data; The frequency configuration execution module can be used to write the target frequency to the device driver, system interface, or vendor management interface. The task execution module can be used to execute corresponding segments according to a defined batch size and frequency, and pass the execution results, intermediate tensors or gradients to the next segment or cooperating device.

[0101] For example, assuming the application scenario of this application is an edge-cloud collaborative inference scenario, the multiple computing devices in the model collaborative training system can be configured with front-end execution devices such as cameras, robots, or mobile terminals, and back-end execution devices such as edge servers or cloud GPUs. The front-end execution devices can be used to perform the first few layers of feature extraction according to the existing model segmentation scheme, and send the intermediate tensors to the back-end execution devices in the system to execute subsequent segments.

[0102] Once the model co-training system is started, the co-scheduling device can receive a predetermined model segmentation scheme through the model segmentation task interface. This model segmentation scheme may include the number of model segments, the set of layers or operators corresponding to each segment, the segment deployment device, the size of the segment input and output tensors, the segment latency budget, and the data transmission relationship between segments. It should be noted that model segmentation in this application is mainly used as an application scenario; that is, this application can integrate existing graph partitioning, pipelined parallelism, edge-cloud collaboration, or manually specified model segmentation schemes, and does not require changing the model structure or retraining the model.

[0103] Next, as Figure 3 As shown, the device behavior monitoring module of the collaborative scheduling equipment continuously collects the temperature, power consumption, and segment latency of each execution device. Before each inference batch begins, the temperature-sensing batch size control module can determine the temperature status of the computing device corresponding to the current execution hotspot segment based on the device temperature transmitted by the device behavior detection module, i.e., the aforementioned real-time operating temperature. At this point, if the temperature sensing batch size control module detects that the GPU temperature of the computing device is within a safe range based on the temperature warning value, it gradually increases the micro-batch size to improve throughput; if the real-time operating temperature is close to the warning range or it is judged based on the rate of temperature increase that the computing device is about to exceed the safe operating temperature of the device, it reduces the micro-batch size to reduce the computing load of that segment.

[0104] Subsequently, the online GPU frequency scheduling module can select the low-power frequency that meets the segment latency budget as the optimal GPU frequency, i.e., the aforementioned target operating frequency, based on the target model segment and real-time batch interval corresponding to each computing device, i.e., the multiple configuration operating frequencies supported above.

[0105] Finally, the frequency configuration execution module writes the target operating frequency into the computing devices, enabling each computing device to perform iterative training of the corresponding target model segment. At this point, after the online energy consumption prediction model and online latency prediction model, which are scheduled by model segment identifier and batch interval in this application, are updated based on the historical GPU frequency transmitted by the device behavior detection module, the updated target energy consumption prediction model and target latency prediction model can quickly identify the energy consumption differences of the current model segment under different selectable configuration frequencies, avoiding expensive offline measurements for all frequencies and all batch combinations.

[0106] In summary, the end-side energy consumption optimization method for model segmentation proposed in this embodiment can achieve the following effects: First, by proactively reducing the batch / micro-batch size before thermal throttling occurs, the probability of the GPU of the computing device entering passive thermal protection is reduced; Second, by learning the frequency-energy consumption and frequency-latency relationships online, the cost of offline testing can be reduced, and it can be adapted to different model segmentation schemes, different devices and different ambient temperatures; Third, by putting batch control and frequency scheduling into the same closed loop, the energy consumption per unit task can be reduced while meeting the requirements of segment delay or end-to-end delay. Fourth, by maintaining scheduling information according to model segments, hot segments, lightweight segments, and communication-sensitive segments can adopt different energy consumption optimization strategies, thereby improving the overall stability of the model segmentation system.

[0107] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the end-side energy consumption optimization method for model segmentation in this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0108] This application also provides an end-side energy consumption optimization device for model segmentation; please refer to [reference needed]. Figure 4 , Figure 4 This is a schematic diagram of the module structure of the end-side energy consumption optimization device for model segmentation according to an embodiment of this application. In this embodiment, the end-side energy consumption optimization device for model segmentation includes: The model segmentation module 401 is used to divide the target large model to be trained into several different model segments according to a preset model segmentation scheme, and send the several different model segments to multiple computing devices respectively. The device behavior monitoring module 402 is used to obtain the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; The batch adjustment module 403 is used to dynamically adjust the current batch size according to the temperature warning value corresponding to each computing device and the operating temperature information, so as to obtain the optimized batch size of each computing device executing the target model segment in the current task cycle; The energy consumption optimization module 404 is used to control each of the computing devices to perform iterative training of the corresponding target model segment based on the optimized batch size.

[0109] The edge-side energy consumption optimization device for model segmentation provided in this application adopts the edge-side energy consumption optimization method for model segmentation in the above embodiments. It can solve the technical problem of how to ensure the service stability of each computing device in the model segmentation scenario, thereby avoiding the increase of unit task energy consumption of the computing device. Compared with the prior art, the beneficial effects of the edge-side energy consumption optimization device for model segmentation provided in this application are the same as the beneficial effects of the edge-side energy consumption optimization method for model segmentation provided in the above embodiments, and other technical features in the edge-side energy consumption optimization device for model segmentation are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0110] This application provides a collaborative scheduling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the model-segmentation-oriented end-side energy consumption optimization method in Embodiment 1 above.

[0111] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a collaborative scheduling device suitable for implementing the embodiments of this application. The collaborative scheduling device in the embodiments of this application may include, but is not limited to, independent scheduling servers, cluster management nodes, or edge control nodes, and other computing platforms with high computing power and storage capabilities; it may also be a collection of servers consisting of edge servers and cloud servers. This embodiment does not specifically limit this. Figure 5 The cooperative scheduling device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0112] like Figure 5 As shown, the collaborative scheduling device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the collaborative scheduling device. In this application, user-related data (e.g., user attribute data, user behavior data, and user geographical location, etc.; the data types here should be adapted according to the content of the solution) involved in the model training process are all obtained after obtaining the user's permission or consent; that is, when this application is applied to a specific product or technology, user permission is required to obtain and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations, and regulatory standards of the relevant countries and regions.

[0113] Furthermore, the processing unit 1001, read-only memory 1002, and random access memory 1004 in the collaborative scheduling device are interconnected via bus 1005. Input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the collaborative scheduling device to exchange data wirelessly or via wired communication with other computing devices in the system. Although a collaborative scheduling device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems can be implemented or possessed alternatively.

[0114] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in this application include a model-segmentation-oriented edge power optimization program product, which includes a model-segmentation-oriented edge power optimization program carried on a computer-readable medium, the model-segmentation-oriented edge power optimization program containing program code for performing the methods shown in the flowcharts. In such embodiments, the model-segmentation-oriented edge power optimization program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the model-segmentation-oriented edge power optimization program is executed by processing device 1001, the functions defined in the methods of the embodiments disclosed in this application are performed.

[0115] The collaborative scheduling device provided in this application employs the edge-side energy consumption optimization method for model segmentation described in the above embodiments. This method addresses the technical problem of ensuring service stability for each computing device during model segmentation training in a model segmentation scenario, thereby preventing increased energy consumption per unit task for the computing device. Compared to existing technologies, the beneficial effects of the collaborative scheduling device provided in this application are the same as those of the edge-side energy consumption optimization method for model segmentation provided in the above embodiments. Furthermore, other technical features of this collaborative scheduling device are the same as those disclosed in the previous embodiment, and will not be elaborated upon here.

[0116] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0117] Furthermore, to achieve the above objectives, this application also proposes a model collaborative training system, including at least one collaborative scheduling device mentioned above and multiple computing devices, which can be used to implement the end-side energy consumption optimization method for model segmentation as mentioned in Embodiment 1 and / or Embodiment 2 above. The collaborative workflow of the end-side energy consumption optimization method for model segmentation can refer to Embodiment 1 and / or Embodiment 2 above. Since this model collaborative training system adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated here.

[0118] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0119] This application provides a storage medium having computer-readable program instructions (i.e., an end-side energy consumption optimization program for model segmentation) stored thereon, which are used to execute the end-side energy consumption optimization method for model segmentation in the above embodiments.

[0120] The storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of the storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0121] The aforementioned storage medium may be included in the collaborative scheduling device; or it may exist independently and not be assembled into the collaborative scheduling device.

[0122] The aforementioned storage medium carries one or more programs. When these programs are executed by the coordinated scheduling device, the coordinated scheduling device performs end-side energy consumption optimization for model segmentation.

[0123] Model-segmented end-side energy consumption optimization program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and model-based segmentation end-side energy optimization program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0126] The readable storage medium provided in this application is a storage medium that stores computer-readable program instructions (i.e., an edge-side energy consumption optimization program for model segmentation) for executing the aforementioned edge-side energy consumption optimization method for model segmentation. This solves the technical problem of ensuring the service stability of each computing device executing model segment training in a model segmentation scenario, thereby avoiding increased energy consumption per unit task for the corresponding computing device. Compared with the prior art, the beneficial effects of the storage medium provided in this application are the same as those of the edge-side energy consumption optimization method for model segmentation provided in the above embodiments, and will not be repeated here.

[0127] The above are only some embodiments of this application and do not limit the scope of the solution of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.

Claims

1. An end-side energy consumption optimization method for model segmentation, characterized in that, The method is applied to a model co-training system, the system including at least one co-scheduling device and multiple computing devices, the method being executed by the co-scheduling device, and the method comprising: The target large model to be trained is divided into several different model segments according to a preset model segmentation scheme, and the several different model segments are sent to the multiple computing devices respectively; Obtain the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; The current batch size is dynamically adjusted based on the temperature warning value corresponding to each computing device and the operating temperature information to obtain the optimized batch size for each computing device to execute the target model fragment in the current task cycle; Based on the optimized batch size, each computing device is controlled to perform iterative training of the corresponding target model segment.

2. The end-side energy consumption optimization method for model segmentation as described in claim 1, characterized in that, The operating temperature information includes real-time operating temperature and temperature rise rate; the step of dynamically adjusting the current batch size based on the temperature warning value corresponding to each computing device and the operating temperature information to obtain the optimized batch size for each computing device to execute the target model fragment in the current task cycle includes: Obtain a temperature comparison result generated by comparing the temperature warning value corresponding to each of the computing devices with the real-time operating temperature, and obtain a temperature exceedance warning result generated based on the real-time operating temperature and the temperature rise rate. The target adjustment range and target adjustment direction are determined based on the temperature comparison results and / or the temperature exceedance warning results. The current batch size is dynamically adjusted based on the target adjustment magnitude and the target adjustment direction to obtain the optimized batch size for each computing device to execute the target model fragment in the current task cycle.

3. The method as described in claim 1, characterized in that, The step of controlling each computing device to perform iterative training of the corresponding target model segment based on the optimized batch size includes: Obtain the model fragment identifier when each of the aforementioned computing devices executes the corresponding target model fragment; Based on the optimized batch size and the model fragment identifier, the target operating frequency of each computing device is generated; Based on the optimized batch size and the target running frequency, each computing device is controlled to perform iterative training of the corresponding target model segment.

4. The method as described in claim 3, characterized in that, The step of generating the target operating frequency for each computing device based on the optimized batch size and the model fragment identifier includes: Based on the optimized batch size, select a matching batch range from the preset representative batch range; Based on the matching batch interval and the model segment identifier, the target energy consumption prediction model and target latency prediction model corresponding to each computing device are determined from the pre-configured energy consumption prediction model and latency prediction model. The target latency prediction model is used to predict the execution latency of the target model segment corresponding to each computing device under multiple supported configuration operating frequencies, and the multiple configuration operating frequencies are filtered according to the segment execution latency to obtain at least one candidate operating frequency corresponding to each computing device. The target energy consumption prediction model predicts the execution energy consumption of the target model segment at the candidate running frequency, and the candidate running frequency with the minimum execution energy consumption is taken as the target running frequency.

5. The method as described in claim 4, characterized in that, The step of determining the target energy consumption prediction model and target latency prediction model corresponding to each computing device from pre-configured energy consumption prediction models and latency prediction models based on the matching batch interval and the model segment identifier includes: Based on the matching batch interval and the model segment identifier, the online energy consumption prediction model and online latency prediction model corresponding to each computing device are determined from the pre-configured energy consumption prediction model and latency prediction model; Acquire historical running data of each computing device when executing the target model fragment in a historical task cycle. The historical running data includes energy consumption data and latency data corresponding to the target model fragment under different historical running frequencies and different batch sizes. Based on the historical operating data, the online energy consumption prediction model and the online latency prediction model are updated in real time to obtain the target energy consumption prediction model and the target latency prediction model corresponding to each computing device.

6. The end-side energy consumption optimization method for model segmentation as described in claim 4, characterized in that, The step of filtering the plurality of configured operating frequencies based on the fragment execution delay to obtain at least one candidate operating frequency corresponding to each computing device includes: Based on the preset delay constraints and the execution delay of the segment, the multiple configured operating frequencies are filtered to obtain the initial frequency set corresponding to each computing device; Based on a pre-set confidence interval, the initial frequency set is conservatively screened to obtain at least one candidate operating frequency corresponding to each computing device.

7. An end-side energy consumption optimization device for model segmentation, characterized in that, The device includes: The model segmentation module is used to divide the target large model to be trained into several different model segments according to a preset model segmentation scheme, and send the several different model segments to several computing devices respectively; The device behavior monitoring module is used to obtain the operating temperature information and current batch size of each computing device when executing the corresponding target model segment; The batch adjustment module is used to dynamically adjust the current batch size according to the temperature warning value corresponding to each computing device and the operating temperature information, so as to obtain the optimized batch size of each computing device executing the target model segment in the current task cycle; The energy consumption optimization module is used to control each of the computing devices to perform iterative training of the corresponding target model segment based on the optimized batch size.

8. A collaborative scheduling device, characterized in that, The device includes: a memory, a processor, and an edge-side energy consumption optimization program for model segmentation stored in the memory and executable on the processor, the edge-side energy consumption optimization program for model segmentation configured to implement the steps of the edge-side energy consumption optimization method for model segmentation as described in any one of claims 1 to 7.

9. A model collaborative training system, characterized in that, The system includes at least one collaborative scheduling device as described in claim 8 and several computing devices.

10. A storage medium, characterized in that, The storage medium stores an edge power consumption optimization program for model segmentation, which, when executed by a processor, implements the steps of the edge power consumption optimization method for model segmentation as described in any one of claims 1 to 6.

Images