5G Smart Chip Dynamic Computing Power Allocation Method Supporting Multimodal Data
By using multimodal data acquisition and preprocessing, an improved LSTM-Transformer hybrid model for prediction, weighted allocation, energy consumption optimization, and cross-layer collaborative scheduling, the problem of lagging and low accuracy in computing power demand prediction caused by the heterogeneity of multimodal data is solved. This addresses the disconnect between computing power, energy consumption, and memory in existing technologies, enabling accurate prediction and proactive allocation of computing power demand, and improving computing power utilization and adaptive capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU CHICOWAY INFORMATION TECH CO LTD
- Filing Date
- 2026-04-14
- Publication Date
- 2026-07-10
AI Technical Summary
Existing 5G smart chips suffer from several problems in multimodal data processing, including a lack of multimodal adaptability in computing power allocation, a lack of accurate computing power demand prediction mechanisms, a disconnect between multimodal task priority scheduling and computing power allocation, an imbalance between energy consumption and computing power allocation, and insufficient cross-layer collaboration. These issues make it difficult to meet the high-concurrency, real-time, and low-energy consumption processing requirements of multimodal data.
By employing multimodal data acquisition and preprocessing, an improved LSTM-Transformer hybrid model for computing power demand prediction, a weighted allocation algorithm for initial allocation, dynamic computing power adjustment, energy consumption optimization and computing power constraints, deep reinforcement learning algorithm iterative optimization, and cross-layer collaborative scheduling, we can achieve precise allocation of computing power resources and energy consumption management.
It achieves deep coordination of multimodal task priority, 5G communication rate, chip power consumption, graphics memory resources and computing power allocation, solves the problem of disconnect between computing power and power consumption and graphics memory, realizes the four-dimensional balance of "computing power-power consumption-performance-graphics memory", adapts to the development trend of integrated computing, and improves computing power utilization and adaptive capability.
Smart Images
Figure CN122363902A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a dynamic computing power allocation method for 5G smart chips that supports multimodal data. Background Technology
[0002] With the widespread adoption of 5G communication technology and the rapid development of multimodal artificial intelligence, 5G smart chips have become the core carrier connecting communication transmission and data processing. The rationality of their computing power allocation directly determines the efficiency, latency, and energy consumption of multimodal data processing. Currently, multimodal data applications in 5G scenarios exhibit significant characteristics of "high concurrency, heterogeneity, strong real-time performance, and high volatility." For example, in the vehicle-to-everything (V2X) scenario, it is necessary to simultaneously process image data from vehicle cameras, distance sensor data from radar, audio data from voice interaction, and control data from 5G communication. In the industrial internet scenario, it is necessary to simultaneously process video data from equipment monitoring, real-time data collected by sensors, and text data from command interaction. These multimodal data vary greatly in terms of data volume, processing complexity, and real-time requirements, posing extremely high challenges to the computing power allocation of 5G smart chips.
[0003] Most existing 5G smart chip computing power allocation methods are designed based on traditional single-modal data processing scenarios, which have many technical bottlenecks and cannot adapt to the needs of multimodal data concurrent processing. Specific shortcomings include: 1. Lack of multimodal adaptability in computing power allocation: Existing methods mostly adopt fixed computing power allocation strategies or only design dynamic adjustment logic for single-modal data, failing to consider the heterogeneity of multimodal data—different modal data (e.g., image data requires high computing power for parallel processing, while text data requires low computing power for serial processing) have significantly different computing power requirements. Fixed allocation modes lead to insufficient computing power for high-demand modalities and idle computing power for low-demand modalities, resulting in wasted computing resources. Furthermore, they cannot meet the real-time requirements of multimodal data collaborative processing. For example, when a 5G smart chip simultaneously receives image and text data, if an average computing power allocation is used, image data processing latency increases due to insufficient computing power, while text data resource waste occurs due to excessive computing power, severely impacting the experience of multimodal applications.
[0004] 2. Lack of accurate computing power demand prediction mechanism: The generation of multimodal data is random and volatile. For example, the amount of video call data increases suddenly during peak hours, and sensor data is generated intensively in sudden scenarios. Existing methods are mostly based on real-time computing power load and make passive adjustments. They do not predict the computing power demand of multimodal data in advance, which leads to the lag in computing power allocation adjustment. This makes it impossible to cope with sudden fluctuations in computing power demand and is prone to extreme situations of "computing power overload" or "computing power idle". Especially in 5G low-latency scenarios (such as telemedicine and autonomous driving), the lag will directly affect the reliability and security of the application.
[0005] 3. Disconnect between multimodal task priority scheduling and computing power allocation: Multimodal tasks in 5G scenarios have clear priority differences. For example, in the Internet of Vehicles, the collision warning task of radar data has a higher priority than the voice interaction task. In the Industrial Internet, the sensor data processing for equipment fault monitoring has a higher priority than the video surveillance data processing. However, most existing methods do not deeply bind task priority with computing power allocation, but only allocate computing power based on the order of data generation or the size of data. This results in high-priority tasks being unable to be processed in time due to insufficient computing power, while low-priority tasks consume too much computing power, affecting the operating efficiency and reliability of the entire system.
[0006] 4. Imbalance between energy consumption and computing power allocation: The computing power output of 5G smart chips is positively correlated with energy consumption. Existing dynamic computing power allocation methods mostly focus on improving computing power utilization and neglect energy consumption optimization. Especially in power-sensitive scenarios such as mobile terminals and edge nodes, excessive pursuit of high computing power output will lead to a surge in chip energy consumption and shorten device battery life. On the other hand, excessive restriction of computing power will affect the efficiency of multimodal data processing and make it impossible to achieve a balance between "computing power-energy consumption-performance".
[0007] 5. Lack of cross-layer collaborative computing power scheduling capabilities: Existing computing power allocation methods are mostly limited to local adjustments at the chip computing layer, failing to coordinate with the 5G communication layer and data preprocessing layer. This leads to a mismatch between computing power allocation and data transmission rates and data preprocessing efficiency. For example, when the 5G communication layer's transmission rate increases, the amount of data input increases, but the computing power allocation is not adjusted in time, resulting in data backlog. After the data preprocessing layer is optimized, the data processing complexity decreases, but the computing power allocation is not reduced accordingly, resulting in wasted computing power. At the same time, existing methods do not consider the dynamic changes in computing power requirements during multimodal data fusion, and cannot adapt to the complex scenarios of cross-modal fusion computing.
[0008] In summary, current methods for allocating computing power in 5G smart chips that support multimodal data suffer from drawbacks such as poor adaptability, prediction lag, priority disconnect, energy consumption imbalance, insufficient cross-layer collaboration, and inability to adapt to the trend of integrated computing. These shortcomings make it difficult to meet the high-concurrency, real-time, and low-energy-consumption processing requirements of multimodal data in 5G scenarios. Therefore, developing a dynamic computing power allocation method for 5G smart chips that can accurately adapt to the characteristics of multimodal data, achieve dynamic optimization of computing power allocation, balance performance and energy consumption, and support cross-layer collaboration and integrated computing has become an urgent technical problem to be solved in this field. Summary of the Invention
[0009] In view of the problems mentioned in the background art, the purpose of this invention is to provide a dynamic computing power allocation method for 5G smart chips that supports multimodal data, so as to solve the problems mentioned in the background art.
[0010] The above-mentioned technical objective of the present invention is achieved through the following technical solution: a dynamic computing power allocation method for 5G smart chips that supports multimodal data, comprising the following steps: S1. Multimodal data acquisition and preprocessing: the 5G smart chip receives external multimodal data through the 5G communication unit, and simultaneously acquires multimodal data generated locally by the chip. All multimodal data are classified, denoised, and standardized preprocessed. The core features of each modal data are extracted, and the processing tasks and task priorities corresponding to each modal data are labeled.
[0011] S2. Multimodal computing power demand prediction: Based on the preprocessed multimodal data characteristics, combined with historical computing power allocation data, 5G communication rate fluctuation data and multimodal task execution logs, the prediction unit uses an improved LSTM-Transformer hybrid model to predict the peak computing power demand, average computing power demand and demand fluctuation range of each modal data processing task in the future preset time period. At the same time, it predicts the impact of 5G communication rate changes on computing power demand and outputs the computing power demand prediction results.
[0012] S3. Initial allocation of computing power resources: The computing power scheduling unit performs initial allocation of computing power using a weighted allocation algorithm based on the real-time computing power requirements of each modality, task priorities, and the chip's total computing power limit. The weights of task priority, data processing complexity, and real-time requirements are dynamically adjusted according to the 5G application scenario.
[0013] S4. Dynamic computing power adjustment: The computing power scheduling unit monitors the actual computing power consumption of each mode of data processing, task execution progress, 5G communication rate changes and chip power consumption status in real time, and performs dynamic computing power adjustment based on the computing power demand prediction results in step S2.
[0014] S5. Energy Consumption Optimization and Computing Power Constraints: The energy consumption monitoring unit collects the chip's energy consumption data and temperature data in real time. When the chip's energy consumption exceeds the preset threshold or the temperature exceeds the safety threshold, the computing power scheduling unit dynamically reduces the computing power output of low-priority tasks without affecting the real-time processing of high-priority tasks, or uses dynamic voltage and frequency scaling technology to adjust the computing power output level.
[0015] S6. Computing power allocation feedback and model iteration: Record the results of each computing power allocation, the execution efficiency of each modal task, latency data, energy consumption data, and 5G communication adaptation status to form feedback data; based on the feedback data, use deep reinforcement learning algorithms to iteratively optimize the parameters of the computing power demand prediction model in step S2, the weights of the weighted allocation algorithm in step S3, and the dynamic adjustment strategy in step S4.
[0016] S7. Integrated Computing and Coordinated Scheduling: When 5G communication load and multimodal AI processing load coexist, the computing power scheduling unit realizes cross-load dynamic scheduling of computing resources based on the real-time load fluctuations of both. When 5G communication is busy, priority is given to ensuring the computing power demand of vRAN load and reducing the computing power of non-real-time multimodal AI tasks. When 5G communication is idle, idle computing power is fed back to multimodal AI processing tasks.
[0017] Preferably, the multimodal data includes text data, image data, audio data, video data, and sensor data, and the core features include data volume, processing complexity, real-time requirements, and data sparsity; the task priority is preset to three levels: high, medium, and low according to the 5G application scenario, wherein emergency response tasks (collision warning, fault monitoring) are set to high priority, regular interaction tasks (voice interaction, text transmission) are set to medium priority, and non-real-time processing tasks (data backup, historical data analysis) are set to low priority.
[0018] Preferably, in step S2, the improved LSTM-Transformer hybrid model introduces a cross-modal attention mechanism to mine the correlation between features of different modal data and improve the accuracy of computing power demand prediction; at the same time, it combines gradient checkpointing technology to optimize memory usage and adapt to the memory requirements of multimodal model processing; the value range of the future preset time period is 10ms-100ms, which can be dynamically adjusted according to the real-time requirements of 5G application scenarios.
[0019] Preferably, in step S3, the calculation formula of the weighted allocation algorithm is: computing power allocation ratio = (task priority weight × priority coefficient + processing complexity weight × complexity coefficient + real-time requirement weight × real-time coefficient) / total weight sum; wherein, the priority coefficient, complexity coefficient, and real-time coefficient are all obtained by normalizing the quantized values of the corresponding features, and the value range is 0-1; the total weight sum is the sum of the task priority weight, processing complexity weight, and real-time requirement weight, and the value is 1.
[0020] Preferably, in step S4, the dynamic computing power adjustment specifically includes: S41. When the actual computing power consumption of a certain modality data processing exceeds the predicted peak and the task execution progress is lagging, computing power is preferentially allocated from the idle computing power of low-priority tasks. If the idle computing power is insufficient, the computing power allocation ratio of low-priority tasks is dynamically compressed.
[0021] S42. When the actual computing power consumption of a certain modality data processing is lower than the predicted average and there is idle computing power, the idle computing power is allocated to modality tasks with a large computing power demand gap, or the computing power output of the modality processing unit is reduced.
[0022] S43. When the 5G communication rate increases and the amount of multimodal data input increases, the computing power allocation of the corresponding mode processing unit is increased synchronously; when the 5G communication rate decreases and the amount of data input decreases, the computing power allocation of the corresponding mode processing unit is reduced accordingly.
[0023] S44. During cross-modal fusion, the collaborative computing power of each modal processing unit is dynamically allocated according to the complexity of the fusion task.
[0024] Preferably, in step S5, the energy consumption optimization further includes: combining a sparse matrix optimization algorithm to perform redundant calculation pruning on the processing of low-complexity multimodal data, thereby reducing energy consumption and computing power consumption; the safety threshold is 75℃-80℃, which can be dynamically adjusted according to the chip model.
[0025] Preferably, in step S6, the deep reinforcement learning algorithm adopts the PPO algorithm, whose state space is a set of multimodal data features, computing power consumption, 5G communication rate, and chip energy consumption, whose action space is the adjustment amount of computing power allocation ratio, and whose reward function is a comprehensive evaluation function of computing power utilization, task latency, and energy consumption.
[0026] Preferably, in step S7, the computing power requirement of the vRAN load has a higher priority than that of non-real-time multimodal AI tasks but lower priority than that of high-priority multimodal tasks; the response time of computing power cross-load scheduling does not exceed 10ms, ensuring the coordinated stability of 5G communication and multimodal processing.
[0027] Preferably, the 5G smart chip also integrates a video memory scheduling unit. During the initialization allocation process in step S3, the coordinated allocation of video memory resources and computing power resources is considered simultaneously. The ratio of video memory to computing power is reasonably divided according to the video memory requirements of multimodal data to avoid video memory bottlenecks. The video memory requirements are calculated based on the feature sparsity, processing complexity and model parameter scale of the multimodal data.
[0028] Preferably, the method is applicable to 5G multimodal application scenarios such as edge computing, vehicle networking, industrial internet, smart healthcare, and smart terminals; the computing power output level of the multimodal processing unit can be continuously adjusted from 0.1 TOPS to 10 TOPS according to a dynamic adjustment strategy to adapt to the multimodal data processing needs of different complexities.
[0029] In summary, the present invention has the following main advantages: The present invention proposes a closed-loop computing power allocation architecture of multimodal feature perception-demand prediction-dynamic allocation-feedback iteration. For the first time, it combines an improved LSTM-Transformer hybrid model with a cross-modal attention mechanism, which solves the problems of lagging and low accuracy in computing power demand prediction caused by the heterogeneity of multimodal data. At the same time, it combines gradient checkpointing technology to optimize memory usage and adapt to the problem of surge in memory demand caused by multimodal model fine-tuning. It breaks through the limitation of existing methods that can only passively respond to computing power demand, and realizes accurate prediction and active allocation of computing power demand. This is a core innovation point that has not been addressed in the prior art.
[0030] This invention achieves deep coordination of multimodal task priority, 5G communication rate, chip power consumption, graphics memory resources and computing power allocation. It designs a dynamic weight allocation algorithm and cross-layer collaborative scheduling logic, which not only considers the heterogeneity of multimodal data, but also realizes the linkage of the 5G communication layer, data preprocessing layer and chip computing layer. At the same time, it solves the problem of the disconnect between computing power and power consumption and graphics memory in existing methods, and achieves a four-dimensional balance of "computing power-power consumption-performance-graphics memory", which is in line with the development trend of integrated computing.
[0031] This invention introduces deep reinforcement learning algorithms and sparse matrix optimization algorithms to construct an adaptive computing power allocation iterative mechanism. It can automatically optimize the computing power allocation strategy according to the dynamic changes of 5G application scenarios, the fluctuation of multimodal data, and the chip operating status, without manual intervention. This overcomes the shortcomings of existing methods that require manual parameter preset and have poor adaptability, and significantly improves the versatility and adaptability of the method.
[0032] This invention designs a unified computing power collaborative scheduling logic to realize dynamic computing power sharing between 5G communication load and multimodal AI processing load. It solves the problem of computing power silos caused by the separation of the two in the existing technology. It can flexibly allocate computing power resources according to the peak and trough changes of 5G communication load and AI load, and greatly improve the computing power utilization rate. This is fundamentally different from the existing computing power allocation methods that focus on multimodal processing or 5G communication.
[0033] This invention addresses the computational power coordination requirements in the multimodal data fusion process by designing a dedicated computational power allocation strategy. Combined with a cross-modal attention mechanism, it ensures the efficiency and accuracy of multimodal data fusion while avoiding coordination imbalance caused by excessive occupation of single-modal computational power. This breakthrough overcomes the technical bottleneck of existing methods that do not consider the computational power requirements of cross-modal fusion. Attached Figure Description
[0034] Figure 1 This is a flowchart of the present invention. Detailed Implementation
[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0036] refer to Figure 1 The method for dynamic computing power allocation of 5G smart chips that supports multimodal data includes the following steps: S1. Multimodal data acquisition and preprocessing: The 5G smart chip receives external multimodal data through the 5G communication unit, and at the same time acquires multimodal data generated locally by the chip. All multimodal data are classified, denoised, and standardized preprocessed. The core features of each modal data are extracted, and the processing tasks and task priorities corresponding to each modal data are marked.
[0037] S2. Multimodal computing power demand prediction: Based on the preprocessed multimodal data characteristics, combined with historical computing power allocation data, 5G communication rate fluctuation data and multimodal task execution logs, the prediction unit uses an improved LSTM-Transformer hybrid model to predict the peak computing power demand, average computing power demand and demand fluctuation range of each modal data processing task in the future preset time period. At the same time, it predicts the impact of 5G communication rate changes on computing power demand and outputs the computing power demand prediction results.
[0038] S3. Initial allocation of computing power resources: The computing power scheduling unit performs initial allocation of computing power using a weighted allocation algorithm based on the real-time computing power requirements of each modality, task priorities, and the chip's total computing power limit. The weights of task priority, data processing complexity, and real-time requirements are dynamically adjusted according to the 5G application scenario.
[0039] S4. Dynamic computing power adjustment: The computing power scheduling unit monitors the actual computing power consumption of each mode of data processing, task execution progress, 5G communication rate changes and chip power consumption status in real time, and performs dynamic computing power adjustment based on the computing power demand prediction results in step S2.
[0040] S5. Energy Consumption Optimization and Computing Power Constraints: The energy consumption monitoring unit collects the chip's energy consumption data and temperature data in real time. When the chip's energy consumption exceeds the preset threshold or the temperature exceeds the safety threshold, the computing power scheduling unit dynamically reduces the computing power output of low-priority tasks without affecting the real-time processing of high-priority tasks, or uses dynamic voltage and frequency scaling technology to adjust the computing power output level.
[0041] S6. Computing power allocation feedback and model iteration: Record the results of each computing power allocation, the execution efficiency of each modal task, latency data, energy consumption data, and 5G communication adaptation status to form feedback data; based on the feedback data, use deep reinforcement learning algorithms to iteratively optimize the parameters of the computing power demand prediction model in step S2, the weights of the weighted allocation algorithm in step S3, and the dynamic adjustment strategy in step S4.
[0042] S7. Integrated Computing and Coordinated Scheduling: When 5G communication load and multimodal AI processing load coexist, the computing power scheduling unit realizes cross-load dynamic scheduling of computing resources based on the real-time load fluctuations of both. When 5G communication is busy, priority is given to ensuring the computing power demand of vRAN load and reducing the computing power of non-real-time multimodal AI tasks. When 5G communication is idle, idle computing power is fed back to multimodal AI processing tasks.
[0043] refer to Figure 1 Multimodal data includes text data, image data, audio data, video data, and sensor data. Its core characteristics include data volume, processing complexity, real-time requirements, and data sparsity. Task priorities are preset to three levels—high, medium, and low—based on 5G application scenarios. Emergency response tasks (collision warning, fault monitoring) are set to high priority, routine interaction tasks (voice interaction, text transmission) are set to medium priority, and non-real-time processing tasks (data backup, historical data analysis) are set to low priority.
[0044] refer to Figure 1 In S2, the improved LSTM-Transformer hybrid model introduces a cross-modal attention mechanism to explore the correlation between features of different modal data and improve the accuracy of computing power demand prediction. At the same time, it combines gradient checkpointing technology to optimize memory usage and adapt to the memory requirements of multimodal model processing. The future preset time period range is 10ms-100ms, which can be dynamically adjusted according to the real-time requirements of 5G application scenarios.
[0045] refer to Figure 1 In S3, the weighted allocation algorithm is calculated as follows: Computing power allocation ratio = (task priority weight × priority coefficient + processing complexity weight × complexity coefficient + real-time requirement weight × real-time coefficient) / total weight sum; where the priority coefficient, complexity coefficient, and real-time coefficient are all obtained by normalizing the quantized values of the corresponding features, and the value range is 0-1; the total weight sum is the sum of the task priority weight, processing complexity weight, and real-time requirement weight, and the value is 1.
[0046] refer to Figure 1In S4, dynamic computing power adjustment specifically includes: S41. When the actual computing power consumption of a certain modality data processing exceeds the predicted peak and the task execution progress is lagging behind, computing power is allocated from the idle computing power of low priority tasks first. If the idle computing power is insufficient, the computing power allocation ratio of low priority tasks is dynamically compressed.
[0047] S42. When the actual computing power consumption of a certain modality data processing is lower than the predicted average and there is idle computing power, the idle computing power is allocated to modality tasks with a large computing power demand gap, or the computing power output of the modality processing unit is reduced.
[0048] S43. When the 5G communication rate increases and the amount of multimodal data input increases, the computing power allocation of the corresponding mode processing unit is increased synchronously; when the 5G communication rate decreases and the amount of data input decreases, the computing power allocation of the corresponding mode processing unit is reduced accordingly.
[0049] S44. During cross-modal fusion, the collaborative computing power of each modal processing unit is dynamically allocated according to the complexity of the fusion task.
[0050] refer to Figure 1 In S5, energy consumption optimization also includes: combining sparse matrix optimization algorithms to perform redundant calculations in the processing of low-complexity multimodal data, thereby reducing energy consumption and computing power consumption; the safety threshold is 75℃-80℃, which can be dynamically adjusted according to the chip model.
[0051] In S6, the deep reinforcement learning algorithm adopts the PPO algorithm. Its state space is a set of multimodal data features, computing power consumption, 5G communication rate, and chip power consumption. The action space is the adjustment amount of the computing power allocation ratio. The reward function is a comprehensive evaluation function of computing power utilization, task latency, and energy consumption. In S7, the computing power requirement of vRAN load is prioritized higher than that of non-real-time multimodal AI tasks but lower than that of high-priority multimodal tasks. The response time of computing power cross-load scheduling does not exceed 10ms, ensuring the coordinated stability of 5G communication and multimodal processing.
[0052] refer to Figure 1 The 5G smart chip also integrates a memory scheduling unit. During the initialization allocation process in step S3, the coordinated allocation of memory and computing resources is considered simultaneously. The ratio of memory to computing power is reasonably divided according to the memory requirements of multimodal data to avoid memory bottlenecks. The memory requirements are calculated based on the feature sparsity, processing complexity, and model parameter scale of the multimodal data. This method is applicable to 5G multimodal application scenarios such as edge computing, vehicle networking, industrial internet, smart healthcare, and smart terminals. The computing power output level of the multimodal processing unit can be continuously adjusted from 0.1 TOPS to 10 TOPS according to a dynamic adjustment strategy to adapt to the processing needs of multimodal data with different complexities.
[0053] refer to Figure 1 This invention proposes a closed-loop computing power allocation architecture of multimodal feature perception, demand prediction, dynamic allocation, and feedback iteration. It is the first to combine an improved LSTM-Transformer hybrid model with a cross-modal attention mechanism, which solves the problems of lagging and low accuracy in computing power demand prediction caused by the heterogeneity of multimodal data. At the same time, it combines gradient checkpointing technology to optimize memory usage and adapt to the problem of surge in memory demand caused by fine-tuning of multimodal models. It breaks through the limitation of existing methods that can only passively respond to computing power demand, and realizes accurate prediction and active allocation of computing power demand. This is a core innovation point that has not been addressed in existing technologies.
[0054] refer to Figure 1 This invention achieves deep coordination of multimodal task priority, 5G communication rate, chip power consumption, graphics memory resources and computing power allocation. It designs a dynamic weight allocation algorithm and cross-layer collaborative scheduling logic, which not only considers the heterogeneity of multimodal data, but also realizes the linkage of the 5G communication layer, data preprocessing layer and chip computing layer. At the same time, it solves the problem of the disconnect between computing power and power consumption and graphics memory in existing methods, and realizes a four-dimensional balance of "computing power-power consumption-performance-graphics memory", which is in line with the development trend of integrated computing.
[0055] refer to Figure 1 This invention introduces deep reinforcement learning algorithms and sparse matrix optimization algorithms to construct an adaptive computing power allocation iteration mechanism. It can automatically optimize the computing power allocation strategy according to the dynamic changes of 5G application scenarios, the fluctuation of multimodal data, and the chip operating status, without manual intervention. This overcomes the shortcomings of existing methods that require manual parameter preset and have poor adaptability, and significantly improves the versatility and adaptability of the method. This invention designs a computing power collaborative scheduling logic that integrates 5G communication load and multimodal AI processing load, realizing dynamic computing power sharing. It solves the problem of computing power silos caused by the separation of the two in the existing technology. It can flexibly allocate computing power resources according to the peak and trough changes of 5G communication load and AI load, and greatly improve the computing power utilization rate. This is fundamentally different from the existing computing power allocation methods that focus on multimodal processing or 5G communication.
[0056] refer to Figure 1 This invention addresses the computational power coordination requirements in the multimodal data fusion process by designing a dedicated computational power allocation strategy. Combined with a cross-modal attention mechanism, it ensures the efficiency and accuracy of multimodal data fusion while avoiding the coordination imbalance caused by excessive occupation of single-modal computational power. This breakthrough overcomes the technical bottleneck of existing methods that do not consider the computational power requirements of cross-modal fusion.
[0057] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for dynamic computing power allocation in 5G smart chips supporting multimodal data, characterized in that: Includes the following steps: S1. Multimodal data acquisition and preprocessing: The 5G smart chip receives external multimodal data through the 5G communication unit, and at the same time acquires multimodal data generated locally by the chip. All multimodal data are classified, denoised, and standardized preprocessed. The core features of each modality are extracted, and the processing tasks and task priorities corresponding to each modality are labeled. S2. Multimodal computing power demand prediction: Based on the preprocessed multimodal data characteristics, combined with historical computing power allocation data, 5G communication rate fluctuation data and multimodal task execution logs, the prediction unit uses an improved LSTM-Transformer hybrid model to predict the peak computing power demand, average computing power demand and demand fluctuation range of each data processing task in the future preset time period. At the same time, it predicts the impact of 5G communication rate changes on computing power demand and outputs the computing power demand prediction results. S3. Initial allocation of computing power resources: The computing power scheduling unit performs initial allocation of computing power based on the real-time computing power requirements of each modality, task priorities, and the chip's total computing power limit, using a weighted allocation algorithm. The weights of task priority, data processing complexity, and real-time requirements are dynamically adjusted according to the 5G application scenario. S4. Dynamic computing power adjustment: The computing power scheduling unit monitors the actual computing power consumption of each mode of data processing, task execution progress, 5G communication rate changes and chip power consumption status in real time, and performs dynamic computing power adjustment based on the computing power demand prediction results in step S2. S5. Energy consumption optimization and computing power constraints: The energy consumption monitoring unit collects the chip's energy consumption data and temperature data in real time. When the chip's energy consumption exceeds the preset threshold or the temperature exceeds the safety threshold, the computing power scheduling unit dynamically reduces the computing power output of low-priority tasks without affecting the real-time processing of high-priority tasks, or uses dynamic voltage frequency scaling technology to adjust the computing power output level. S6. Computing power allocation feedback and model iteration: Record the results of each computing power allocation, the execution efficiency of each modal task, latency data, energy consumption data and 5G communication adaptation status to form feedback data; Based on the feedback data, a deep reinforcement learning algorithm is used to iteratively optimize the parameters of the computing power demand prediction model in step S2, the weights of the weighted allocation algorithm in step S3, and the dynamic adjustment strategy in step S4. S7. Integrated Computing and Coordinated Scheduling: When 5G communication load and multimodal AI processing load coexist, the computing power scheduling unit realizes cross-load dynamic scheduling of computing resources based on the real-time load fluctuations of both. When 5G communication is busy, priority is given to ensuring the computing power demand of vRAN load and reducing the computing power of non-real-time multimodal AI tasks. When 5G communication is idle, idle computing power is fed back to multimodal AI processing tasks.
2. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: The multimodal data includes text data, image data, audio data, video data, and sensor data. The core characteristics include data volume, processing complexity, real-time requirements, and data sparsity.
3. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: In S2, the value range of the future preset time period is 10ms-100ms, which is dynamically adjusted according to the real-time requirements of 5G application scenarios.
4. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: In S3, the calculation formula of the weighted allocation algorithm is: computing power allocation ratio = (task priority weight × priority coefficient + processing complexity weight × complexity coefficient + real-time requirement weight × real-time coefficient) / total weight sum; where the priority coefficient, complexity coefficient, and real-time coefficient are all obtained by normalizing the quantized values of the corresponding features, and the value range is 0-1; the total weight sum is the sum of task priority weight, processing complexity weight, and real-time requirement weight, and the value is 1.
5. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: In step S4, the dynamic computing power adjustment specifically includes: S41. When the actual computing power consumption of a certain modality data processing exceeds the predicted peak and the task execution progress is lagging, computing power is allocated from the idle computing power of low priority tasks first. If the idle computing power is insufficient, the computing power allocation ratio of low priority tasks is dynamically compressed. S42. When the actual computing power consumption of a certain modality data processing is lower than the predicted average and there is idle computing power, the idle computing power will be allocated to modality tasks with a large computing power demand gap, or the computing power output of the modality processing unit will be reduced. S43. When the 5G communication rate increases and the amount of multimodal data input increases, the computing power allocation of the corresponding mode processing unit is increased synchronously; when the 5G communication rate decreases and the amount of data input decreases, the computing power allocation of the corresponding mode processing unit is reduced accordingly. S44. During cross-modal fusion, the collaborative computing power of each modal processing unit is dynamically allocated according to the complexity of the fusion task.
6. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: In S5, the energy consumption optimization further includes: combining a sparse matrix optimization algorithm to perform redundant calculations in the processing of low-complexity multimodal data, thereby reducing energy consumption and computing power consumption; the safety threshold is 75℃-80℃, which can be dynamically adjusted according to the chip model.
7. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: In S6, the deep reinforcement learning algorithm adopts the PPO algorithm, whose state space is a set of multimodal data features, computing power consumption, 5G communication rate and chip energy consumption, action space is the adjustment amount of computing power allocation ratio, and reward function is a comprehensive evaluation function of computing power utilization, task latency and energy consumption.
8. The 5G smart chip dynamic computing power allocation method supporting multimodal data according to claim 1, characterized in that: In step S7, the computing power requirement of the vRAN load has a higher priority than that of non-real-time multimodal AI tasks but a lower priority than that of high-priority multimodal tasks; the response time for cross-load scheduling of computing power does not exceed 10ms, ensuring the coordinated stability of 5G communication and multimodal processing.