A low-power evaluation method and system for large model inference performance on mobile devices

Abstract

This application discloses a low-power evaluation method and system for large-model inference performance on mobile devices, relating to the field of performance testing. The method includes: constructing a standardized task slice library for large-model inference on the device side, defining standardized parameters for each task slice, generating a standardized task slice package, loading the slice package onto the mobile device under test for consistency verification and preheating calibration, constructing a temperature-power-performance mapping relationship, and determining a temperature threshold; issuing inference tasks to the mobile device under test according to a hierarchical evaluation system, and monitoring the phone's temperature in real time; dynamically adjusting the inference task issuance density when the temperature is greater than or equal to the temperature threshold; synchronously collecting multi-dimensional indicator data, preprocessing it to generate a standardized dataset; determining the energy efficiency ratio of large-model inference on the device side based on the standardized dataset, and obtaining the evaluation result by combining it with a comprehensive scoring model. This application can achieve a scientific, objective, and comparable comprehensive evaluation of the performance and low power consumption of large-model inference on the device side for different chip architectures and different mobile devices.

Description

Technical Field

[0001] This application relates to the field of performance testing, and in particular to a low-power evaluation method and system for large model inference performance on a mobile device. Background Technology

[0002] With the deployment of large-scale model technology on mobile devices, smartphones have become the core operating platform for these models. Applications such as text-to-image processing, speech recognition, code generation, and natural language understanding are gradually becoming core functions of smartphones. The inference performance of large-scale models directly determines the user experience. As a portable mobile terminal, power consumption and battery life are key factors affecting user experience. Therefore, evaluating the balance between inference performance and power consumption of large-scale models has become a core requirement for smartphone technology evaluation. Meanwhile, the architectures of dedicated Artificial Intelligence (AI) computing units offered by various chip manufacturers differ significantly.

[0003] Current evaluation technologies for mobile AI performance mainly focus on traditional AI computing power benchmarks and cloud-based large model performance assessments. Existing technologies related to edge-side large model inference performance and low-power consumption evaluation mainly include: (1) Traditional AI computing power benchmarking: By running standardized basic AI computing tasks, the theoretical computing power (Tera Operations Per Second, TOPS) of the mobile phone AI chip is tested, with the core focus on the numerical value of computing power.

[0004] (2) Performance evaluation of large cloud models: The inference speed, number of parameters, task completion rate and other indicators of large cloud models are evaluated, with a focus on model processing capabilities, without considering the computing power and power consumption constraints of edge hardware.

[0005] (3) Mobile phone power consumption test: The power consumption of the mobile phone under different usage scenarios (such as video and games) was tested by power meter and other equipment. No special power consumption data was collected for the specific scenario of large model inference on the terminal side.

[0006] (4) Single task inference test: Some manufacturers conduct inference latency tests on single tasks (such as text generation) of their own large on-device models, focusing only on task completion speed and not combining multi-dimensional indicators such as power consumption, memory usage, and computing power utilization.

[0007] Based on the above description, although existing technologies have made some progress in mobile AI computing power and general performance evaluation, there are still many shortcomings in low-power evaluation of large model inference performance on mobile devices, as follows: 1) Evaluation focuses on the cloud and is detached from the actual scenario on the device: Existing large model performance evaluations are mostly aimed at the cloud and do not take into account the hardware limitations of computing power, memory and battery capacity on the mobile device. The evaluation results cannot reflect the user's actual experience of using large models on the device.

[0008] 2) Lack of performance-power balance assessment: Traditional AI evaluation only focuses on performance indicators such as inference speed and computing power, without collecting power consumption data at the same time. It cannot measure the performance-power balance of large models on the edge, nor can it reflect the battery consumption loss during the inference process.

[0009] 3) Single evaluation metric, failing to achieve multi-dimensional comprehensive evaluation: Existing related tests only focus on single metrics (such as latency and computing power), without combining multi-dimensional metrics such as the computing power utilization of the Neural-network Processing Unit (NPU), memory usage, peak power consumption, and continuous inference stability, thus failing to comprehensively evaluate the overall performance of large-scale model inference on the edge.

[0010] 4) Non-standardized task load and incomparable evaluation results: Different evaluation parties use different large-scale model tasks on the device side, and there is no unified task load standard, which makes the evaluation results of different mobile phones and different chip architectures incomparable and unable to form a unified industry standard.

[0011] 5) The impact of thermal throttling on the evaluation results was not considered: The large model inference process on the mobile side is prone to heat generation, which will trigger hardware thermal throttling. Existing evaluations have not been adapted to this problem, resulting in a deviation between the evaluation results and the actual performance in actual use.

[0012] 6) Lack of unified energy efficiency evaluation indicators: The industry has not yet introduced an energy efficiency ratio indicator for large-scale model inference on the edge, making it impossible to make horizontal comparisons of AI computing energy efficiency of different chip architectures, and also unable to provide quantitative basis for the technical optimization of chip manufacturers and mobile phone manufacturers.

[0013] In summary, existing technologies simply cannot meet the actual needs of evaluating the performance and low power consumption of large model inference on mobile devices. They suffer from problems such as evaluation scenarios being detached from reality, single indicators, non-standard loads, failure to consider thermal throttling, and lack of unified energy efficiency indicators. Summary of the Invention

[0014] The purpose of this application is to provide a low-power evaluation method and system for large model inference performance on mobile devices, which can effectively solve the shortcomings of existing technologies and realize a scientific, objective, and comparable comprehensive evaluation of large model inference performance and low power consumption on mobile devices with different chip architectures and mobile phones, providing strong support for the formulation of industry standards and the optimization of chip and mobile phone technologies.

[0015] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a low-power evaluation method for large model inference performance on a mobile device, including: Construct a standardized task slice library for the large-scale edge model, and define the parameters of the task slices in the standardized task slice library for the large-scale edge model to generate a standardized task slice package. The standardized task slice package is loaded into the phone under test and a consistency check is performed. The phone under test was preheated and calibrated to establish a temperature-power consumption-performance mapping relationship and determine the temperature threshold. According to the hierarchical evaluation system, inference tasks are sent to the mobile phone under test based on the standardized task slice package that has passed the consistency verification, and the temperature of the mobile phone under test is monitored in real time during the execution of the inference tasks; when the temperature reaches or exceeds the temperature threshold, the inference task sending density is dynamically adjusted. During the execution of the inference task, multi-dimensional indicator data are collected synchronously and preprocessed to generate a standardized dataset; The inference efficiency ratio of the large edge model is determined based on the standardized dataset, and the evaluation results are obtained by combining the comprehensive scoring model.

[0016] Secondly, this application provides a low-power evaluation system for large model inference performance on a mobile device, comprising: The task loading module is used to load the standardized task slice package into the mobile phone under test and perform consistency verification; wherein, the construction process of the standardized task slice package includes: constructing a standardized task slice library of the terminal large model, defining the parameters of the task slices in the terminal large model standardized task slice library, and generating the standardized task slice package. An adaptive control module is used to distribute inference tasks to the mobile phone under test according to the hierarchical evaluation system based on the standardized task slice package that has passed the consistency verification, and to monitor the temperature of the mobile phone under test in real time during the execution of the inference task; when the temperature reaches or exceeds the temperature threshold, the distribution density of the inference task is dynamically adjusted. The data acquisition module is used to synchronously collect multi-dimensional indicator data during the execution of inference tasks and preprocess it to generate standardized datasets. The energy efficiency calculation module is used to determine the end-side large model inference energy efficiency ratio based on the standardized dataset; The results output module is used to obtain the evaluation results based on the energy efficiency ratio and the comprehensive scoring model of the end-side large model.

[0017] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a low-power evaluation method and system for large-scale model inference performance on mobile devices. By constructing a standardized task slice library for large-scale models on the device, it achieves uniformity in evaluation tasks, making the evaluation results of different chip architectures and mobile phones comparable, laying the foundation for the formulation of industry evaluation standards. By monitoring the temperature of the mobile phone under test in real time during the execution of the inference task, and dynamically adjusting the inference task distribution density when the temperature reaches or exceeds the temperature threshold, it can effectively avoid the impact of thermal throttling on the evaluation results, ensuring the authenticity and reliability of the evaluation results. By introducing energy efficiency ratio as a core indicator, it realizes the quantification of the performance-power consumption balance capability of large-scale models on the device, providing a scientific quantitative basis for chip manufacturers and mobile phone manufacturers to optimize hardware and schedule software for large-scale models on the device. Based on this, this application can achieve a scientific, objective, and comparable comprehensive evaluation of the inference performance and low power consumption of large-scale models on the device for different chip architectures and mobile phones. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a low-power evaluation method for large model inference performance on a mobile device side, provided as an embodiment of this application; Figure 2 A flowchart illustrating the overall evaluation process of a low-power evaluation method for large model inference performance on a mobile device, as provided in an embodiment of this application. Figure 3 This is a schematic diagram of an evaluation report generation interface provided in an embodiment of this application; Figure 4 A functional module diagram of a low-power evaluation system for large model inference performance on a mobile device side, provided in an embodiment of this application; Figure 5 A flowchart of the task loading module provided in one embodiment of this application; Figure 6 A schematic diagram of a data acquisition module provided in one embodiment of this application; Figure 7 A flowchart of the adaptive control module provided in one embodiment of this application; Figure 8 A schematic diagram of an energy efficiency calculation module provided in an embodiment of this application; Figure 9 This is a schematic diagram of a comprehensive scoring model for a result output module provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Definitions: (1) Large-scale terminal model: Large language model / multimodal model deployed on terminal hardware such as smartphones, tablets, and IoT devices, with local reasoning capabilities, without relying on cloud servers; (2) Inference latency: The time from issuing the edge-side large model inference task to generating the first result (first token) / completing all results is the core indicator for measuring the inference speed of the edge-side large model; (3) First token latency: The time from task issuance to the generation of the first result token by the large model on the client side is a key indicator affecting the real-time user experience; (4) Energy efficiency ratio (TOPS / W): The ratio of the actual output computing power (TOPS) of the NPU to the average power consumption (W) during the large model inference process on the edge is the core indicator for measuring the performance-power balance capability. (5) Task Slicing: The complete reasoning task of the large edge model is divided into multiple standardized basic reasoning units, each of which is a task slice, which facilitates standardized evaluation. (6) Thermal throttling: When mobile phone hardware (such as NPU / CPU) heats up during operation, it automatically reduces the operating frequency to protect the hardware, which will lead to a decrease in inference performance; (7) Dynamic power consumption threshold: The upper limit of power consumption is determined based on the critical temperature of thermal throttling of the mobile phone, so that the mobile phone will not trigger thermal throttling when running within this power consumption range.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In one exemplary embodiment, this application provides a low-power evaluation method for large model inference performance on a mobile device. This method is executed by a computer device, specifically by a terminal or server alone, or by both. In this embodiment, the method is described using a server as an example. Figure 1 As shown, the method includes: Step 100: Construct a standardized task slice library for the large-scale edge model, and define the parameters of the task slices in the standardized task slice library for the large-scale edge model to generate a standardized task slice package.

[0024] Step 101: Load the standardized task slice package into the phone under test and perform a consistency check.

[0025] Step 102: Perform preheating calibration on the phone under test, establish a temperature-power consumption-performance mapping relationship, and determine the temperature threshold.

[0026] Step 103: Following the tiered evaluation system, inference tasks are distributed to the phone under test based on standardized task slices that have passed consistency verification. The temperature of the phone under test is monitored in real time during the execution of the inference tasks. When the temperature reaches or exceeds a temperature threshold, the distribution density of inference tasks is dynamically adjusted. This dynamic adjustment of task density only changes the task distribution interval, without modifying the tasks themselves, thus ensuring the standardization of the evaluation workload.

[0027] Step 104: During the execution of the inference task, multi-dimensional indicator data are collected synchronously and preprocessed to generate a standardized dataset.

[0028] Step 105: Determine the inference efficiency ratio of the large edge model based on the standardized dataset, and obtain the evaluation results by combining the comprehensive scoring model.

[0029] By implementing steps 100-105 above, this application has achieved a low-power special evaluation of the performance of large model inference on the mobile device side, filling an industry gap. The evaluation scenario is in line with the actual user needs and solves the problem that existing technologies focus on the cloud and are detached from the actual device side.

[0030] In one exemplary embodiment of this application, in order to cover typical edge-side large model application scenarios, the edge-side large model standardized task slice library adopted in this application covers at least four major scenarios: text, vision, speech, and code. Based on this, the edge-side large model standardized task slice library mainly includes the following slice types: (1) Text slicing: text generation (fixed input character count 20, output character count 100), natural language understanding (semantic classification, question answering matching).

[0031] (2) Visual slices: text-generated image (with fixed text prompts, generating a 512×512 resolution image), image description (generating descriptive text for standard test images).

[0032] (3) Speech segmentation: speech recognition (fixed 10-second standard speech segment, converted to text), speech synthesis (fixed text, synthesized speech).

[0033] (4) Code segmentation: code generation (generating Python code snippets based on fixed requirements descriptions) and code explanation (providing functional descriptions for simple code).

[0034] In practical applications, step 100 above involves constructing a standardized task slice library for large-scale edge models to cover typical application scenarios. Standardizing the parameters of the task slices in this library is to fix the input size, model accuracy, and execution strategy. Specifically, fixing the core parameters for each task slice ensures that different mobile phones execute the exact same workload. 1) Input scale: fixed number of text characters, image resolution, voice duration, and code requirement description length.

[0035] 2) Model precision: The model is uniformly set to the mainstream precision on the edge (INT4 / INT8 / FP16 (4-bit integer / 8-bit integer / 16-bit floating point, 4-bit Integer / 8-bit Integer / 16-bit Floating Point), supporting multi-precision switching for evaluation).

[0036] 3) Execution strategy: fixed number of NPU scheduling threads, memory allocation ratio, and number of background processes.

[0037] Based on the above description, the construction of a standardized task slice library can achieve the unification and reproducibility of task loads, providing a basis for horizontal comparison, while covering the mainstream application scenarios of large edge models and ensuring the comprehensiveness of the evaluation.

[0038] In an exemplary embodiment of this application, in order to avoid the distortion of evaluation results due to task tampering, the implementation process of step 101 above can be described as follows: the hash value of the standardized task slice package is determined on the server side and the mobile phone under test respectively, and compared. If the hash values ​​are consistent, the verification is passed. If the hash values ​​are inconsistent, the evaluation is stopped or the standardized task slice package is automatically reloaded for secondary verification.

[0039] In practical applications, after the standardized task slice package is loaded onto the phone under test, the verification process begins: First, the hash value (e.g., SHA-256) of the task slice package is calculated on both the server and the phone under test. Then, the hash value calculated on the phone under test is compared with the standard hash value on the server. If they match perfectly, the verification passes, and the evaluation continues. If they do not match, the verification fails, the evaluation process is immediately terminated, and a clear message "Task package verification failed" is displayed on the interface, along with a failure log. Subsequently, the system automatically attempts to reload the standardized task slice package and perform a second verification. If the second verification fails again, the system is deemed "unable to evaluate," an error report is generated, and the operator is prompted to check the phone's storage, network, or the standardized task slice package itself.

[0040] Based on the above description, steps 100 and 101 can standardize and reproducibly implement the evaluation task of large-scale edge models, laying the foundation for horizontal comparisons across different mobile phones and chip architectures. The task slices cover mainstream application scenarios of large-scale edge models, and the evaluation results are more closely aligned with actual user needs. It supports switching between multiple model accuracies for evaluation and can adapt to the deployment requirements of different large-scale edge models.

[0041] In an exemplary embodiment of this application, in order to effectively avoid the impact of mobile phone thermal throttling on the evaluation results and ensure that the evaluation results are consistent with the actual performance in actual use by users, so as to adapt to the heat dissipation and power consumption characteristics of different mobile phones, avoid the impact of thermal throttling, and ensure the authenticity of the evaluation results, the implementation process of the above step 102 includes: Step 102-1: Run a lightweight text-based task slice to periodically collect the temperature, NPU operating frequency, and real-time power consumption of the back panel area of ​​the phone under test, and construct a temperature-power consumption-performance mapping relationship. Based on this mapping relationship, determine the dynamic power consumption threshold of the phone under test in the non-thermal throttling range. The dynamic power consumption threshold is adaptively determined based on the hardware characteristics of the phone under test, adapting to the heat dissipation and power consumption characteristics of different brands and models of phones.

[0042] Step 102-2: Based on the temperature-power consumption-performance mapping relationship, use the temperature below the critical temperature setting value that triggers thermal throttling of the mobile phone as the temperature threshold.

[0043] Based on the above description, in practical applications, the implementation process of steps 102 and 103 can be described as follows: (1) Preheating calibration: Select a lightweight text-based task slice, run for 3 minutes, and collect the temperature of the core area of ​​the phone's back panel (every 500ms), NPU frequency, and real-time power consumption to construct the temperature ( T -NPU frequency ( F - Power consumption P A three-dimensional mapping table (i.e., temperature-power consumption-performance mapping relationship).

[0044] (2) Evaluation process for other slice types: The purpose of preheating calibration is to establish a temperature-power-performance baseline. Lightweight tasks are used to avoid triggering high temperatures during the initial calibration. After calibration is completed and the dynamic power consumption threshold is determined, all task slices, including text, vision, speech, and code, will be run during the formal tiered evaluation execution phase. The evaluation process distributes tasks according to the preset order in the tiered evaluation system. The basic inference layer in the tiered evaluation system will execute each task slice independently, the continuous load layer in the tiered evaluation system will execute all task slices continuously in a random order, and the mixed scenario layer in the tiered evaluation system will execute various slices simultaneously under background simulation tasks. Throughout the entire tiered evaluation process, this adaptive control method will continue to play a role, dynamically adjusting the task distribution density by monitoring the temperature to ensure that the frequency is not reduced due to heat during the entire evaluation process. Therefore, lightweight tasks are used for calibration, while all slice types will be used in the formal evaluation.

[0045] (3) Determination of dynamic power consumption threshold: The critical temperature at which the mobile phone triggers thermal throttling (usually 43℃) is taken as the temperature threshold, and the average power consumption corresponding to 2℃ before the critical temperature is taken as the dynamic power consumption threshold P0.

[0046] (4) Real-time control: The temperature is monitored in real time by temperature sensor. When the temperature is greater than or equal to 41℃ (the critical temperature for the mobile phone to trigger thermal throttling is usually 43℃), the task density adjustment is started, and the interval between continuous task delivery is adjusted from 0s to 500ms~2s (adaptive to temperature changes) until the temperature drops below 41℃.

[0047] In an exemplary embodiment of this application, to achieve synchronous and high-precision collection of multi-dimensional indicators, comprehensively covering the performance, computing power, memory, and power consumption dimensions of large-scale model inference on the edge, and providing complete data support for comprehensive evaluation, the processing flow of step 104 above can be described as follows: the phone under test runs an inference task → multi-dimensional data collection (latency collection, computing power collection, memory collection, power consumption collection) → data preprocessing (denoising, normalization, timestamp alignment) → standardized dataset storage. Among these, data preprocessing ensures the accuracy and consistency of the dataset, avoiding deviations in evaluation results due to abnormal raw data. Based on this, the data collected in this step includes: (1) Inference latency metrics: first token latency (time from task issuance to generation of the first result, in ms), average inference speed (in tokens / s, for image classes it is pixel generation speed px / s), and total task time (in s).

[0048] (2) NPU computing power indicators: NPU computing power utilization rate (%), NPU operating frequency (MHz), and NPU actual output computing power (TOPs).

[0049] (3) Memory usage indicators: peak / average physical memory usage (GB), peak / average video memory usage (GB), and Swap memory usage rate (%).

[0050] (4) Power consumption indicators: real-time power consumption of the whole machine (W, collected by an external power meter), power consumption of AI subsystem (W, collected by the underlying power consumption interface of the mobile phone), peak power consumption (W), and average power consumption (W).

[0051] Based on the above description, by collecting the power consumption of the entire machine and the specific power consumption of the AI ​​subsystem, the actual power consumption loss of the large model inference on the edge can be accurately located.

[0052] In an exemplary embodiment of this application, in order to quantify the performance-power balance capability, this application introduces the energy efficiency ratio (unit: TOPS / W) into the evaluation of large-scale model inference on the mobile device side for the first time. Based on this, the implementation process of determining the energy efficiency ratio of the large-scale model inference on the mobile device side based on a standardized dataset in step 105 of this application can be described as: standardized dataset → actual computing power calculation → average power consumption extraction → energy efficiency ratio calculation → energy efficiency level classification. Wherein: Step 1: Extract Actual Output Computing Power. Extract the actual NPU output computing power for each stage of the tiered evaluation (i.e., basic inference layer, continuous load layer, and hybrid scenario layer) from the standardized dataset of the data acquisition module. Take the average computing power of each stage as the actual computing power. T (Unit: TOPS).

[0053] Step 2: Extract Average Power Consumption. Simultaneously extract the average power consumption specific to each AI subsystem stage. If the phone lacks a power consumption interface for the underlying AI subsystem, extract the overall average power consumption and subtract the basic standby power consumption to obtain the average power consumption for large-scale model inference on the device side. P (Unit: W).

[0054] Step 3: Energy Efficiency Ratio Calculation. The end-side large-scale model inference energy efficiency ratio (hereinafter referred to as energy efficiency ratio) is calculated using the following formula: .

[0055] In the formula, The inference efficiency ratio of the large-scale end-side model is expressed in TOPS / W.

[0056] Step 4: Energy Efficiency Rating. Inference latency, NPU computing power utilization, peak memory usage, average power consumption, and energy efficiency ratio are selected as core scoring indicators. Based on experimental data and industry expert opinions, the weighting coefficients and scoring standards for each indicator are determined. According to the energy efficiency ratio, five energy efficiency levels are defined (Level 1 is the most efficient, and Level 5 is the least efficient), providing an intuitive reference for industry applications.

[0057] Step 5: The Relationship Between Energy Efficiency Levels and Overall Scores. In the final evaluation report generated based on the test results, energy efficiency levels (levels 1-5) and overall scores (0-100 points) are two complementary evaluation methods. Energy efficiency levels (based on energy efficiency ratio) focus on intuitively reflecting the balance between performance and power consumption, providing users with a quick and tiered assessment of energy consumption. The overall score, in addition to energy efficiency evaluation, incorporates comprehensive quantitative results such as latency, computing power utilization, and memory usage, comprehensively reflecting inference performance. In the final evaluation report, energy efficiency levels (e.g., level 2) and overall scores (e.g., 85 points) will be presented simultaneously, providing multi-faceted references for users with different needs.

[0058] Based on the above description, this application can distinguish between the specific power consumption of the AI ​​subsystem and the overall power consumption, making the calculation results more accurately reflect the actual energy efficiency of edge-side large model inference. The energy efficiency level classification makes the evaluation results more intuitive, making it easier for users and manufacturers to quickly judge the energy efficiency performance of edge-side large model inference.

[0059] In an exemplary embodiment of this application, in order to achieve a quantitative comprehensive evaluation of the inference performance and low power consumption of large-scale edge models, and to make the scoring results more scientific and objective, in this embodiment, the process of obtaining the evaluation results and generating the evaluation report by combining the comprehensive scoring model in step 105 above can be described as follows: multi-dimensional index data + energy efficiency ratio data → comprehensive scoring model (indicator weighting, score calculation) → evaluation report generation (single index analysis, comprehensive analysis, horizontal comparison) → multi-format report output (PDF / Excel / visual charts). Wherein: Step (1), Construction of the comprehensive scoring model: Inference latency, NPU computing power utilization, peak memory usage, average power consumption, and energy efficiency ratio are selected as the core scoring indicators. The raw data for these indicators all come from the standardized datasets collected above and processed by energy efficiency calculation. They are the quantitative output of the hierarchical evaluation process, and the core formula is: .

[0060] In the formula, The overall score is 100 points. Scoring for inference latency (weight) =0.25), NPU computing power utilization score (weight) =0.20), Memory peak usage score (weight) =0.15), Average power consumption score (weight) =0.20), Energy efficiency ratio score (weight) =0.20).

[0061] Each individual indicator is scored from 0 to 100 points, and quantitative standards are established based on the principle that the better the performance, the higher the score (e.g., the shorter the inference latency). The higher the efficiency ratio, the better. The higher (the higher).

[0062] Step (2), the connection between scoring and process: The core function of the comprehensive scoring model is to integrate a series of raw data generated by the aforementioned evaluation process (task execution, data collection, energy efficiency calculation) into an intuitive and horizontally comparable score through weighted quantification, thereby giving an overall evaluation of the comprehensive performance and power consumption of the end-side large model inference of the evaluated object.

[0063] Step (3), as follows Figure 3 As shown, the automatically generated evaluation report is a standardized evaluation report that includes single-index analysis, comprehensive analysis, energy efficiency rating, and horizontal comparison. Specific content includes: 1) Single index curves: time variation curves of inference latency, real-time power consumption, and NPU utilization.

[0064] 2) Core metrics: Key metrics such as first token latency, average inference speed, and energy efficiency ratio at each tier evaluation stage.

[0065] 3) Overall score and energy efficiency rating: Overall score result, score of each individual indicator, and energy efficiency rating.

[0066] 4) Horizontal comparison: If you import the evaluation data of multiple mobile phones, you can generate comparison tables and visualization charts of the core indicators and comprehensive scores of multiple models.

[0067] Based on the above description, the report generation method combines template filling and data-driven plotting. It pre-loads report templates in various formats and supports output in PDF (formal reports), Excel (data details), visualization charts (line charts, bar charts), JPG, and PNG formats to meet diverse usage scenarios. Specifically: PDF Format (Official Report): This format stores a structured PDF template file. After the evaluation is completed, the final indicator data, comprehensive score, and energy efficiency level are extracted from the database and populated into the corresponding placeholders in the PDF template. Simultaneously, a graphing engine is invoked to generate the necessary visualization charts (such as line charts and bar charts) based on the data, and these charts are embedded in designated locations within the PDF, ultimately generating a complete, printable, official evaluation report.

[0068] Excel format (data details): This also includes a predefined Excel template file containing multiple worksheets (such as "Raw Data," "Core Indicators," "Comprehensive Score," etc.). The raw time-series data, processed data, and final calculation results from the evaluation process are automatically written into the corresponding cells and worksheets according to the headers and formats defined in the Excel template. The generated data tables can be used by technical personnel for in-depth analysis and secondary processing.

[0069] Visualization Charts: Chart generation is data-driven and document-format independent. Processed data (such as time-series power consumption and latency data) is passed to built-in chart generation libraries (such as Matplotlib or the ECharts engine), which generate image files (such as PNG and SVG formats) according to predefined types (line charts, bar charts, pie charts) and styles. These images can be embedded in the aforementioned PDF reports or exported separately for presentation or display purposes.

[0070] Based on the above description, the standardized evaluation report in this application includes multi-dimensional analysis, providing accurate data support for chip manufacturers and mobile phone manufacturers to optimize their technologies. It supports horizontal comparisons of multiple models, offering intuitive references for industry evaluations and user purchasing decisions. Multiple report formats are available to adapt to the needs of different usage scenarios.

[0071] In summary, the implementation process of the low-power evaluation method for large model inference performance on the mobile device side provided in this application can be as follows: Figure 2 As shown, the overall evaluation process is as follows: evaluation environment initialization → standardized task slice loading → preheating calibration and power consumption threshold determination → layered inference task execution → multi-dimensional data synchronous collection → energy efficiency ratio calculation → comprehensive scoring → evaluation report generation. By constructing a standardized task slice library for large-scale edge models, a layered evaluation system is designed for basic inference, continuous load, and mixed scenarios. While running evaluation tasks locally on the mobile phone, multi-dimensional indicators such as inference latency, NPU computing power utilization, memory usage, and real-time power consumption are collected simultaneously. Energy efficiency ratio is introduced as a core evaluation indicator, and a dynamic power consumption threshold adaptive adjustment mechanism avoids the impact of thermal throttling on the evaluation results. Simultaneously, a comprehensive scoring model is constructed to quantitatively score the performance, power consumption, and energy efficiency of large-scale edge model inference. Through standardized task load, multi-dimensional indicator collection, energy efficiency ratio quantification, and dynamic power consumption adaptive adjustment, the shortcomings of existing technologies can be effectively addressed. This enables scientific, objective, and comparable comprehensive evaluation of the performance and low power consumption of large-scale edge model inference across different chip architectures and mobile phones, truly reflecting the actual user experience and providing a scientific basis for chip manufacturers, mobile phone manufacturers to optimize their technologies, and for the formulation of industry evaluation standards. Based on this, compared with the prior art, this application has at least the following advantages: 1. This application is the first to achieve low-power evaluation of large model inference performance on mobile devices, filling an industry gap. The evaluation scenario is close to the actual user needs and solves the problem that existing technologies focus on the cloud and are detached from the actual device side.

[0072] 2. By standardizing the task slice library, the evaluation tasks of large-scale models on the device side have been unified, making the evaluation results of different chip architectures and different mobile phones comparable, laying the foundation for the formulation of industry evaluation standards.

[0073] 3. A multi-dimensional evaluation system for inference performance, computing power utilization, memory usage, power consumption, and energy efficiency has been constructed, which solves the problem of single technical indicators in the existing system and comprehensively reflects the overall performance of large-scale model inference on the edge.

[0074] 4. The dynamic power consumption threshold adaptive adjustment mechanism effectively avoids the impact of thermal throttling on the evaluation results, ensuring that the evaluation results are true, reliable, and consistent with the actual user experience.

[0075] 5. By introducing energy efficiency ratio as a core indicator, the "performance-power consumption" balance capability of the large-scale edge model is quantified, providing a scientific quantitative basis for chip manufacturers and mobile phone manufacturers to optimize hardware and schedule software for the large-scale edge model.

[0076] 6. The designed automated evaluation process realizes unmanned operation of the entire evaluation process, improves evaluation efficiency, reduces labor costs, and ensures the consistency of evaluation results.

[0077] Based on the same inventive concept, this application also provides a low-power evaluation system for mobile-side large model inference performance, used to implement the aforementioned low-power evaluation method for mobile-side large model inference performance. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the low-power evaluation system for mobile-side large model inference performance provided below can be found in the limitations of the low-power evaluation method for mobile-side large model inference performance described above, and will not be repeated here.

[0078] In one exemplary embodiment, to scientifically, objectively, and comprehensively evaluate the performance and low-power consumption of large model inference on the mobile device side, such as Figure 4 As shown, this application proposes a systematic, verifiable, and automated low-power evaluation system for large-model inference performance on mobile devices, comprising five core modules: a task loading module, an adaptive control module, a data acquisition module, an energy efficiency calculation module, and a result output module. Among them: (1) Task loading module.

[0079] The task loading module loads standardized task slice packages onto the phone under test and performs consistency verification. The construction process of the standardized task slice package includes: building a standardized task slice library for the large-scale client-side model, defining standardized parameters for the task slices in the library, and generating a standardized task slice package. For example... Figure 5 As shown, the task loading module workflow includes: Step 1: Build a standardized task slice library for large edge models, covering typical application scenarios of large edge models.

[0080] Step 2: Parameter Standardization. Define the parameters for the task slices to standardize them, fixing the input size, model accuracy, and execution strategy.

[0081] Step 3: Task package loading and consistency verification. Load the standardized task slice package into the phone under test and complete the task consistency verification.

[0082] Step 4: Layered Task Distribution. Based on the layered evaluation system, inference task instructions are distributed to the test mobile phones in a preset order.

[0083] (2) Adaptive control module.

[0084] The adaptive control module is used to distribute inference tasks to the phone under test according to a hierarchical evaluation system, based on standardized task slices that have passed consistency verification, and to monitor the phone's temperature in real time during the execution of the inference tasks. When the temperature reaches or exceeds a temperature threshold, the module dynamically adjusts the density of inference task distribution. For example... Figure 7 As shown, the implementation process of this module includes: Step 1: Warm-up calibration and construction of a temperature-power consumption-performance mapping table. Before evaluation, the phone under test is warmed up and calibrated, and a short-term lightweight inference task is run to collect the temperature-power consumption-performance mapping relationship.

[0085] Step 2: Determine the dynamic power consumption threshold. Based on the mapping relationship, determine the dynamic power consumption threshold of the phone under test in the non-thermal throttling range.

[0086] Step 3: Real-time temperature monitoring. During the tiered evaluation process, the core temperature of the phone and the operating frequency of the NPU are monitored in real time.

[0087] Step 4: Determine if the temperature is close to the throttling threshold. If the temperature is close to the throttling threshold (i.e., the temperature threshold), dynamically adjust the task density (reduce the continuous task delivery speed) to keep the power consumption within the preset threshold.

[0088] Step 5: If the temperature returns to normal, restore the original task density to ensure that the entire evaluation is conducted without frequency reduction, and then end the adjustment.

[0089] (3) Data acquisition module.

[0090] The data acquisition module is used to synchronously collect multi-dimensional indicator data during the execution of inference tasks and preprocess it to generate a standardized dataset. For example, Figure 6 As shown, the implementation process of the data acquisition module can be described as follows: the mobile phone under test runs the inference task → multi-dimensional data acquisition unit (latency acquisition unit, computing power acquisition unit, memory acquisition unit, power consumption acquisition unit) → data preprocessing unit (denoising, normalization, timestamp alignment) → standardized dataset.

[0091] In practical applications, the data acquisition module can achieve millisecond-level multi-dimensional synchronous data acquisition through the mobile phone's underlying interface (API (Application Programming Interface) / SDK (Software Development Kit)) and external professional testing equipment.

[0092] (4) Energy efficiency calculation module.

[0093] The energy efficiency calculation module is used to determine the energy efficiency ratio of large-scale edge model inference based on a standardized dataset. For example, Figure 8 As shown, the implementation process of this module can be described as follows: standardized dataset → actual computing power extraction unit → average power consumption extraction unit → energy efficiency ratio calculation unit → energy efficiency level classification unit.

[0094] (5) Result output module.

[0095] The results output module is used to derive evaluation results based on the energy efficiency ratio and comprehensive scoring model derived from the large-scale edge model. The implementation flow of this module can be described as follows: multi-dimensional indicator data + energy efficiency ratio data → comprehensive scoring model unit (indicator weighting, score calculation) → evaluation report generation unit (single indicator analysis, comprehensive analysis, horizontal comparison) → multi-format report output (PDF / Excel / visual charts). In practical applications, the implementation process of this module is as follows: Figure 9 As shown.

[0096] Based on the above description, the system provided in this application achieves unified and reproducible task load by constructing a standardized task slice library, providing a foundation for horizontal comparison. It also covers mainstream application scenarios of large-scale edge models, ensuring comprehensive evaluation. Through multi-dimensional synchronous data collection, it comprehensively collects inference, computing power, memory, and power consumption indicators, providing complete data support for comprehensive evaluation. Millisecond-level collection accuracy ensures data accuracy. Dynamic adaptive adjustment of power consumption thresholds adapts to the heat dissipation and power consumption characteristics of different mobile phones, avoiding the impact of thermal throttling and ensuring the authenticity of evaluation results. By setting an energy efficiency ratio calculation module, it quantifies the energy efficiency performance of large-scale edge model inference, achieving a balanced evaluation of performance and power consumption, providing a quantitative basis for technology optimization. By adopting a comprehensive scoring model, multi-dimensional indicators are transformed into intuitive comprehensive scores, facilitating rapid judgment of the overall performance of large-scale edge model inference. By employing an automated evaluation process, it achieves unmanned operation throughout the entire evaluation process, improving evaluation efficiency and reducing human error.

[0097] In an exemplary embodiment, based on the system architecture provided above, the basic implementation of the low-power evaluation method for large model inference performance on the mobile device side provided in this application can be described as follows: (1) Evaluation environment setup: Set up a standardized laboratory evaluation environment with temperature controlled at 25±2℃ and humidity at 50±10%. Clear background processes and turn off automatic brightness / fast charging / Bluetooth / WiFi on the mobile phone under test (only keep the basic system running). Connect an external power meter (accuracy 0.01W) and a data acquisition terminal, and connect to the underlying interface of the mobile phone through a dedicated SDK.

[0098] (2) Evaluation system deployment: The evaluation system of this application is deployed on the data acquisition terminal, which includes five modules: task loading, data acquisition, adaptive control, energy efficiency calculation and result output.

[0099] (3) Standardized task slice loading: Select all four types of task slices from the task slice library, load them onto the mobile phone under test according to the standardized parameters, and complete the consistency verification.

[0100] (4) Preheating calibration and power consumption threshold determination: Run a lightweight text task to perform preheating calibration, construct a temperature-power consumption-performance mapping table, and determine the dynamic power consumption threshold P0.

[0101] (5) Layered evaluation execution: Inference tasks are issued in the order of basic inference layer → continuous load layer → mixed scenario layer. The adaptive control module monitors the temperature in real time and adjusts the task density, and the data acquisition module collects multi-dimensional indicators synchronously.

[0102] 1) Basic inference layer: Single task slices are executed independently, each slice is executed 3 times, and the average value is taken.

[0103] 2) Continuous load layer: 4 types of task slices are executed continuously in random order, with a total duration of 10 minutes.

[0104] 3) Hybrid Scenario Layer: Simulates real user usage, with music playback (standard audio) running in the background and continuous execution of large-scale model inference tasks on the foreground, for a total duration of 10 minutes.

[0105] (6) Data processing and energy efficiency calculation: preprocess the collected raw data to generate a standardized dataset and calculate the energy efficiency ratio (TOPS / W) of each layer stage and the whole.

[0106] (7) Comprehensive scoring and report generation: Calculate the comprehensive score through the comprehensive scoring model, determine the energy efficiency level, and automatically generate a multi-format standardized evaluation report.

[0107] In one exemplary embodiment, the basic implementation method provided above can be flexibly modified according to actual evaluation needs, while the core technical features remain unchanged. The main directions of modification include: (1) Task slice customization: Add or delete task slice types (such as adding multimodal large model slices: image and text hybrid generation) and adjust task slice parameters (such as modifying image resolution and text input / output length) according to evaluation requirements.

[0108] (2) Flexible adjustment of layered evaluation: The number of times and total duration of each layer of evaluation can be adjusted according to the evaluation duration requirements, or only a single layer can be executed (such as only executing the basic inference layer to quickly evaluate the core performance).

[0109] (3) Expansion of data collection indicators: Data collection indicators (such as NPU heating temperature, battery charging and discharging efficiency, and background process interference rate) can be added as needed.

[0110] (4) Weight coefficient adjustment: The weight coefficients of each indicator in the comprehensive scoring model can be dynamically adjusted according to industry needs and technological development (e.g., as the large-scale terminal model becomes lighter, the weight of the memory usage indicator can be appropriately reduced).

[0111] (5) Evaluation scenario expansion: The evaluation can be expanded from static laboratory evaluation to outdoor dynamic evaluation (such as different ambient temperatures: -10℃ / 35℃) to simulate the actual usage scenarios of users in different regions.

[0112] (6) Enhanced automation: Mechanical devices for automatic feeding, docking, and evaluation of mobile phones can be added to achieve batch automated evaluation of multiple mobile phones.

[0113] Based on the above description, the core technical solutions claimed in this application mainly include six core technical points, as well as combinations of these technical points, as follows: (I) Six core technologies.

[0114] 1. Method for constructing and loading standardized task slices of large-scale edge models.

[0115] We will build a large-scale model task slice library covering four major categories: text, vision, speech, and code, and fix the parameters such as input size, model accuracy, and execution strategy for each slice.

[0116] The consistency verification method for task slice packages, and the task distribution method based on a hierarchical evaluation system.

[0117] 2. A method for synchronously collecting multi-dimensional indicators for large-scale model inference on the mobile device side.

[0118] Methods for collecting and calculating inference latency metrics (first token latency, average inference speed).

[0119] Methods for collecting NPU computing power utilization and actual output computing power.

[0120] Methods for collecting peak / average memory usage.

[0121] A method for synchronously collecting power consumption of the whole machine and power consumption of the AI ​​subsystem (bottom interface + external power meter).

[0122] Methods for denoising, normalizing, and timestamp alignment preprocessing of collected data.

[0123] 3. A dynamic power consumption threshold adaptive adjustment method based on thermal throttling avoidance.

[0124] A preheating calibration method was developed before evaluation, and a three-dimensional mapping table of temperature, power consumption, and performance was constructed.

[0125] A method for determining the dynamic power consumption threshold in the non-thermal throttling range.

[0126] A method for dynamically adjusting task density based on temperature monitoring during the evaluation process.

[0127] 4. Calculation method for the energy efficiency ratio of large model inference on mobile devices.

[0128] A method for extracting and averaging the actual computing power of large-scale edge-side inference models.

[0129] The calculation method for the average power consumption of the AI ​​subsystem (total power consumption minus basic standby power consumption).

[0130] The core formula for calculating energy efficiency ratio (energy efficiency ratio = actual computing power / average power consumption).

[0131] A method for classifying energy efficiency levels based on energy efficiency ratio.

[0132] 5. A comprehensive scoring method for low power consumption and large model inference performance on mobile devices.

[0133] Selection of core scoring metrics (inference latency, NPU computing power utilization, peak memory usage, average power consumption, energy efficiency ratio).

[0134] Methods for determining the weight coefficients of each indicator.

[0135] The core formula for comprehensive scoring and the quantitative scoring standards for each individual indicator.

[0136] 6. A low-power evaluation system for large model inference performance on mobile devices that implements the above methods.

[0137] The functions, interfaces, and collaborative working methods of each module.

[0138] The system's automated evaluation process and multi-format report output methods.

[0139] (ii) Combination scheme of each technical point.

[0140] The core technologies in this application can be combined to form different technical solutions, all of which are innovative and practical. The main combination solutions include: Option 1: Standardized task slice construction and loading method + multi-dimensional indicator synchronous collection method to achieve standardized and multi-dimensional evaluation of the inference performance of large models on the edge.

[0141] Combination Scheme 2: Multi-dimensional indicator synchronous acquisition method + dynamic power consumption threshold adaptive adjustment method, to achieve multi-dimensional data acquisition for end-side large model inference without thermal throttling interference.

[0142] Combination Scheme 3: Multi-dimensional indicator synchronous collection method + energy efficiency ratio calculation method to achieve quantitative evaluation of "performance-power consumption-energy efficiency" for edge-side large model inference.

[0143] Combination Scheme 4: Standardized task slice construction and loading method + dynamic power consumption threshold adaptive adjustment method + energy efficiency ratio calculation method, to achieve standardized, interference-free, and highly energy-efficient quantification of low-power evaluation for edge-side large model inference.

[0144] Combination Scheme 5: Multi-dimensional indicator synchronous collection method + energy efficiency ratio calculation method + comprehensive scoring method, to achieve multi-dimensional, energy efficiency quantification and comprehensive scoring integrated evaluation of the low power consumption of large-scale model inference performance on the edge.

[0145] Combination Scheme Six: The combination of all core technologies, namely the complete technical solution of this application, realizes the standardized, multi-dimensional, interference-free, energy efficiency quantification, comprehensive scoring, and automated full-process evaluation of large model inference performance and low power consumption on the mobile terminal side.

[0146] Furthermore, the core innovations of the mobile-side large model inference performance low-power evaluation method and system proposed in this application are standardized task slicing, multi-dimensional synchronous acquisition, dynamic power consumption adaptation, energy efficiency ratio quantification, and comprehensive scoring. Based on this core design idea, the following possible design changes and variations are proposed, while the core technical features remain unchanged.

[0147] 1. Overall Alternative Solution. The overall alternative solution retains the core design concept but changes only the deployment method or evaluation scenario of the evaluation system. Specifically, it includes: Deployment of cloud-based evaluation system: The evaluation system of this application is deployed from the local data collection terminal to the cloud, supporting multiple mobile phones under test to access the cloud system through the network to realize remote automated evaluation, which is suitable for batch evaluation needs across regions.

[0148] Portable testing equipment integration: The core testing algorithm of this application is integrated into portable hardware devices (such as portable power meters and mobile phone testing instruments) to achieve rapid offline testing without the need to build a professional laboratory environment. It is suitable for offline quality control by mobile phone manufacturers and on-site testing by the media.

[0149] Full-scenario evaluation expansion: From static laboratory evaluation to dynamic evaluation in all scenarios, adding outdoor, high and low temperature, and humid environment scenarios, as well as multi-task parallel use scenarios (such as edge-side large model inference + video playback + navigation), to more comprehensively reflect the actual user experience.

[0150] 2. Alternative or modified solutions for core technical features.

[0151] The core technological features are replaced or modified in a way that keeps the core innovation point unchanged, but only optimizes or adjusts the specific implementation method, including: 2-1. Variations of the standardized task slice library: The task is divided into slices based on the number of parameters of the large end-side model (e.g., small model < 1 billion parameters, medium model 1-5 billion parameters, large model > 5 billion parameters) to adapt to the evaluation needs of large end-side models with different parameter numbers.

[0152] Add multimodal task slices (such as text-image hybrid generation and audio-visual understanding) to adapt to the development trend of large multimodal edge models.

[0153] 2-2. Alternatives to multi-dimensional data collection: If the phone does not have a power consumption interface for the underlying AI subsystem, a chip-level power consumption acquisition method (acquiring NPU core power consumption through the chip debugging interface) is used to replace the whole-device power consumption deduction method, thereby improving the accuracy of power consumption acquisition.

[0154] Added battery health correlation analysis to collect the inference power consumption of the large model on the device under different battery health conditions, adapting to the evaluation needs of older mobile phones.

[0155] 2-3. Variation of dynamic power consumption threshold adaptation: By introducing a machine learning model and training it with a large amount of preheating calibration data from mobile phones, intelligent prediction of dynamic power consumption thresholds can be achieved, replacing the traditional mapping table method and improving the efficiency and accuracy of threshold determination.

[0156] Enhanced heat dissipation adaptation: Adjusted dynamic power consumption thresholds based on the heat dissipation method (air cooling / liquid cooling / passive heat dissipation) of the phone under test to adapt to phones with different heat dissipation designs.

[0157] 2-4. Alternatives to Energy Efficiency Ratio Calculation: An energy efficiency ratio decay rate index is introduced to calculate the decay of the energy efficiency ratio during continuous inference and to evaluate the stability of large-scale inference at the end side.

[0158] The energy efficiency ratio is calculated separately for different model accuracies, forming an accuracy-energy efficiency mapping table, which provides a basis for the accuracy selection of the large-scale end-side model.

[0159] 2-5. Variations of the comprehensive scoring model: By introducing personalized weights based on user needs, the weights of indicators in the comprehensive scoring model are adjusted according to the needs of different user groups (such as business users, game users, and creative users) to achieve personalized scoring.

[0160] A machine learning-based dynamic scoring model is adopted, which continuously trains the model through industry data and user feedback to automatically optimize the weight coefficients, replacing fixed weight coefficients.

[0161] 3. Possible design solutions to avoid this problem.

[0162] Based on the core technical features of this application, we anticipate possible design solutions to avoid these limitations, including: 3-1. Change the classification method of task slices: Task slices can be divided according to reasoning type (generative / comprehension) instead of application scenario, but the core is still standardized task slices, which does not deviate from the design concept of this application.

[0163] 3-2. Adjusting the energy efficiency ratio calculation index: Inference speed / power consumption (token / (s·W)) can be used as the energy efficiency index instead of energy efficiency ratio, but the core is still the quantification of the ratio of performance to power consumption, which does not deviate from the energy efficiency quantification idea of ​​this application.

[0164] 3-3. Modify the dynamic control method: thermal throttling can be avoided by reducing model accuracy instead of adjusting task density, but the core is still adaptive control based on temperature monitoring, which does not deviate from the thermal throttling avoidance idea of ​​this application.

[0165] Adding non-core evaluation metrics: Non-core metrics such as battery temperature and CPU utilization can be added to the multi-dimensional data collection, but the core remains the synchronous collection of core metrics such as inference, computing power, memory and power consumption, which does not deviate from the multi-dimensional evaluation system of this application.

[0166] Regarding the aforementioned design schemes that circumvent these limitations, the core protection scope of this application already covers the core design concepts of standardized task slicing, multi-dimensional synchronous acquisition, dynamic power consumption adaptation, energy efficiency ratio quantification, and comprehensive scoring. Regardless of how the specific implementation method is adjusted, as long as it does not deviate from the core concept, it falls within the protection scope of this application.

[0167] Furthermore, based on this application and considering the rapid development of edge-side large-scale model technology and the changing needs of the smartphone evaluation industry, the following supplementary viewpoints and ideas are proposed to further improve and expand the application scope and technical solutions of this application, and enhance its innovation and practicality: 1. Lifecycle evaluation of large-scale edge model inference performance.

[0168] Building upon this application, long-term continuous inference testing (such as running a large-scale on-device inference task continuously for 24 hours) is added to collect the degradation patterns of inference performance, power consumption, and energy efficiency, and to evaluate the stability of large-scale on-device inference throughout its entire lifecycle. The implementation plan involves: designing a long-term testing task process, adding hardware aging monitoring indicators (such as NPU computing power degradation rate and battery capacity degradation rate), and constructing a performance-power consumption mapping model for large-scale on-device inference throughout its entire lifecycle, providing a basis for hardware reliability design for mobile phone manufacturers.

[0169] 2. Compatibility evaluation of the large-scale terminal model with mobile phone hardware.

[0170] This application primarily evaluates the inference performance of large-scale mobile phone models. It can be further expanded to evaluate the compatibility of large-scale models with mobile phone hardware, analyzing the compatibility of different large-scale models on the same mobile phone and the same large-scale model on different mobile phones. The implementation plan involves: constructing a large-scale mobile phone model library (containing large-scale models from different manufacturers, with different parameter values, and of different types); and calculating the compatibility score (based on inference performance, power consumption, and energy efficiency) of each model with each mobile phone using the evaluation method described in this application. This provides a reference for mobile phone manufacturers' pre-installation of large-scale models and for users' selection of large-scale models.

[0171] 3. Automatic generation of optimization suggestions for large-scale inference on the edge side.

[0172] Based on the evaluation report, an automatic optimization suggestion generation module is added. This module provides targeted optimization suggestions for large-scale on-device model inference to chip and mobile phone manufacturers based on the evaluation results. The implementation plan involves: building an optimization suggestion knowledge base (including NPU scheduling optimization, memory allocation optimization, model quantization optimization, power consumption control optimization, etc.); and automatically generating quantified optimization suggestions by matching the evaluation results with the knowledge base (e.g., increasing the number of NPU scheduling threads from 4 to 6 can improve computing power utilization by 15%).

[0173] 4. Establishment of industry standards for edge-side large model inference evaluation.

[0174] The standardized task slicing library, multi-dimensional indicator system, energy efficiency ratio indicator, and comprehensive scoring model of this application can serve as a draft industry standard for low-power evaluation of large-scale model inference performance on mobile devices, promoting the formation of unified evaluation standards in the industry. The implementation plan involves collaborating with chip manufacturers, mobile phone manufacturers, and industry associations to verify and optimize the technical solutions of this application, forming industry standard documents that include task slicing standards, indicator collection standards, energy efficiency level standards, and comprehensive scoring standards, thereby promoting the standardized development of the industry.

[0175] 5. Real-time evaluation tool for edge-side large model inference performance.

[0176] This application's core evaluation algorithm is lightweighted, and a real-time mobile app for evaluating the inference performance of large-scale models is developed, allowing ordinary users to test the inference performance, power consumption, and energy efficiency of large-scale models on mobile devices. The implementation plan involves: lightweighting the evaluation algorithm, developing a mobile app adapted for mobile devices, simplifying task segments into a lightweight version, achieving one-click evaluation and rapid results, and providing a reference for users when choosing mobile phones and using large-scale models on mobile devices. The main lightweighting process includes: (1) Simplified evaluation tasks: Select the most representative basic reasoning tasks from the standardized task slice library, such as text generation and visual description tasks, and significantly reduce the input scale, output length and iteration number of a single task to ensure that the evaluation tasks can be completed quickly on the mobile phone (e.g. within 1 minute) without generating significant heat and power consumption.

[0177] (2) Data acquisition and algorithm simplification: a) Data Acquisition: Only core experience metrics such as overall device power consumption, task latency, and task completion rate are collected. Overall device power consumption is obtained through the power estimation interface built into the mobile phone system (such as the PowerProfile API), and latency is obtained by the APP's local timer. It no longer relies on external high-precision power consumption devices, nor does it collect underlying data such as NPU computing power utilization and memory usage.

[0178] b) Core Algorithm: The energy efficiency ratio calculation is simplified to "number of tasks completed / unit power consumption" or "average processing speed / unit power consumption," focusing on "efficiency" from the user experience perspective. Adaptive regulation is simplified to be based on the phone's battery temperature rather than the core temperature, dynamically controlling the pause / resumption of the evaluation task queue to ensure the stable operation of the APP itself.

[0179] (3) Integration and Real-time Performance: All modules, including the lightweight "task loading, execution, data acquisition, energy efficiency calculation, and result output", are integrated into the mobile APP. Tasks are deployed with the APP in the form of fixed model files and configuration files. After the APP runs, it automatically completes the entire process from task execution to score display and presents users with simple performance levels (such as S / A / B / C / D) and core energy efficiency assessment.

[0180] The above supplementary viewpoints and ideas are all based on the core technical solution of this application and do not deviate from the innovation of this application. They further expand the application scope and use value of this application, which can not only provide support for the technological development of the industry, but also provide practical tools for ordinary users, and have good industrialization prospects.

[0181] Based on the above description, compared with the prior art, this application has at least the following key differences, and these key differences complement each other to form a complete technical solution: 1. Standardized task slicing system for large-scale edge models.

[0182] Existing technologies lack a unified evaluation task for large edge models. This application addresses typical application scenarios of large edge models by breaking them down into four standardized task segments: text, vision, speech, and code. By fixing parameters such as input size, model accuracy, and execution strategy, the evaluation task becomes standardized and reproducible.

[0183] As can be seen, this application is the first to construct a standardized task slice library dedicated to large-scale models on the device side, which is adapted to the hardware limitations of mobile devices and fits the actual usage scenarios of users, laying the foundation for horizontal comparison of different mobile phones and different chip architectures.

[0184] 2. A multi-dimensional simultaneous evaluation system for performance, power consumption, and energy efficiency.

[0185] Existing technologies only focus on single performance indicators (such as latency and computing power) without considering indicators such as power consumption and memory. This application achieves millisecond-level synchronous collection of inference latency, NPU computing power utilization, memory usage, and real-time power consumption through underlying interfaces and external devices, and constructs a multi-dimensional evaluation system.

[0186] As can be seen, this application is the first to achieve multi-dimensional synchronous and high-precision collection of inference performance, computing power utilization, memory usage and power consumption of large-scale inference on the edge, comprehensively reflecting the overall performance of large-scale inference on the edge.

[0187] 3. Dynamic power consumption threshold adaptive adjustment mechanism.

[0188] Existing technologies do not consider the impact of thermal throttling on evaluation results. This application constructs a temperature-power consumption-performance mapping relationship through preheating calibration, determines the dynamic power consumption threshold in the non-throttling range, monitors the temperature in real time and dynamically adjusts the task density during the evaluation process, and ensures that the evaluation is carried out in a non-throttling state throughout the entire process.

[0189] As can be seen, this application adaptively determines the power consumption threshold based on the hardware characteristics of the mobile phone under test, and dynamically adjusts the task density rather than modifying the task itself. This avoids the impact of thermal throttling and ensures the standardization of the evaluation load, making it compatible with different brands and models of mobile phones.

[0190] 4. Quantitative indicators of energy efficiency ratio for end-side large-scale model inference.

[0191] Existing technologies lack energy efficiency evaluation indicators for edge-side large model inference. This application introduces the energy efficiency ratio (TOPS / W) as a core indicator to distinguish between the specific power consumption of the AI ​​subsystem and the overall power consumption, accurately calculate the "computing power-power consumption" balance capability of edge-side large model inference, and classify energy efficiency levels.

[0192] As can be seen, this application is the first to introduce energy efficiency ratio into the evaluation of large model inference on the mobile device side, realize the quantification of performance-power consumption balance capability, and provide the industry with a unified energy efficiency evaluation standard.

[0193] 5. End-side large-scale model reasoning comprehensive scoring model.

[0194] Existing technologies lack comprehensive quantitative scoring models. This application selects inference latency, NPU computing power utilization, peak memory usage, average power consumption, and energy efficiency ratio as core indicators. Weighting coefficients are determined through experimental data and expert opinions to construct a comprehensive scoring model, transforming multi-dimensional indicators into an intuitive comprehensive score.

[0195] As can be seen, this application constructs a comprehensive scoring model that takes into account performance, power consumption, and energy efficiency, and realizes a quantitative evaluation of the comprehensive performance of the large-scale inference model on the edge. The scoring results are scientific, objective, and intuitive.

[0196] 6. Low-power automated evaluation system for large-scale edge model inference performance Existing technologies mostly rely on manual evaluation, which is inefficient and prone to errors. This application integrates five major modules: task loading, data acquisition, adaptive control, energy efficiency calculation, and result output, to automate the entire evaluation process and support multi-format report output and cross-model comparison.

[0197] As can be seen, this application is the first to design and implement an automated evaluation system for low-power edge-side large model inference performance, which improves evaluation efficiency, reduces human error, and is suitable for batch evaluation needs in laboratories.

[0198] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 10 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores low-power evaluation data for large-model inference performance on mobile devices. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a low-power evaluation method for large-model inference performance on mobile devices.

[0199] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment to which the present application is applied. Specific computer equipment may include, for example, [the following is a list of possible additional structures]. Figure 10 The diagram shows more or fewer components, or combinations of certain components, or different component arrangements.

[0200] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0201] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0202] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0203] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0204] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (RRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0205] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0206] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A low-power evaluation method for large model inference performance on a mobile device, characterized in that, include: Construct a standardized task slice library for the large-scale edge model, and define the parameters of the task slices in the standardized task slice library for the large-scale edge model to generate a standardized task slice package. The standardized task slice package is loaded into the phone under test and a consistency check is performed. The phone under test was preheated and calibrated to establish a temperature-power consumption-performance mapping relationship and determine the temperature threshold. According to the hierarchical evaluation system, inference tasks are sent to the mobile phone under test based on the standardized task slice package that has passed the consistency verification, and the temperature of the mobile phone under test is monitored in real time during the execution of the inference tasks; when the temperature reaches or exceeds the temperature threshold, the inference task sending density is dynamically adjusted. During the execution of the inference task, multi-dimensional indicator data are collected synchronously and preprocessed to generate a standardized dataset; The inference efficiency ratio of the large edge model is determined based on the standardized dataset, and the evaluation results are obtained by combining the comprehensive scoring model.

2. The low-power evaluation method for large model inference performance on the mobile device side according to claim 1, characterized in that, The standardized task slice library for large edge models covers scenarios including text, vision, speech, and code.

3. The low-power evaluation method for large model inference performance on the mobile device side according to claim 1, characterized in that, The standardized task slice package is loaded into the phone under test, and a consistency check is performed, including: The hash values ​​of the standardized task slice package are determined on both the server and the mobile device under test, and then compared. If the hash values ​​match, the verification passes; if the hash values ​​do not match, the evaluation is terminated or the standardized task slice package is automatically reloaded for a second verification.

4. The low-power evaluation method for large model inference performance on the mobile device side according to claim 1, characterized in that, The phone under test is preheated and calibrated to establish a temperature-power consumption-performance mapping relationship and determine the temperature threshold, including: Run lightweight text-based task slices to periodically collect the temperature, NPU operating frequency, and real-time power consumption of the back panel area of ​​the phone under test, and construct the temperature-power consumption-performance mapping relationship. Based on the temperature-power consumption-performance mapping relationship, the temperature threshold is set at a temperature lower than the critical temperature setting value that triggers thermal throttling of the mobile phone.

5. The low-power evaluation method for large model inference performance on the mobile device side according to claim 1, characterized in that, Also includes: The average power consumption corresponding to the temperature threshold is used as the dynamic power consumption threshold to control the test power consumption.

6. The low-power evaluation method for large model inference performance on the mobile device side according to claim 1, characterized in that, When the temperature reaches or exceeds the temperature threshold, the distribution density of the inference task is dynamically adjusted, including: When the temperature reaches or exceeds the temperature threshold, the inference task distribution interval is adaptively adjusted from 0s to 500ms~2s until the temperature falls back to within the temperature threshold, and the original distribution interval is restored.

7. A low-power evaluation system for large model inference performance on a mobile device, characterized in that, include: The task loading module is used to load the standardized task slice package into the mobile phone under test and perform consistency verification; wherein, the construction process of the standardized task slice package includes: constructing a standardized task slice library of the terminal large model, defining the parameters of the task slices in the terminal large model standardized task slice library, and generating the standardized task slice package. An adaptive control module is used to distribute inference tasks to the mobile phone under test according to the hierarchical evaluation system based on the standardized task slice package that has passed the consistency verification, and to monitor the temperature of the mobile phone under test in real time during the execution of the inference task; when the temperature reaches or exceeds the temperature threshold, the distribution density of the inference task is dynamically adjusted. The data acquisition module is used to synchronously collect multi-dimensional indicator data during the execution of inference tasks and preprocess it to generate standardized datasets. The energy efficiency calculation module is used to determine the end-side large model inference energy efficiency ratio based on the standardized dataset; The results output module is used to obtain the evaluation results based on the energy efficiency ratio and the comprehensive scoring model of the end-side large model.

8. The low-power evaluation system for large model inference performance on the mobile device side according to claim 7, characterized in that, The data acquisition module works in conjunction with an external professional power meter through the underlying API / SDK interface of the mobile phone to achieve synchronous acquisition of multi-dimensional indicator data at the millisecond level.

9. The low-power evaluation system for large model inference performance on the mobile device side according to claim 7, wherein the energy efficiency calculation module is further used to classify energy efficiency levels according to the energy efficiency ratio of the large model inference on the mobile device side; the energy efficiency level and the comprehensive score are presented synchronously in the evaluation results.

10. The low-power evaluation system for large model inference performance on the mobile device side according to claim 7, characterized in that, The result output module has a built-in structured report template, which supports the automatic generation of evaluation reports from the evaluation results; the evaluation report can be presented in at least one of the following formats: PDF, Excel, JPG, and PNG.

Images