Hybrid precision DNN performance cost model construction system based on heterogeneous SoC

Abstract

The invention relates to the technical field of deep learning acceleration, and discloses a heterogeneous SoC-based hybrid precision DNN performance cost model construction system, which obtains time and power consumption of different data volumes from low precision to high precision and from high precision to low precision in precision conversion, quantifies the time for executing the precision conversion, and calculates the performance cost of the hybrid precision DNN performance cost model according to the power consumption of the hybrid precision DNN performance cost model. The method comprises the following steps: analyzing execution overhead and data handling overhead of precision conversion on a calculation pipeline, quantifying execution time, power consumption, interlayer switching overhead and precision conversion overhead of different mapping plans on a heterogeneous SoC by constructing a mixed precision execution cost model to construct a system, and performing layer configuration and statistical information through the system to determine calculation precision of each layer. The power consumption and the execution time under different environment conditions are estimated, the power consumption and the performance table of the system under different temperature conditions are simulated, a real-time monitoring mechanism is implemented to detect the temperature change, and the realized system has higher energy efficiency and is suitable for an energy-sensitive application scene.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning acceleration and discloses a mixed-precision DNN performance cost model construction system based on heterogeneous SoC. Background Art

[0002] In order to build a mixed-precision DNN performance cost model based on heterogeneous SoCs, researchers first need to characterize the overhead caused by precision conversion. When precision conversion occurs between layers of a DNN, such as from INT8 to FP16, or vice versa, this conversion not only involves changes in data size, but also causes pipeline interruptions and additional data movement. Therefore, researchers need to have a deep understanding of the hardware architecture of each DSA, as this will directly affect the read / write behavior of the DSA and how they interact with each other.

[0003] In the experiment, the researchers selected AlexNet as a representative neural network model, mapped its layers to NVIDIA's Xavier SoC, and adopted three different precision configurations: pure INT8, mixed precision, and pure FP16. The experimental results show that although INT8 execution performs best in terms of speed and energy efficiency, it has the lowest accuracy due to further quantization loss; while FP16 execution performs well in accuracy, but is not the most ideal option in terms of speed and energy consumption. Therefore, mixed precision execution becomes a compromise that can find a balance between accuracy, performance, and energy consumption.

[0004] In order to achieve the best performance and energy efficiency on heterogeneous systems, it is not enough to simply configure the computational precision of each layer. It is also necessary to reasonably allocate the layers to accelerators (such as DLA and GPU) with higher performance or energy efficiency. These two optimization goals are closely linked and jointly affect the final execution time and energy consumption. Given the huge design space, it is almost impossible to obtain the optimal mapping through random search. Therefore, the researchers proposed using the system to determine the optimal mapping strategy, predict performance through a cost model and provide feedback, so as to find the best balance between multiple precision configurations and accelerator allocation.

[0005] Mixed-precision inference provides a more viable option that helps balance speed and power consumption while ensuring model accuracy. For example, when the input is affected by weather conditions, higher computational precision can be used to restore accuracy, while when weather conditions are good, more quantized weights and intermediate results can be used to benefit from lower latency and energy consumption. The researchers' preliminary experiments only demonstrated the impact of different precision configurations based on a single device mapping plan, and further exploration is needed to find the optimal mixed-precision execution scheme in more complex heterogeneous systems.

[0006] For example, the present invention with publication number CN112633516A provides a performance prediction system and device, which includes: determining the operator planning of the performance to be predicted; taking each tensor in the operator planning as a node and the connection relationship between each tensor as an edge, constructing a description graph of the operator planning; wherein the node feature of any node is the planning information of the corresponding tensor, and the edge feature of any edge is the size information of the tensor corresponding to the point connected by the edge; inputting the description graph into the performance prediction model to obtain the performance prediction result of the operator planning output by the performance prediction model; wherein the performance prediction model is obtained based on the training of sample operator planning and its sample performance parameters. A performance prediction system and device provided by the present invention fully considers the planning information of each tensor in the internal data flow of the operator planning, and improves the accuracy of the performance prediction of the operator planning.

[0007] The above patent description diagram is built based on static operator planning and cannot fully reflect the actual behavior at runtime. For example, dynamic factors such as memory access patterns and cache utilization may have a significant impact on performance during actual execution. The effectiveness of the performance prediction model depends largely on the quality and diversity of training samples. If the training set is not extensive or representative enough, the model may not be well generalized to unseen operator planning. Once the performance prediction model is trained and deployed, if there is no effective mechanism to continuously collect feedback and update the model, then as the hardware or software changes, the prediction accuracy of the model may gradually decrease. Summary of the invention

[0008] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0009] In order to solve the above technical problems, the main purpose of the present invention is to provide a mixed-precision DNN performance cost model construction system based on heterogeneous SoC, including:

[0010] The data acquisition module is used to obtain the time and power consumption of converting different amounts of data from low precision to high precision and from high precision to low precision;

[0011] The data processing module is used to quantify the time and power consumption cost of executing the precision conversion, and analyze the execution overhead and data handling overhead of the precision conversion on the computing pipeline;

[0012] A cost calculation module, including a mixed precision execution cost model and a Boolean variable simplification model, wherein the mixed precision execution cost model is used to quantify the execution time, power consumption, inter-layer switching overhead and precision conversion overhead of different mapping plans on heterogeneous SoCs, and the Boolean variable simplification model is used to solve the execution time and power consumption process;

[0013] A candidate module, including a candidate mapping unit and a cost model, wherein the candidate mapping unit schedules the total execution time by setting constraints and calculates the calculation accuracy of layer configuration and statistical information, and the cost model is used to feedback the execution time of each layer of the neural network, the inter-layer switching overhead and the precision conversion overhead;

[0014] The estimation module is used to estimate the power consumption and execution time under different environmental conditions, simulate the power consumption and performance of the system under different temperature conditions, and implement a real-time monitoring mechanism to detect temperature changes and dynamically adjust the system settings according to the current temperature.

[0015] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0016] The method for the data acquisition module to acquire the time and power consumption of different data amounts from low precision to high precision and from high precision to low precision includes:

[0017] S101, obtaining the running time and power consumption cost of different data amounts on the specified computing device, and measuring the execution time of each neural network layer under different computing complexities and different computing precisions;

[0018] S102, determining the execution time of each layer when using low precision and high precision on the DLA and GPU respectively, and recording corresponding power consumption data;

[0019] S103. If the accuracy is improved by converting lower low precision to higher high precision, the memory bandwidth requirement will increase due to the increase in data volume. If the device resource consumption limit converts high precision to low precision, the data volume will decrease and the memory bandwidth requirement will decrease.

[0020] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0021] The execution overhead includes the time of precision conversion and the power consumption of precision conversion;

[0022] The data handling overhead includes data transmission time and data transmission power consumption.

[0023] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0024] The mixed precision execution cost model includes layer mapping scheduling, total execution time function and Boolean variables, evaluation function and calculation unit;

[0025] The layer mapping schedule is used to determine the allocation of each layer in the neural network to a specific accelerator and select a precision conversion method;

[0026] The total execution time function includes layer execution time, inter-layer switching overhead and precision conversion overhead;

[0027] The inter-layer switching and precision conversion calculation function is used to calculate the memory layer switching overhead and the precision conversion overhead.

[0028] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0029] Constructing the cost model to calculate the execution time and power consumption of the neural network, determining the execution accuracy conversion strategy of each layer of the neural network, and predicting the execution time and power consumption under different mapping schemes;

[0030] The cost model includes a calculation unit for calculating the execution time and power consumption of inter-layer switching, precision conversion, and excessive costs.

[0031] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0032] The layer execution time is used to allocate a specific accelerator and select the execution time of the precision conversion strategy for each layer in the processing neural network.

[0033] The inter-layer switching overhead is used to handle the data conversion between two layers of continuous processing, switching different accelerators, and calculating the additional time cost;

[0034] If the data precision between the front and back layers is different, the precision format conversion is performed and the precision conversion overhead is calculated.

[0035] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0036] The Boolean variables include TRi and CRi, which are used to indicate whether inter-layer switching or precision conversion is required, and TRi and CRi are binary numbers;

[0037] The evaluation function calculation expression is as follows:

[0038] T=[L,S(L)]

[0039] Where T is the total time and power consumption required to evaluate the cost model to execute a given solution, L is the cost model, and S(L) is the given solution;

[0040] The computing unit is used to calculate the execution time and precision conversion cost of the precision conversion strategy of the i-th layer Li of the neural network using the S(L) given solution on the accelerator Aj;

[0041] The calculation unit is used to calculate the transition cost and precision conversion cost of the precision conversion strategy of the i-th layer Li from the accelerator Aj to Aj+1 using the given solution of S(L);

[0042] The calculation unit is used to calculate the transition cost and precision conversion cost of the i-th layer Li to S(L+1) using the precision conversion strategy of the given solution S(L) in the accelerator Aj.

[0043] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0044] The candidate module is used to generate multiple candidate mapping units, and use the cost model to evaluate the total execution time of each candidate plan, and select the one with the performance constraint and the minimum total execution time;

[0045] Candidate modules include;

[0046] S501, generating multiple candidate mapping units, and using a cost model to evaluate the total execution time of each candidate plan;

[0047] S502, mapping candidate mapping units of all layers of the neural network to the DLA accelerator, and calculating the minimum energy consumption by the computing unit;

[0048] S503, the computing layer configuration and statistical information determine the computing accuracy of each layer, and calculate the execution time and precision conversion cost of the precision conversion strategy of the i-th layer Li of the neural network on the accelerator Aj using S(L) given the scheme through the cost model, guide the agent to adjust the precision setting to meet the preset performance requirements, and if the performance target is achieved, provide the neural network with additional training cycles, and feedback to the agent through the verification accuracy as a reward function until the precision conversion strategy of the given scheme converges;

[0049] S504, if the precision conversion strategy of the current given solution still cannot meet the constraints, gradually switch the accelerator from DLA to GPU until the constraints are met;

[0050] S505. Complete the construction of an optimized mapping plan that can efficiently perform mixed-precision neural network inference on heterogeneous SoCs.

[0051] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0052] Execute the i-th layer Li of the neural network through the neural network on the accelerator Aj using the precision conversion strategy of the given scheme S(L), and calculate the execution time and power consumption cost;

[0053] Through modeling and simulation, we can calculate the power consumption and execution time under different environmental conditions, simulate the power consumption and performance of the system under different temperature conditions, implement a real-time monitoring mechanism to detect temperature changes, and dynamically adjust the DLA switching GPU strategy according to the temperature.

[0054] As a preferred solution of the mixed precision DNN performance cost model construction system based on heterogeneous SoC of the present invention, wherein:

[0055] Predicting the execution time and power consumption cost of the precision conversion strategy for a given solution through a neural network;

[0056] After the neural network is layered, the computing device and its computing accuracy used in each layer of the network are specified. By matching the model of the corresponding device, the time and power consumption cost of running the solution on the current device can be calculated.

[0057] Beneficial effects of the present invention:

[0058] The present invention predicts network time and power consumption costs through convolution operations, making the system more energy efficient, suitable for energy-sensitive application scenarios, and applied to deep learning reasoning for image processing or massive parallel computing.

[0059] The present invention takes into account data flow, computing efficiency and storage requirements by setting up a neural network. The work of each submodule is coordinated through the control bus and configuration module, and the required data is quickly accessed through the memory interface, ensuring the efficient execution of the reasoning task, which is suitable for real-time applications and large-scale computing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0061] Figure 1 A system flow chart is constructed for the mixed-precision DNN performance cost model based on heterogeneous SoC of the present invention;

[0062] Figure 2A schematic diagram of a system computing architecture for constructing a mixed-precision DNN performance cost model based on heterogeneous SoCs in the present invention;

[0063] Figure 3 A schematic diagram of the reasoning process of constructing a system neural network based on a mixed-precision DNN performance cost model of a heterogeneous SoC according to the present invention;

[0064] Figure 4 A schematic diagram of the architecture of a system convolutional neural network accelerator constructed based on a mixed-precision DNN performance cost model of a heterogeneous SoC according to the present invention;

[0065] Figure 5 A system composition diagram is constructed for the mixed-precision DNN performance cost model based on heterogeneous SoCs in the present invention. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0068] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0069] Embodiment 1:

[0070] like Figure 1 As shown in the figure, a method for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC includes:

[0071] S1, the time and power consumption of obtaining different data amounts from low precision to high precision and from high precision to low precision;

[0072] S2. quantify the time and power consumption cost of executing the precision conversion, and analyze the execution overhead and data handling overhead of the precision conversion on the computing pipeline;

[0073] S3. Quantify the execution time, power consumption, inter-layer switching overhead, and precision conversion overhead of different mapping plans on heterogeneous SoCs by building a mixed-precision execution cost model, and simplify the process of solving the execution time and power consumption of the model through Boolean variables;

[0074] S4. Establish a candidate mapping plan, set constraints to schedule the total execution time, determine the calculation accuracy of each layer through layer configuration and statistical information, and use the cost model calculation results to feedback the execution time of each layer of the neural network, the cost of switching between layers, and the cost of precision conversion;

[0075] S5. Estimate the power consumption and execution time under different environmental conditions, simulate the power consumption and performance of the system under different temperature conditions, and implement a real-time monitoring mechanism to detect temperature changes and adjust the system settings according to the current temperature.

[0076] like Figure 5 As shown, the mixed precision DNN performance cost model construction system based on heterogeneous SoC provided by this application includes:

[0077] The data acquisition module is used to obtain the time and power consumption of converting different amounts of data from low precision to high precision and from high precision to low precision;

[0078] The time and power consumption for obtaining different data amounts from low precision to high precision and from high precision to low precision conversion include,

[0079] S101, obtaining the running time and power consumption cost of different data amounts on a specified computing device, and measuring the execution time of each neural network layer under different computational complexities and different computational precisions;

[0080] S102, determine the execution time of each layer when using low precision and high precision on DLA and GPU respectively, and record the corresponding power consumption data

[0081] S103, if the accuracy is increased by converting lower low precision to higher high precision, the memory bandwidth requirement will increase due to the increase in data volume; if the device resource consumption is limited by converting high precision to low precision, the data volume will decrease and the memory bandwidth requirement will decrease;

[0082] Collect the time and power cost of precision conversion under different data amounts. This step covers conversion from lower precision (such as INT8) to higher precision (such as FP16), and vice versa, because both cases will lead to changes in memory bandwidth requirements. In this process, it is necessary to measure not only the time required for the conversion operation itself, but also the impact of the conversion on the computing pipeline, including potential increase in data movement and changes in access patterns to the internal memory hierarchy.

[0083] The data processing module is used to quantify the time and power consumption cost of executing the precision conversion, and analyze the execution overhead and data handling overhead of the precision conversion on the computing pipeline;

[0084] The execution overhead includes the time of precision conversion and the power consumed by precision conversion;

[0085] Data handling overhead includes data transmission time and data transmission power consumption;

[0086] It is necessary to obtain the running time and power consumption cost of different data amounts on a specified computing device (such as DLA, GPU), which means that the execution time of each neural network layer must be measured under different computational complexity and precision;

[0087] Furthermore, we experimentally determined the execution time of each layer when using INT8 and FP16 precision on DLA and GPU respectively, and recorded the corresponding power consumption data. This step can be achieved through NVIDIA's offline analysis tool IProfiler to accurately measure the execution time of each layer. The tool provides time information from startup, execution to completion.

[0088] Furthermore, the time and power consumption cost of moving different data between computing devices are obtained. Because in a heterogeneous computing environment, data may be moved between different accelerators, the additional overhead caused by this data movement operation is measured. This overhead includes data transmission time and the power consumption associated with it. Due to the architectural differences between DLA and GPU, especially the differences in internal memory levels, significant overhead may be incurred when transferring large amounts of data between DLA and GPU. Therefore, in the experiment, the time it takes to transfer data between accelerators under different data volume conditions is recorded, and the corresponding power consumption is calculated, so as to construct a mixed-precision DNN performance cost model suitable for heterogeneous SoCs.

[0089] A cost calculation module, including a mixed precision execution cost model and a Boolean variable simplification model, wherein the mixed precision execution cost model is used to quantify the execution time, power consumption, inter-layer switching overhead and precision conversion overhead of different mapping plans on heterogeneous SoCs, and the Boolean variable simplification model is used to solve the execution time and power consumption process;

[0090] The execution overhead includes the time of precision conversion and the power consumed by precision conversion;

[0091] Data handling overhead includes data transmission time and data transmission power consumption;

[0092] Furthermore, the mixed precision execution cost model includes layer mapping scheduling, total execution time function and Boolean variables, evaluation functions and computational units;

[0093] Layer mapping scheduling is used to determine the allocation of each layer in the neural network to a specific accelerator and select the precision conversion method;

[0094] The total execution time function includes layer execution time, inter-layer switching overhead, and precision conversion overhead;

[0095] Layer execution time is the execution time for allocating a specific accelerator and selecting a precision conversion strategy for each layer in the processing neural network.

[0096] The inter-layer switching overhead is used to handle the data conversion between two layers of continuous processing, switching different accelerators, and calculating the additional time cost;

[0097] If the data precision between the front and back layers is different, the precision format conversion is performed and the precision conversion overhead is calculated.

[0098] The inter-layer switching and precision conversion calculation functions are used to calculate the memory layer switching overhead and precision conversion overhead.

[0099] Furthermore, the constructed cost model calculates the execution time and power consumption of the neural network, determines the execution precision conversion strategy of each layer of the neural network, and predicts the execution time and power consumption under different mapping schemes;

[0100] The cost model includes the execution time and power consumption of the computational units used to calculate inter-layer switching, precision conversion, and excessive costs;

[0101] Boolean variables include TRi and CRi, which are used to indicate whether inter-layer switching or precision conversion is required, and TRi and CRi are binary numbers;

[0102] The evaluation function calculation expression is as follows:

[0103] T=[L,S(L)]

[0104] Where T is the total time and power consumption required to evaluate the cost model to execute a given solution, L is the cost model, and S(L) is the given solution;

[0105] The computing unit is used to calculate the execution time and precision conversion cost of the precision conversion strategy of the i-th layer Li of the neural network on the accelerator Aj using the S(L) given scheme;

[0106] Calculation This unit is used to calculate the transition cost and precision conversion cost of the precision conversion strategy of the i-th layer Li from accelerator Aj to Aj+1 using the given solution of S(L);

[0107] The calculation unit is used to calculate the transition cost and precision conversion cost of the i-th layer Li in the accelerator Aj using the precision conversion strategy of the given solution S(L) to S(L+1);

[0108] Furthermore, a layer mapping scheduling rule is defined to map each layer of the neural network to a specific accelerator (such as DLA, GPU) and precision (such as INT8, FP16), and the total execution time is calculated through the computing unit, including the layer execution time, inter-layer switching overhead and precision conversion overhead. Boolean variables TRi and CRi are used to indicate whether inter-layer switching or precision conversion is required, and then the total execution time and power consumption T(L, S(L)) are calculated, where t(Li, Aj, Pk) represents the execution time and power consumption of the i-th layer Li on the accelerator Aj using precision Pk, τ(Li, Aj, Aj+1, Pk) and τ(Li, Aj, Pk, Pk+1) represent the transition cost and precision conversion cost, respectively. In this way, a cost model is constructed to quantify the execution time and power consumption of different mapping plans on heterogeneous SoCs, and the model is used to evaluate and optimize the mapping plan for mixed-precision neural network inference.

[0109] A candidate module, including a candidate mapping unit and a cost model, wherein the candidate mapping unit schedules the total execution time by setting constraints and calculates the calculation accuracy of layer configuration and statistical information, and the cost model is used to feedback the execution time of each layer of the neural network, the inter-layer switching overhead and the precision conversion overhead;

[0110] The system for optimizing the mapping plan is used to generate a plurality of candidate mapping units, and use the cost model to evaluate the total execution time of each candidate plan, and select the one with the performance constraint and the minimum total execution time;

[0111] Candidate modules include;

[0112] S501, generating multiple candidate mapping units, and using a cost model to evaluate the total execution time of each candidate plan;

[0113] S502, mapping candidate mapping units of all layers of the neural network to the DLA accelerator, and calculating the minimum energy consumption by the computing unit;

[0114] S503, using the system to perform layer configuration and statistical information to determine the calculation accuracy of each layer, and calculating the execution time and precision conversion cost of the precision conversion strategy of the i-th layer Li of the neural network on the accelerator Aj using S(L) given the scheme through the cost model, guiding the agent to adjust the precision setting to meet the preset performance requirements, and if the performance target is achieved, providing the neural network with additional training cycles, and feeding back to the agent through the verification accuracy as a reward function until the precision conversion strategy of the given scheme converges;

[0115] S504, if the precision conversion strategy of the current given solution still cannot meet the constraints, gradually switch the accelerator from DLA to GPU until the constraints are met;

[0116] S505. Complete the construction of an optimized mapping plan that can efficiently perform mixed-precision neural network inference on heterogeneous SoCs

[0117] Furthermore, multiple candidate mapping units use the cost model to evaluate the total execution time of each candidate plan, and select the mapping plan that meets the performance constraints and has the smallest total execution time as the optimal solution. In practical applications, all layers are first mapped to the DLA accelerator to achieve the lowest energy consumption, and then the system is used to perform layer configuration and statistical information to determine the calculation accuracy of each layer. The performance cost model provides direct feedback to guide the agent to adjust the accuracy setting to meet the performance requirements. Once the performance target is achieved, the verification accuracy is used as a reward function to feedback to the agent until the policy converges;

[0118] Furthermore, if the current strategy cannot satisfy the constraints, the accelerator is gradually switched from DLA to GPU until the constraints are satisfied. Through this system, an optimized mapping plan for efficient execution of mixed-precision DNN inference on heterogeneous SoCs is constructed;

[0119] An estimation module that estimates power consumption and execution time under different environmental conditions, simulates the power consumption and performance of the system under different temperature conditions, and implements a real-time monitoring mechanism to detect temperature changes and dynamically adjust system settings based on the current temperature;

[0120] wherein the i-th layer Li of the neural network is executed on the accelerator Aj by the neural network using the precision conversion strategy of the given scheme S(L), and the execution time and power consumption cost are calculated;

[0121] Through modeling and simulation, we calculate the power consumption and execution time under different environmental conditions, simulate the power consumption and performance of the system under different temperature conditions, implement a real-time monitoring mechanism to detect temperature changes, and dynamically adjust the DLA switching GPU strategy according to the temperature;

[0122] Predicting the execution time and power consumption cost of the precision conversion strategy for a given solution through a neural network;

[0123] After the neural network is layered, the computing device and its computing accuracy used in each layer are specified. By matching the model of the corresponding device, the time and power consumption cost of running the solution on the current device can be calculated;

[0124] Furthermore, the prediction of the network time and power consumption cost of a given solution includes: after the neural network is layered, the computing device and its computing accuracy used in each layer in the network are specified. By matching the model of the corresponding device, the time and power consumption cost of running the solution on the current device can be calculated;

[0125] like Figure 2As shown in Figure 1, a computing architecture mainly used to perform deep neural network (DNN) inference workloads. When the amount of data is small, the local memory is read and written through the intra-core bus for calculation, including

[0126] Shared Memory: Located at the top of the diagram, it is usually used to store intermediate calculation results or data, facilitating efficient access between different computing units. It is designed to be shared, which means that DLA and GPU can access it at the same time, improving the speed and efficiency of data processing;

[0127] The memory bus connects the shared memory with the two computing units, DLA and GPU. The memory bus is responsible for data transmission, allowing efficient and low-latency communication between DLA and GPU.

[0128] DLA (Deep Learning Accelerator): Located in the lower left corner, it is specially designed to accelerate specific computing tasks in deep learning, such as convolution operations. It is usually more energy efficient and suitable for energy-sensitive application scenarios such as mobile devices or embedded systems.

[0129] GPU (Graphics Processing Unit): Located in the lower right corner, it is highly versatile and suitable for handling complex computing tasks, especially in terms of performance. It can be used for high-load computing needs, such as image processing or deep learning reasoning with a large amount of parallel computing.

[0130] The entire architecture design allows DLA and GPU to switch and cooperate flexibly according to different computing needs. In specific tasks, DLA may process parts with high energy efficiency requirements, while GPU processes parts that require high computing performance. The design concept of this architecture is to achieve efficient resource utilization and optimized performance by rationally utilizing different computing resources and combining their respective advantages to adapt to modern computing needs, such as virtual reality / augmented reality (VR / AR) and autonomous driving. In the above step S200, the patient's EEG signal state is divided into interictal, preictal, ictal and postictal periods, and the postictal period is innovatively included in the scope of model recognition, which greatly improves the accuracy of the model.

[0131] Embodiment 2:

[0132] Figure 3 A schematic diagram of the reasoning process of a neural network (AlexNet) for the mixed-precision DNN performance cost model construction system based on heterogeneous SoC provided in this application. This picture shows the reasoning process of the AlexNet neural network in detail and performs performance evaluation for different precision configurations;

[0133] The overall structure is divided into four columns: each column represents a different execution scheme, namely: pure INT8, mixed precision (INT8 and FP16) and pure FP16;

[0134] For pure INT8 precision, all layers use 8-bit integer data format (INT8). Convolution + ReLU: used for feature extraction, calculated in INT8 format; Maximum pooling: reduce the size of the feature map, reduce the amount of calculation, and maintain the INT8 format; Fully connected layer: multiple fully connected layers, each layer uses INT8 for calculation. Final performance indicators: Accuracy: 62.3% (due to quantization, some information is lost, the accuracy is relatively low); Processing time for each picture: 5.9 milliseconds (efficient, suitable for fast reasoning); Energy consumption per picture: 122 millijoules (low energy consumption);

[0135] For mixed precision, some layers use INT8 and some layers use FP16. The network starts with INT8 format and then switches to FP16 format (for higher precision). The final performance indicators are: accuracy: 64.8% (improved compared to pure INT8); processing time for each image: 7.8 milliseconds (slightly slower than pure INT8), energy consumption per image: 143 millijoules (the increase in energy consumption reflects the effect of using FP16);

[0136] For pure FP16, all layers use 16-bit floating point data format (FP16). Final performance indicators: accuracy: 68.2% (the highest accuracy); processing time per image: 9.3 milliseconds (the slowest speed, indicating that FP16 has a larger amount of calculation); energy consumption per image: 158 millijoules (the highest energy consumption, reflecting the complexity of FP16 calculation);

[0137] For mixed precision, some layers use INT8 and some layers use FP16. The network starts with FP16 format and then switches to INT8 format (which plays a role of lower precision). The final performance indicators are: accuracy: 67.8% (an improvement over pure INT8); processing time per image: 6.8 milliseconds (slightly slower than pure INT8); energy consumption per image: 149 millijoules (the increase in energy consumption reflects the effect of using FP16);

[0138] Therefore, we can conclude that: for the INT8 execution scheme: although it is the fastest and most energy-efficient option, it does have shortcomings in accuracy and is suitable for applications that have high real-time requirements but low accuracy requirements; for the FP16 execution scheme: it provides the highest accuracy, but performs poorly in speed and energy efficiency, and is suitable for scenarios that require high accuracy; and for the mixed precision scheme: it provides a compromise between accuracy, speed, and energy consumption, balancing the three application scenarios. This layout and analysis can help researchers and engineers choose the appropriate execution mode and make trade-offs between performance, energy efficiency, and accuracy based on application requirements.

[0139] Embodiment 3

[0140] Figure 4 A schematic diagram of the architecture of a convolutional neural network (CNN) accelerator for a mixed-precision DNN performance cost model building system based on heterogeneous SoC provided in this application. The core of this architecture is how to effectively manage and transmit data to ensure efficient and stable system operation.

[0141] Among them, Control Bus is the control bus;

[0142] Configuration and control block Configuration and control block;

[0143] Input

[0144] Weight bias weight bias;

[0145] Convolution core convolution core;

[0146] Post processing;

[0147] Memory interfaceMemory interface;

[0148] SDRAM external memory;

[0149] Inrernal RAMInternal RAM;

[0150] In the architecture, the control bus serves as a bridge for communication between modules and is responsible for data transmission and control signal distribution. It connects the configuration and control modules and can pass instructions and status information to other components. The configuration and control module is located at the top of the architecture and is responsible for system configuration and management, such as receiving configuration commands from the control bus, setting convolution operation parameters, and monitoring system status to ensure that each module operates in coordination.

[0151] The input module is located on the left side of the architecture, facing the convolution core. Its task is to process and transmit input data to the convolution core and convert the image or feature map into a form suitable for calculation. In addition, it can also perform data preprocessing, such as normalization, to improve calculation accuracy. Next to the input module is the weight bias module, which provides the weights and bias values ​​required for the convolution operation, where the weights are the parameters learned during the training process, and the biases are used to adjust the convolution results to help improve the expressiveness of the model.

[0152] The central part of the architecture is the convolution core, which is the main computing unit of the system and performs convolution operations. The convolution core receives data from the input module and the weight bias module, performs convolution calculations, and outputs convolution feature maps, and supports different types of convolution operations. The post-processing module is located on the right side of the architecture and is connected to the convolution core. It is responsible for processing the output data after the convolution operation, including applying activation functions, pooling operations, etc. to enhance the feature representation and prepare the final feature map for subsequent classification or recognition tasks.

[0153] At the bottom of the architecture is the memory interface, which is responsible for data exchange with external memory (such as SDRAM) and internal memory (such as internal RAM). The memory interface ensures that data flows quickly and efficiently between the convolution core and other modules to reduce latency. SDRAM is used to store large amounts of data, such as input images, weights, and intermediate calculation results, while internal RAM is used to store critical intermediate data and calculation results to reduce data extraction latency and improve overall system performance.

[0154] In general, the design of the CNN accelerator takes into account data flow, computing efficiency, and storage requirements. The work of each sub-module is coordinated through the control bus and configuration module, and the required data is quickly accessed through the memory interface, ensuring efficient execution of reasoning tasks, which is suitable for real-time applications and large-scale computing environments.

[0155] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0156] Importantly, it should be noted that the construction and arrangement of the present application shown in a plurality of different exemplary embodiments are only exemplary. Although only two embodiments are described in detail in this disclosure, it should be readily understood by those who refer to this disclosure that many modifications are possible, for example, the size, scale, structure, shape and ratio of various elements, and parameter values ​​(e.g., temperature, pressure, etc.), installation arrangement, use of materials, color, directional changes, etc., without substantially departing from the novel teachings and advantages of the subject matter described in the application. For example, the element shown as integrally formed can be composed of multiple parts or elements, the position of the element can be inverted or otherwise changed, and the nature or number or position of the discrete element can be changed or changed. Therefore, all such modifications are intended to be included in the scope of the present invention. The order or sequence of any process or method step can be changed or reordered according to an alternative embodiment. Any "device plus function" clause is intended to cover the structure of the execution function described in this article, and is not only structurally equivalent but also equivalent structure. Without departing from the scope of the present invention, other replacements, modifications, changes and omissions can be made in the design, operating conditions and arrangement of the exemplary embodiment. Therefore, the invention is not limited to a specific embodiment, but extends to numerous modifications still falling within the scope of the appended claims.

[0157] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention) may not be described.

[0158] It should be understood that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but for those of ordinary skill having the benefit of this disclosure, the development effort will be a routine task of design, fabrication, and production without undue experimentation.

[0159] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A mixed-precision DNN performance cost model building system based on heterogeneous SoC, including: The data acquisition module is used to obtain the time and power consumption of converting different amounts of data from low precision to high precision and from high precision to low precision; The data processing module is used to quantify the time and power consumption cost of executing the precision conversion, and analyze the execution overhead and data handling overhead of the precision conversion on the computing pipeline; A cost calculation module, including a mixed precision execution cost model and a Boolean variable simplification model, wherein the mixed precision execution cost model is used to quantify the execution time, power consumption, inter-layer switching overhead and precision conversion overhead of different mapping plans on heterogeneous SoCs, and the Boolean variable simplification model is used to solve the execution time and power consumption process; A candidate module, including a candidate mapping unit and a cost model, wherein the candidate mapping unit schedules the total execution time by setting constraints and calculates the calculation accuracy of layer configuration and statistical information, and the cost model is used to feedback the execution time of each layer of the neural network, the inter-layer switching overhead and the precision conversion overhead; The estimation module is used to estimate the power consumption and execution time under different environmental conditions, simulate the power consumption and performance of the system under different temperature conditions, and implement a real-time monitoring mechanism to detect temperature changes and dynamically adjust the system settings according to the current temperature.

2. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 1, characterized in that: The method for the data acquisition module to acquire the time and power consumption of different data amounts from low precision to high precision and from high precision to low precision includes: S101, obtaining the running time and power consumption cost of different data amounts on the specified computing device, and measuring the execution time of each neural network layer under different computing complexities and different computing precisions; S102, determining the execution time of each layer when using low precision and high precision on the DLA and GPU respectively, and recording corresponding power consumption data; S103. If the accuracy is improved by converting lower low precision to higher high precision, the memory bandwidth requirement will increase due to the increase in data volume. If the device resource consumption limit converts high precision to low precision, the data volume will decrease and the memory bandwidth requirement will decrease.

3. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 2, characterized in that: The execution overhead includes the time of precision conversion and the power consumption of precision conversion; The data handling overhead includes data transmission time and data transmission power consumption.

4. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 3, characterized in that: The mixed precision execution cost model includes layer mapping scheduling, total execution time function and Boolean variables, evaluation function and calculation unit; The layer mapping schedule is used to determine the allocation of each layer in the neural network to a specific accelerator and select a precision conversion method; The total execution time function includes layer execution time, inter-layer switching overhead and precision conversion overhead; The inter-layer switching and precision conversion calculation function is used to calculate the memory layer switching overhead and the precision conversion overhead.

5. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 4, characterized in that: Constructing the cost model to calculate the execution time and power consumption of the neural network, determining the execution accuracy conversion strategy of each layer of the neural network, and predicting the execution time and power consumption under different mapping schemes; The cost model includes a calculation unit for calculating the execution time and power consumption of inter-layer switching, precision conversion, and excessive costs.

6. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 5, characterized in that: The layer execution time is used to allocate a specific accelerator and select the execution time of the precision conversion strategy for each layer in the processing neural network. The inter-layer switching overhead is used to handle the data conversion between two layers of continuous processing, switching different accelerators, and calculating the additional time cost; If the data precision between the front and back layers is different, the precision format conversion is performed and the precision conversion overhead is calculated.

7. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 6, characterized in that: The Boolean variables include TRi and CRi, which are used to indicate whether inter-layer switching or precision conversion is required, and TRi and CRi are binary numbers; The evaluation function calculation expression is as follows: T=[L,S(L)] Where T is the total time and power consumption required to evaluate the cost model to execute a given solution, L is the cost model, and S(L) is the given solution; The computing unit is used to calculate the execution time and precision conversion cost of the precision conversion strategy of the i-th layer Li of the neural network using the S(L) given solution on the accelerator Aj; The calculation unit is used to calculate the transition cost and precision conversion cost of the precision conversion strategy of the i-th layer Li from the accelerator Aj to Aj+1 using the given solution of S(L); The calculation unit is used to calculate the transition cost and precision conversion cost of the i-th layer Li to S(L+1) using the precision conversion strategy of the given solution S(L) in the accelerator Aj.

8. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 7, characterized in that: The candidate module is used to generate multiple candidate mapping units, and use the cost model to evaluate the total execution time of each candidate plan, and select the one with the performance constraint and the minimum total execution time; Candidate modules include; S501, generating multiple candidate mapping units, and using a cost model to evaluate the total execution time of each candidate plan; S502, mapping candidate mapping units of all layers of the neural network to the DLA accelerator, and calculating the minimum energy consumption by the computing unit; S503, the computing layer configuration and statistical information determine the computing accuracy of each layer, and calculate the execution time and precision conversion cost of the precision conversion strategy of the i-th layer Li of the neural network on the accelerator Aj using S(L) given the scheme through the cost model, guide the agent to adjust the precision setting to meet the preset performance requirements, and if the performance target is achieved, provide the neural network with additional training cycles, and feedback to the agent through the verification accuracy as a reward function until the precision conversion strategy of the given scheme converges; S504, if the precision conversion strategy of the current given solution still cannot meet the constraints, gradually switch the accelerator from DLA to GPU until the constraints are met; S505. Complete the construction of an optimized mapping plan that can efficiently perform mixed-precision neural network inference on heterogeneous SoCs.

9. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 7, characterized in that: Execute the i-th layer Li of the neural network through the neural network on the accelerator Aj using the precision conversion strategy of the given scheme S(L), and calculate the execution time and power consumption cost; Through modeling and simulation, we can calculate the power consumption and execution time under different environmental conditions, simulate the power consumption and performance of the system under different temperature conditions, implement a real-time monitoring mechanism to detect temperature changes, and dynamically adjust the DLA switching GPU strategy according to the temperature.

10. The system for constructing a mixed-precision DNN performance cost model based on heterogeneous SoC according to claim 9, characterized in that: Predicting the execution time and power consumption cost of the precision conversion strategy for a given solution through a neural network; After the neural network is layered, the computing device and its computing accuracy used in each layer are specified. By matching the model of the corresponding device, the time and time required to run the solution on the current device can be calculated. Power consumption cost.

Images