Dynamic precision adaptive computing system and method for edge ai inference processing devices

Abstract

The application provides a dynamic precision self-adaptive calculation system and method of an edge AI inference processing device, relates to the technical field of artificial intelligence inference acceleration and edge calculation, and can dynamically adapt inference precision to real-time changes of an edge scene through two-dimensional sensing of task semantic importance information and a running environment state, guarantee high-precision inference performance for key tasks and complex scenes with high importance, and adopt low-precision calculation for ordinary non-key tasks to reduce power consumption and calculation cost, so that the technical problem that existing edge AI inference cannot simultaneously consider precision guarantee and overall power consumption control of key tasks in a real-time changing edge scene is solved, and the application is suitable for precision automatic control of an edge AI inference device.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence inference acceleration and edge computing technology, and in particular to a dynamic precision adaptive computing system and method for edge AI inference processing devices. Background Technology

[0002] As artificial intelligence applications continue to extend to intelligent security, vehicle sensing, industrial vision, wearable devices, and other network edge terminals, edge inference systems face a more prominent balance challenge in terms of power consumption, computing power, real-time performance, and recognition accuracy. Higher accuracy modes are generally beneficial for improving recognition performance in complex scenarios, but they increase computational load, power consumption, and heat dissipation burden; lower accuracy modes are beneficial for reducing power consumption and increasing throughput, but they are prone to insufficient feature representation in low-light, occlusion, noise interference, or critical event scenarios.

[0003] In existing technologies, edge AI inference devices mostly employ fixed-precision configurations or rely solely on coarse-grained adjustments based on the state of a single resource. While computing circuits supporting multiple bit widths or precision modes exist, these solutions typically focus on supporting the different precision modes themselves, rather than dynamically switching between them during inference based on task semantic requirements and the operating environment. Therefore, in real-time changing edge scenarios, it remains difficult to balance ensuring accuracy for critical tasks with overall power consumption control.

[0004] Furthermore, existing dynamic adjustment schemes primarily control at the complete task granularity, rarely considering switching precision modes at task switching boundaries or between adjacent computational stages within a task without disrupting the continuity of current computation. Therefore, in dynamic scenarios, issues remain regarding insufficient flexibility in precision switching and inadequate resource utilization. Summary of the Invention

[0005] The technical problem solved by this invention is to provide a dynamic accuracy adaptive computing system and method for edge AI inference processing devices, which solves the technical problem that existing edge AI inference systems cannot simultaneously ensure the accuracy of critical tasks and control overall power consumption in real-time changing edge scenarios.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a dynamic precision adaptive computing system for an edge AI inference processing device, comprising a task and environment perception module, a dynamic precision decision module, and a precision reconfigurable computing array; The task and environment perception module is used to obtain the task semantic importance information and the running environment status corresponding to the current inference task. The dynamic accuracy decision module is connected to the task and environment perception module and is used to determine the accuracy corresponding to the current inference task based on the task semantic importance information and the state of the operating environment. The precision reconfigurable computing array is connected to the dynamic precision decision module and is used to switch the computing bit width and storage access bit width corresponding to the precision within the state transition window according to the precision corresponding to the current inference task, and to perform edge AI inference. The state transition window includes the time between the previous inference task and the current inference task or between adjacent computing stages within the current inference task.

[0007] Furthermore, the task and environment perception module includes a task semantic analysis submodule and an environment monitoring submodule; the task semantic analysis submodule is used to extract task semantic importance information representing the current task processing priority based on input data features and / or edge AI intermediate layer features; the environment monitoring submodule is used to collect the operating environment status.

[0008] Furthermore, the operating environment status includes the current task execution status, remaining battery power, chip temperature, and computing power utilization.

[0009] Furthermore, the task semantic importance information includes at least one of the following: channel-level activation value corresponding to the current inference task, target detection confidence, image region saliency, image brightness and contrast statistical features, edge density, and event urgency.

[0010] Furthermore, the dynamic precision decision module uses a lookup table method or a rule-based judgment method to determine the precision corresponding to the current inference task.

[0011] Furthermore, when determining the precision corresponding to the current inference task, if the semantic importance information of the task is inconsistent with the precision corresponding to the state of the operating environment, the precision is determined according to a preset arbitration rule, which includes a priority rule, an interval mapping rule, or a weighted judgment rule.

[0012] Furthermore, the precision-reconfigurable computing array includes a data path reconstruction network, a multiplexer matrix, an arithmetic logic unit, and a storage controller. The data path reconstruction network outputs a reconstruction control signal based on the precision corresponding to the current inference task. The reconstruction control signal includes the precision-corresponding data path connection relationship, the precision-corresponding computation bit width, and the precision-corresponding storage access bit width. The multiplexer matrix is ​​used to change the data path connection relationship to the precision-corresponding data path connection relationship. The arithmetic logic unit is used to change the computation bit width to the precision-corresponding computation bit width. The storage controller is used to change the storage access bit width to the precision-corresponding storage access bit width.

[0013] Furthermore, the precision reconfigurable computing array is also used to feed back the current task execution status to the task and environment awareness module. The current task execution status is used to update the running environment status. The current task execution status information includes at least one of the following: the precision used by the current task, the current task execution duration, and resource usage.

[0014] The present invention also provides a dynamic precision adaptive calculation method for an edge AI inference processing device, applied to the dynamic precision adaptive calculation system of the edge AI inference processing device as described above, the method comprising the following steps: S1. Obtain the semantic importance information of the current inference task and the state of the runtime environment; S2. Determine the accuracy corresponding to the current inference task based on the task semantic importance information and the operating environment state; S3. Based on the precision corresponding to the current inference task, switch the computation bit width and storage access bit width corresponding to the precision within the state transition window, and execute edge AI inference.

[0015] Furthermore, the method also includes: S4, obtaining the current task execution status and updating the runtime environment status using the current task execution status.

[0016] The beneficial effects of this invention are as follows: This invention provides a dynamic precision adaptive computing system and method for edge AI inference processing devices. Through dual-dimensional perception of task semantic importance information and operating environment state, it can dynamically adapt the inference precision to real-time changes in edge scenarios. It ensures high-precision inference performance for critical tasks and complex scenarios, while using low-precision calculations for ordinary non-critical tasks to reduce power consumption and computing power overhead. This solves the technical problem of existing edge AI inference systems, which struggle to simultaneously ensure the accuracy of critical tasks and control overall power consumption in real-time changing edge scenarios. Precision switching is completed within a state transition window without interrupting the continuity of current calculations. This achieves flexible and dynamic precision switching during inference, quickly adjusting the computation bit width and storage access bit width, while avoiding additional computational latency and resource waste introduced by precision switching, further improving the resource utilization of edge AI inference devices. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a dynamic precision adaptive computing system for an edge AI inference processing device provided by the present invention; Figure 2 This is a schematic diagram of the precision reconfigurable computing array in the dynamic precision adaptive computing system of an edge AI inference processing device provided by the present invention; Figure 3This is a flowchart illustrating a dynamic precision adaptive calculation method for an edge AI inference processing device provided by the present invention. Detailed Implementation

[0018] This invention addresses the technical problem of existing edge AI inference systems, which struggle to simultaneously ensure accuracy for critical tasks and control overall power consumption in real-time, changing edge scenarios. It provides a dynamic accuracy-adaptive computing system for edge AI inference processing devices, such as... Figure 1 As shown, it includes a task and environment perception module, a dynamic accuracy decision-making module, and an accuracy reconfigurable computing array; The task and environment perception module is used to obtain the task semantic importance information and the running environment status corresponding to the current inference task. The dynamic accuracy decision module is connected to the task and environment perception module and is used to determine the accuracy corresponding to the current inference task based on the task semantic importance information and the state of the operating environment. The precision reconfigurable computing array is connected to the dynamic precision decision module and is used to switch the computing bit width and storage access bit width corresponding to the precision within the state transition window according to the precision corresponding to the current inference task, and to perform edge AI inference. The state transition window includes the time between the previous inference task and the current inference task or between adjacent computing stages within the current inference task.

[0019] In some optional implementations, the task and environment awareness module includes a task semantic analysis submodule and an environment monitoring submodule. The task semantic analysis submodule is used to extract task semantic importance information representing the processing priority of the current task based on input data features and / or edge AI intermediate layer features. The environment monitoring submodule is used to collect the operating environment status. The operating environment status includes the current task execution status, remaining battery power, chip temperature, and computing power utilization. The task semantic importance information includes at least one of the following: channel-level activation value corresponding to the current inference task, target detection confidence, image region saliency, image brightness and contrast statistical features, edge density, and event urgency.

[0020] For example, the task semantic analysis submodule connects to the input data bus and intermediate feature bus of the inference system. It can extract statistical features such as image brightness, contrast, and edge density from the input image data, and extract target detection confidence, channel-level activation value, and image region saliency information from the intermediate output features of the AI ​​inference network. Finally, it fuses these features to obtain a task semantic importance score that represents the task priority of the current processing unit. The score ranges from 0 to 1, with a higher score indicating a more critical semantic task and a higher requirement for inference accuracy. The environmental monitoring submodule connects to the power management unit, on-chip temperature sensor, and computing power scheduling unit of the edge chip. It collects real-time operating environment information such as the chip's remaining power, current core temperature, real-time computing power utilization, and current task execution status. Finally, it outputs the task semantic importance information and the operating environment status to the dynamic accuracy decision module.

[0021] In some optional implementations, the dynamic precision decision module determines the precision corresponding to the current inference task using a lookup table method or a rule-based judgment method. When determining the precision corresponding to the current inference task, if the semantic importance information of the task is inconsistent with the precision corresponding to the runtime environment state, the precision is then determined according to a preset arbitration rule, which includes priority rules, interval mapping rules, or weighted judgment rules.

[0022] For example, the dynamic precision decision module supports multiple precision levels, such as low precision, medium precision, and high precision. When making a decision, the semantic importance of the task and the state of the operating environment are mapped to obtain the corresponding candidate precision. If the two candidate precisions are consistent, the target precision is directly output. If the candidate precisions are inconsistent, the final precision is determined according to the preset priority arbitration rules.

[0023] Arbitration rules include: task semantics take precedence over environment state. That is, when the task semantics require high precision but the environment state indicates that power consumption needs to be reduced, the high-precision configuration is still retained to ensure the inference effect of critical tasks; when the task semantics require low precision and the environment state indicates that power consumption needs to be reduced, the low-precision configuration is directly confirmed to control power consumption.

[0024] The interval mapping rules are as follows: the task semantic importance score and various parameters of the operating environment are divided into different numerical intervals, and each interval corresponds to a predefined precision level. For example, the task semantic importance interval of 0~0.3 is mapped to low precision, 0.3~0.7 to medium precision, and 0.7~1.0 to high precision. The remaining battery power of the chip is also divided into intervals: more than 70% of the remaining battery power corresponds to the high precision level, 30%~70% to the medium precision level, and less than 30% to the low precision level. The core temperature is also divided into intervals: the core temperature below 60℃ allows high precision, 60℃~80℃ corresponds to medium precision, and above 80℃ it is forced to reduce to low precision. Finally, after matching the multiple parameter intervals, the target precision that meets the requirements is obtained.

[0025] The weighted judgment rule is as follows: pre-set weights are assigned to the semantic importance of the task and the state of the operating environment, and a comprehensive decision score is calculated and then mapped to the corresponding accuracy. For example, in the critical business scenario of security monitoring, a higher weight of 0.7 is assigned to the semantic importance of the task and a weight of 0.3 is assigned to the state of the operating environment to prioritize ensuring inference accuracy; in the scenario of battery-powered mobile edge devices, a higher weight of 0.6 is assigned to the state of the operating environment and a weight of 0.4 is assigned to the semantic importance of the task to prioritize controlling chip power consumption and heat generation. Finally, based on the range of the comprehensive score obtained by weighted summation, the matching inference accuracy is selected, while taking into account the accuracy of the inference task and the stability and power consumption requirements of chip operation.

[0026] In some alternative implementations, the precision-reconfigurable computing array, such as Figure 2 As shown, the array includes a data path reconstruction network, a multiplexer matrix, an arithmetic logic unit, and a storage controller. The data path reconstruction network outputs a reconstruction control signal based on the precision corresponding to the current inference task. This reconstruction control signal includes the data path connection relationship corresponding to the precision, the computation bit width corresponding to the precision, and the storage access bit width corresponding to the precision. The multiplexer matrix is ​​used to change the data path connection relationship to the precision-corresponding data path connection relationship. The arithmetic logic unit is used to change the computation bit width to the precision-corresponding computation bit width. The storage controller is used to change the storage access bit width to the precision-corresponding storage access bit width. Furthermore, the precision-reconfigurable computing array is used to feed back the current task execution status to the task and environment awareness module. The current task execution status is used to update the runtime environment status, and the current task execution status information includes at least one of the following: the precision used by the current task, the current task execution duration, and resource usage.

[0027] For example, after obtaining the target accuracy output by the dynamic accuracy decision module, the accuracy reconfigurable computing array will complete the accuracy reconstruction within a preset state transition window: the state transition window is the idle gap between the end of the previous inference task and the start of the current inference task, or the scheduling gap between adjacent convolution and fully connected computation stages within the current inference task, without interrupting the continuity of the existing computation; during the reconstruction process, the data path reconstruction network outputs reconstruction control signals to control the multiplexer matrix to switch to the data path connection relationship corresponding to the target accuracy, the arithmetic logic unit is adjusted to the computation bit width corresponding to the target accuracy, and the storage controller synchronously adjusts the storage access bit width. After the reconstruction is completed, the computation of the current inference task can be started; during the inference process, the accuracy reconfigurable computing array will also feed back the currently used accuracy, execution time, and resource usage as the task execution status to the task and environment awareness module to update the running environment status for the next accuracy decision.

[0028] This invention also provides a dynamic precision adaptive calculation method for an edge AI inference processing device, applied to the dynamic precision adaptive calculation system of the edge AI inference processing device described above. The method, as... Figure 3 As shown, it includes the following steps: S1. Obtain the semantic importance information of the current inference task and the state of the runtime environment; S2. Determine the accuracy corresponding to the current inference task based on the task semantic importance information and the operating environment state; S3. Based on the precision corresponding to the current inference task, switch the computation bit width and storage access bit width corresponding to the precision within the state transition window, and execute edge AI inference.

[0029] In some optional implementations, the method further includes: S4, obtaining the current task execution status and updating the runtime environment status using the current task execution status.

[0030] For example: When the smart security camera is in routine monitoring mode, and the captured image contains no key targets such as pedestrians or vehicles, the task semantic importance score obtained from task semantic analysis is lower than a set threshold. At the same time, the current chip temperature and remaining battery power are within the normal range. The dynamic precision decision module outputs low precision, and the precision reconfigurable computing array maintains low precision inference. The overall computing power consumption is low, and a high inference frame rate can be maintained. When a pedestrian target appears in the image, the target detection confidence exceeds the set threshold, the task semantic importance score increases, and the chip is operating normally at this time. Within the state transition window after the current inference frame ends, the dynamic precision decision module switches the precision to high precision to ensure the accuracy of subsequent face recognition and meet the recognition requirements of key events. When the system runs under high load continuously, the chip temperature rises to the overheating threshold. At this time, the image temporarily leaves the key scene, and there are no high-importance tasks. The dynamic precision decision module automatically switches back to low precision to reduce computing power consumption and heat generation. It controls the chip temperature without affecting the performance of key tasks, achieving a balance between accuracy assurance and power consumption control.

Claims

1. A dynamic precision adaptive computing system for an edge AI inference processing device, characterized in that, It includes a task and environment perception module, a dynamic accuracy decision-making module, and an accuracy reconfigurable computing array; The task and environment perception module is used to obtain the task semantic importance information and the running environment status corresponding to the current inference task. The dynamic accuracy decision module is connected to the task and environment perception module and is used to determine the accuracy corresponding to the current inference task based on the task semantic importance information and the state of the operating environment. The precision reconfigurable computing array is connected to the dynamic precision decision module and is used to switch the computing bit width and storage access bit width corresponding to the precision within the state transition window according to the precision corresponding to the current inference task, and to perform edge AI inference. The state transition window includes the time between the previous inference task and the current inference task or between adjacent computing stages within the current inference task.

2. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 1, characterized in that, The task and environment perception module includes a task semantic analysis submodule and an environment monitoring submodule; the task semantic analysis submodule is used to extract task semantic importance information representing the current task processing priority based on input data features and / or edge AI intermediate layer features; the environment monitoring submodule is used to collect the operating environment status.

3. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 2, characterized in that, The operating environment status includes the current task execution status, remaining battery power, chip temperature, and computing power utilization.

4. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 1, characterized in that, The semantic importance information of the task includes at least one of the following: channel-level activation value corresponding to the current inference task, target detection confidence, image region saliency, image brightness and contrast statistical features, edge density, and event urgency.

5. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 1, characterized in that, The dynamic precision decision module determines the precision corresponding to the current inference task by using a lookup table method or a rule-based judgment method.

6. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 5, characterized in that, When determining the precision corresponding to the current inference task, if the semantic importance information of the task is inconsistent with the precision corresponding to the state of the operating environment, the precision is determined according to the preset arbitration rules. The preset arbitration rules include priority rules, interval mapping rules, or weighted judgment rules.

7. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 1, characterized in that, The precision reconfigurable computing array includes a data path reconfiguration network, a multiplexer matrix, an arithmetic logic unit, and a storage controller. The data path reconfiguration network is used to output a reconfiguration control signal according to the precision corresponding to the current inference task. The reconfiguration control signal includes the data path connection relationship corresponding to the precision, the computation bit width corresponding to the precision, and the storage access bit width corresponding to the precision. The multiplexer matrix is ​​used to change the data path connection relationship to the data path connection relationship corresponding to the precision. The arithmetic logic unit is used to change the calculation bit width to the calculation bit width corresponding to the precision; the storage controller is used to change the storage access bit width to the storage access bit width corresponding to the precision.

8. The dynamic precision adaptive computing system of the edge AI inference processing device according to claim 1, characterized in that, The precision reconfigurable computing array is also used to feed back the current task execution status to the task and environment awareness module. The current task execution status is used to update the running environment status. The current task execution status information includes at least one of the following: the precision used by the current task, the current task execution time, and resource usage.

9. A dynamic accuracy adaptive calculation method for an edge AI inference processing device, characterized in that, The method, applied to the dynamic precision adaptive computing system of the edge AI inference processing device as described in claim 1, comprises the following steps: S1. Obtain the semantic importance information of the current inference task and the state of the runtime environment; S2. Determine the accuracy corresponding to the current inference task based on the task semantic importance information and the operating environment state; S3. Based on the precision corresponding to the current inference task, switch the computation bit width and storage access bit width corresponding to the precision within the state transition window, and execute edge AI inference.

10. The dynamic accuracy adaptive calculation method for the edge AI inference processing device according to claim 9, characterized in that, The method further includes: S4, obtaining the current task execution status and updating the runtime environment status using the current task execution status.

Images