An edge AI model lightweight deployment method and system for low-power devices

Abstract

The application discloses a kind of edge AI model light weight deployment methods and systems for low-power equipment, belong to the field of edge artificial intelligence model deployment and low-power embedded device resource management, to solve the problems of loading failure, memory peak overrun, inference time delay jitter, average power consumption exceeds budget and slow cold start loading when low-power edge device deploys model, the application realizes the technical effect that model continuously operates under the constraints of memory, time delay, energy consumption, temperature and cold start by target device constraint perception compilation and end-side closed-loop elastic inference mechanism.

Description

Technical Field

[0001] This invention relates to the field of edge AI model deployment and low-power embedded device resource management, and in particular to a lightweight deployment method and system for edge AI models for low-power devices. Background Technology

[0002] With the development of edge AI technology, more and more models need to be deployed in battery-powered or ultra-low-power edge devices to perform tasks such as recognition, detection, or decision-making close to the data source. Existing solutions typically compress, quantize, or prune the model in the cloud and then directly download the compressed model to the target device for execution.

[0003] However, low-power edge devices are often subject to multiple constraints such as RAM, Flash, computing power, task preemption, energy consumption, temperature rise, and deep sleep cold start. Even if the cloud-compressed model has a small parameter scale, it may still encounter problems such as model loading failure, peak memory exceeding the limit during runtime, or unstable operator scheduling due to not matching the real resource profile of the target device.

[0004] Meanwhile, existing deployment methods typically lack an edge-side closed-loop adaptive mechanism, making it difficult to dynamically adjust the model version, branch exit, input accuracy, power consumption level, and loading segment set based on conditions such as high memory level, task preemption, input complexity, power consumption, temperature, inference latency, inference energy consumption, and cold start loading time. This makes it difficult to simultaneously meet the requirements of real-time performance, energy consumption, temperature rise, and cold start.

[0005] Therefore, there is a need for a lightweight deployment method and system for edge AI models in low-power devices that can address the shortcomings of the existing technologies. Summary of the Invention

[0006] One objective of this invention is to propose a lightweight deployment method and system for edge AI models for low-power devices. Addressing the problems in existing technologies where cloud-compressed models directly deployed are difficult to match target device resource constraints, edge-side operation lacks closed-loop adaptive mechanisms, and power consumption during cold start loading and continuous inference is difficult to balance, this invention proposes a technical solution based on the target device resource profile for constraint-aware compilation and elastic inference control executed on the edge according to the operating state. This invention has the technical effects of reducing the risk of model loading failure, controlling memory peaks, reducing latency jitter, and ensuring that the model continuously meets energy consumption, temperature, and cold start constraints.

[0007] This invention provides a lightweight deployment method for edge artificial intelligence models for low-power devices, comprising:

[0008] S1. Obtain the computation graph, target device resource profile, and task execution constraints of the model to be deployed. Match the operators, tensors, and branch exits in the computation graph with the target device resource profile to generate a set of candidate model configurations.

[0009] S2. Perform constraint-aware compilation on the candidate model configuration set to generate multiple lightweight model versions. For each lightweight model version, generate a static memory reuse plan, operator placement results, flash memory segmentation layout, and tables of latency, energy consumption, and memory costs to form a deployment list.

[0010] S3. Write the deployment list to the target device. During the operation of the target device, collect peak memory usage, task preemption flag, input complexity, power consumption, temperature, inference latency, inference energy consumption and cold start loading time to generate the edge running status vector.

[0011] S4. Calculate the normalized weighted values ​​of memory margin, latency margin, energy consumption margin, temperature margin and cold start margin based on the terminal side running status vector to obtain the resource margin risk score, and generate a set of actionable actions based on the resource margin risk score and deployment list.

[0012] S5. Execute strategy control according to long window and short window. The long window controls the inference frequency, sleep mode and power consumption level. The short window selects the model version, branch exit, input precision and loading segment set from the set of available actions to ensure that the target device meets the preset memory budget, latency budget, energy consumption budget, temperature limit and cold start loading time limit during operation.

[0013] Optionally, S1 includes:

[0014] The target device resource profile includes random access memory capacity, flash memory capacity, processor instruction set, hardware acceleration unit, operating voltage level, sleep / wake duration, temperature sampling interface, and real-time task scheduling cycle.

[0015] The task execution constraints include memory budget, latency budget, energy consumption budget, temperature limit, and cold start loading time limit;

[0016] The computational complexity, tensor size, parameter size, and branch exit position of the computation graph are mapped to the target device resource profile and the task execution constraints, respectively. Configurations exceeding the random access memory capacity or flash memory capacity are eliminated to obtain the candidate model configuration set.

[0017] Optionally, S2 includes:

[0018] For each candidate model, perform weight quantization, activation quantization, channel pruning, and branch exit insertion to obtain the corresponding lightweight model version;

[0019] The static memory reuse plan is obtained by allocating reusable memory blocks to intermediate tensors based on the tensor lifetime.

[0020] The operator placement result is determined based on the operator type, tensor size, and hardware acceleration unit support table;

[0021] The flash memory address range is divided into parameter segments, resident operator segments, on-demand load operator segments, and branch exit segments to obtain the flash memory segment layout;

[0022] The latency, energy consumption, and memory cost table are calculated based on the compiler-time operator cycle estimation and the target device calibration power consumption table.

[0023] Optionally, S3 includes:

[0024] The operation monitoring data is read in units of one inference task or one scheduling cycle.

[0025] The peak memory usage is recorded as the peak reading of random access memory usage within the sampling unit.

[0026] The task preemption marker is recorded as the real-time task preemption count and preemption duration within the sampling unit;

[0027] Input complexity is denoted as a normalized combination of input data size, number of effective channels, and sparsity of preceding features;

[0028] Align the power consumption, temperature, inference latency, inference energy consumption, and cold start loading time with the corresponding budget items in the deployment list to obtain the edge-side operating state vector.

[0029] Optionally, S4 includes:

[0030] The memory margin is obtained by subtracting the peak memory usage from the memory budget, the latency margin is obtained by subtracting the inference latency from the latency budget, the energy margin is obtained by subtracting the inference energy consumption from the energy consumption budget, the temperature margin is obtained by subtracting the temperature from the temperature limit, and the cold start margin is obtained by subtracting the cold start loading time from the cold start loading time limit.

[0031] Divide each surplus by the corresponding budget item and truncate it to a preset range of values ​​to obtain the normalized surplus value.

[0032] The resource reserve risk score is obtained by multiplying each normalized value of the reserve by its corresponding weight and summing the results. The corresponding weight is determined by the priority of memory, latency, energy consumption, temperature and cold start constraints calibrated before deployment.

[0033] Furthermore, the generation of the action set includes: matching the resource surplus risk score with a preset action mapping table, and combining it with the cost table of each lightweight model version in the deployment list to generate candidate actions;

[0034] The candidate actions include switching to a lightweight model version, enabling branch exits, reducing input precision, reducing inference frequency, switching power consumption levels, extending sleep duration, adjusting the loading segment set, and reverting to the baseline model version.

[0035] For each candidate action, verify whether its predicted memory usage, predicted inference latency, predicted inference energy consumption, predicted temperature, and predicted cold start loading time meet the corresponding budget items, and write the candidate actions that meet the corresponding budget items into the set of actionable actions.

[0036] Furthermore, the adjusted loading segment set includes cold start hot segment residency control: counting the number of calls and loading time of each flash segment within the historical window;

[0037] When the number of calls to a certain flash segment is not less than the call count threshold and the loading time is not less than the loading time threshold, the flash segment is marked as a hot segment.

[0038] The set of resident segments is determined based on the remaining memory budget and the segment size of the hot segment;

[0039] Before entering sleep mode, the memory image or checksum corresponding to the resident segment set is retained, and the resident segment set is restored first after waking up to reduce the cold start loading time.

[0040] Furthermore, the fallback to the baseline model version includes: saving the baseline model version number, baseline load segment set, baseline input precision, and baseline branch exit for each task in the deployment manifest;

[0041] When the set of actionable actions is empty, or the battery level of the target device is less than the battery threshold, or the temperature is not less than the temperature threshold, the corresponding set of benchmark loading segments is called according to the benchmark model version number, and inference is performed using the benchmark input precision and the benchmark branch exit.

[0042] Once the resource remaining risk scores all meet the recovery threshold within the continuous recovery cycle, the model version, branch exit, input precision, and loading segment set are updated again according to the short window.

[0043] Optionally, S5 includes:

[0044] The length of the long window is greater than the length of the short window. The long window is input with statistical values ​​from multiple scheduling cycles to update the inference frequency, sleep mode, and power consumption level.

[0045] The short window takes the end-side running state vector of the current scheduling cycle and the set of possible actions as input to update the model version, branch exit, input precision, and loading segment set;

[0046] When the number of times the same constraint is triggered consecutively in the short window is not less than the trigger count threshold, the constraint is written into the long window input, causing the long window to adjust the sleep mode or power consumption level.

[0047] On the other hand, the present invention also provides a lightweight deployment system for edge artificial intelligence models for low-power devices, comprising:

[0048] The configuration generation module is used to obtain the computation graph of the model to be deployed, the target device resource profile, and the task execution constraints, and to generate a set of candidate model configurations;

[0049] The constraint-aware compilation module is used to generate multiple lightweight model versions, static memory reuse plans, operator placement results, flash memory segmentation layout, and tables of latency, energy consumption, and memory costs, forming a deployment list;

[0050] The edge-side status acquisition module is used to collect the peak memory usage, task preemption flag, input complexity, power consumption, temperature, inference latency, inference energy consumption, and cold start loading time of the target device during operation, and generate an edge-side running status vector.

[0051] The strategy control module is used to calculate the resource reserve risk score, generate a set of actionable actions, and select the model version, branch exit, input precision, power consumption level, and loading segment set according to long window and short window.

[0052] The segmented loading execution module is used to perform flash segmented loading, hot segment dwell, sleep wake-up recovery, and model inference according to the selection result of the strategy control module.

[0053] The beneficial effects of this invention are:

[0054] 1. By matching the computation graph of the model to be deployed with the resource profile and task execution constraints of the target device, and generating multiple lightweight model versions, static memory reuse plans, operator placement results, flash memory segmentation layout, and tables of latency, energy consumption, and memory costs, the deployment list reflects the RAM, Flash, computing power, and task constraints of the target device before it is issued, which helps to reduce the risk of model loading failure and memory peak exceeding limits.

[0055] 2. By collecting peak memory usage, task preemption flags, input complexity, power consumption, temperature, inference latency, inference energy consumption, and cold start loading time during the operation of the target device, and generating end-side running status vectors, resource margin risk scores, and action sets accordingly, the model version, branch exit, input accuracy, power consumption level, and loading segment set can be adjusted in a closed loop according to changes in device status, thereby reducing latency jitter and energy consumption over-budget risks.

[0056] 3. By introducing cold start and hot phase segmentation dwell and dual time scale strategy control of long window and short window, the long window is responsible for inference frequency, sleep mode and power consumption level, while the short window is responsible for model version, branch exit, input accuracy and loading segment set. This can take into account the requirements of real-time performance, temperature rise, energy consumption and cold start loading time in continuous inference and deep sleep wake-up scenarios. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0058] Figure 1 A flowchart of a lightweight deployment method for edge AI models for low-power devices;

[0059] Figure 2 This is a flowchart illustrating step S4 of the present invention: resource surplus risk score calculation and generation of the action set. Detailed implementation method.

[0060] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0061] refer to Figures 1-2 A lightweight deployment method for edge AI models in low-power devices includes:

[0062] S1. Obtain the computation graph, target device resource profile, and task execution constraints of the model to be deployed. Match the operators, tensors, and branch exits in the computation graph with the target device resource profile to generate a set of candidate model configurations.

[0063] S2. Perform constraint-aware compilation on the candidate model configuration set to generate multiple lightweight model versions. For each lightweight model version, generate a static memory reuse plan, operator placement results, flash memory segmentation layout, and tables of latency, energy consumption, and memory costs to form a deployment list.

[0064] S3. Write the deployment list to the target device. During the operation of the target device, collect peak memory usage, task preemption flag, input complexity, power consumption, temperature, inference latency, inference energy consumption and cold start loading time to generate the edge running status vector.

[0065] S4. Calculate the normalized weighted values ​​of memory margin, latency margin, energy consumption margin, temperature margin and cold start margin based on the terminal side running status vector to obtain the resource margin risk score, and generate a set of actionable actions based on the resource margin risk score and deployment list.

[0066] S5. Execute strategy control according to long window and short window. The long window controls the inference frequency, sleep mode and power consumption level. The short window selects the model version, branch exit, input precision and loading segment set from the set of available actions to ensure that the target device meets the preset memory budget, latency budget, energy consumption budget, temperature limit and cold start loading time limit during operation.

[0067] In this specific embodiment, S1 includes:

[0068] The edge deployment compiler reads the computation graph of the model to be deployed, the target device resource profile, and the task execution constraints. The computation graph is parsed from the intermediate representation file exported from the training end and written into the computation graph node table according to the node identifier, operator type, input tensor identifier, output tensor identifier, tensor size, parameter size, multiplication and addition times, branch exit position, and successor node set. The target device resource profile is jointly generated by the device's factory resource description and pre-deployment detection records. The task execution constraints are jointly verified by the task configuration file and the device resource profile and written into the constraint table. The above three input objects are associated with the model identifier, device serial number, and task identifier.

[0069] The edge deployment compiler first performs field cleaning and topology verification on the computation graph node table, deletes isolated operators that have no incoming edges and are not input nodes, merges equivalent transformation nodes that are only used for tensor format conversion and do not change tensor size, and maps convolution, fully connected, pooling, normalization and activation operators to operator category fields respectively; when a node is missing tensor size or parameter size, the compiler fills in the corresponding field according to the shape inference rules of upstream tensor shape and operator category. After filling in the field, if there are still inconsistent dimensions or loops in the computation graph version, it is written into the invalid configuration record and its candidate generation is stopped.

[0070] The target device resource profile table uses the device serial number, firmware version, and task identifier as a composite key. Fields include random access memory capacity, flash memory capacity, processor instruction set, hardware acceleration unit, operating voltage level, sleep / wake duration, temperature sampling interface, real-time task scheduling cycle, and system reserved area size. When the profile table does not match the current device serial number, the edge deployment compiler stops generating candidate model configuration sets and outputs missing profile records. When the profile table matches but the firmware version is different from the version reported by the target device, the compiler rereads the firmware capability description and updates the profile table version.

[0071] The task execution constraint table uses the task identifier and profile version as keys. Fields include memory budget, latency budget, energy consumption budget, temperature limit, cold start loading time limit, minimum precision flag, candidate number limit, exit revenue threshold, and matching weight record. Before generating the candidate model configuration set, the edge deployment compiler checks whether the budget unit in the constraint table and the unit in the profile table are consistent. If the units are inconsistent, they are unified to KB, ms, mJ, and degrees Celsius according to the unit conversion factor field in the constraint table before performing candidate filtering.

[0072] The edge deployment compiler establishes a device capability index based on the target device resource profile. The device capability index includes instruction set matching key, hardware acceleration unit matching key, random access memory capacity key, flash memory capacity key, and voltage level key. For each operator in the computation graph, the compiler queries the device capability index according to operator type, number of input channels, number of output channels, and tensor data type to obtain candidate execution locations and candidate data precision. Operators that do not hit the hardware acceleration unit support table are marked as processor execution, and operators that hit the table but whose tensor size exceeds the single input limit of the acceleration unit are split into multiple block execution nodes.

[0073] The edge deployment compiler builds a branch exit candidate table for the model to be deployed. The branch exit candidate table uses the computation graph node identifier as the key and records the exit input tensor size, exit output dimension, exit classification header parameter size, number of subsequent skip operators, number of subsequent skip multiply-accumulate times, and exit confidence source. The exit candidate table is generated from the forward inference log of the pre-deployment verification sample. Each sample in the log records the true label, the confidence of each exit output, and the final exit output result. Exits that do not reach the minimum precision flag are disabled in the table.

[0074] For a candidate model configuration The edge-deployed compiler calculates its static random access memory requirements. and flash memory demand ,in , , Persistent parameter buffer occupies KB. The peak activation occupancy (KB) is estimated based on the tensor lifetime. The runtime scheduling buffer occupies KB. The KB occupied by the quantization weight segment The operator code segment occupies KB. The branch exit and list header occupy KB;

[0075] Edge deployment compiler to and As a capacity filtering predicate, and These are the budgeted values ​​of the random access memory (RAM) capacity and flash memory capacity in the resource profile after deducting the system reserved area. Configurations exceeding the RAM or flash memory capacity are removed, and the exceeding item, exceeding value, profile version, and task constraint version are recorded in the filter reason field. Configurations that do not exceed the limits continue to enter the branch exit matching and resource sorting process.

[0076] After capacity filtering, the edge deployment compiler continues to match the branch exit position with the task execution constraints. For each branch exit, it records the output tensor size, coverage layer range, estimated classification confidence source, and subsequent operator skip ratio, and sets the input precision set to three categories: eight-bit integer, mixed eight-bit integer, and four-bit weighted eight-bit activation. When the branch exit is located before a layer prohibited by the task precision constraints, or when the subsequent skip ratio of the exit is less than the exit benefit threshold, the compiler deletes the candidate configuration containing the branch exit.

[0077] The edge-deployed compiler establishes unified resource matching features for operator computational load, tensor size, parameter size, and branch exit location, with operator computational load accounting for a certain percentage. Equal to configuration The total number of multiply-accumulate operations divided by the total number of multiply-accumulate operations of the original model and truncated to [0,1], tensor size percentage. Equal to configuration Peak tensor size divided by available random access memory capacity and truncated to [0,1], parameter size percentage Equal to configuration The total parameter size is divided by the available flash memory capacity and truncated to [0,1]. All three values ​​are dimensionless resource occupancy ratios.

[0078] Edge deployment compilers match resource costs Sort the candidate configurations that pass the capacity filter. The penalty for premature exit of a branch is calculated by dividing the exit's layer number by the total number of layers in the computation graph. to The matching weight records from the task constraint versions sum to 1; A smaller value indicates a higher degree of match between the candidate configuration and the target device resource profile and task execution constraints. When the number of candidate configurations exceeds the upper limit, the compiler will proceed according to... Retain the previous configuration from smallest to largest;

[0079] After the above matching, capacity filtering, branch exit filtering, resource sorting, and signature deduplication, the edge deployment compiler generates a candidate model configuration set and writes the following fields for each record in the set: configuration identifier, source computation graph version, target device profile version, task constraint version, operator placement candidate, input precision candidate, branch exit candidate, estimated random access memory requirement, estimated flash memory requirement, resource matching cost, and filtering reason field. When the candidate model configuration set is empty, the compiler writes the baseline configuration into the candidate set and marks it as pending rollback verification. When the candidate model configuration set is not empty, it saves it as the input object for S2 constraint-aware compilation to the deployment project cache.

[0080] When the candidate model configuration set is saved, a candidate generation audit record is also written. The audit record includes the number of configurations that were removed, the number of configurations that exceeded the capacity limit, the number of configurations that were unavailable at branch exits, the resource matching cost ranking threshold, the upper limit of the number of candidates, and the baseline configuration enable flag. This audit record uses the same task constraint version as the candidate model configuration set, so that S2 can locate the source of failure when a candidate model configuration fails to compile, and allows the deployment personnel to reproduce the candidate generation process of S1 without changing the order of the claim steps. The audit record also saves the verification values ​​and generation timestamps of the computation graph summary, profile summary, constraint summary, and candidate set summary.

[0081] In this specific embodiment, S2 includes:

[0082] The constraint-aware compiler reads the candidate model configuration set generated by S1 from the deployment project cache, and generates lightweight model versions item by item with the configuration identifier as the compilation key. The input record of each candidate model configuration includes quantization bit width, pruning ratio, branch exit identifier, input precision candidate, operator placement candidate, estimated random access memory requirement and estimated flash memory requirement. The compiler converts the above fields into a compilation task queue, and one task in the compilation task queue corresponds to one lightweight model version.

[0083] When the constraint-aware compiler performs weight quantization and activation quantization for each candidate model configuration, it first reads the quantization parameter table calibrated before deployment. The quantization parameter table stores the quantization bit width, scaling factor, zero point, calibration sample batch, error upper limit, and table version, using the model layer identifier and tensor type as keys. When the quantization parameter table does not match the target layer, the compiler calls the calibration samples to recalculate the scaling factor and zero point. Candidate model configurations that still exceed the error upper limit after recalculation are marked as quantization infeasible.

[0084] For convolutional and fully connected layers, the constraint-aware compiler performs channel pruning based on channel importance. Depend on calculate, To calibrate the normalized value obtained by dividing the mean absolute value of the gradient of this channel on the sample by the mean absolute value of the maximum gradient of the same layer and truncating it to [0,1], This is the normalized value obtained by dividing the mean absolute value of the channel weights by the mean absolute value of the largest weights in the same layer and truncating it to [0,1]. and From the pruning weight table and satisfying , The channel with the smallest value is deleted first;

[0085] When the constraint-aware compiler inserts a branch exit, it connects a lightweight classification head or a task output head at the specified branch exit position in the candidate model configuration. The input tensor of the output head comes from the corresponding intermediate layer, and the output fields include the branch exit identifier, output dimension, early exit confidence threshold, and subsequent skip operator range. The early exit confidence threshold is determined by the confidence distribution of the validation samples on the target task before deployment and is written into the exit threshold table. Exits below the minimum precision flag are marked as unusable and deleted from the corresponding lightweight model version.

[0086] When generating a static memory reuse plan, the constraint-aware compiler performs topological sorting on the computation graph of the lightweight model version to obtain the tensor lifetime of each intermediate tensor. The tensor lifetime includes the first write node, the last read node, the tensor size, and the alignment requirements. The compiler allocates reusable memory blocks to the intermediate tensors according to the tensor lifetimes and allocates intermediate tensors with non-overlapping lifetimes to the same reusable memory block to obtain the static memory reuse plan.

[0087] For the tensors and the first A tensor, if or Then, the two are allowed to share a memory block, where and The first The start and end node numbers of the lifecycle of each tensor, and the memory block size is the maximum value of the shared tensor size; the compiler writes the memory block identifier, start and end nodes, tensor list, block size, alignment bytes, block address offset, and tensor-to-block mapping relationship into the static memory reuse plan table, and allocates memory blocks separately for tensors that cannot be aligned or have lifecycle conflicts.

[0088] When determining the operator placement result, the constraint-aware compiler reads the hardware acceleration unit support table, which includes operator type, tensor size range, data bit width, single call overhead, processing volume per cycle, on-chip buffer requirements, and table version. For each candidate execution location of an operator, the compiler calculates the processor execution cycle and the hardware acceleration execution cycle, and includes the data transfer overhead in the acceleration path, selecting the location that satisfies the tensor size range and has the lowest total cost as the operator placement result.

[0089] The constraint-aware compiler divides the flash address range into parameter segments, resident operator segments, on-demand load operator segments, and branch exit segments, forming a flash segment layout. Each segment records the segment identifier, segment type, start address, end address, segment size, checksum, model version, whether pre-sleep resident is allowed, whether on-demand loading is supported, and recovery priority. Address allocation is performed first according to segment size from largest to smallest. When the address range is insufficient, the on-demand load operator segments are compressed first and the segments are regenerated.

[0090] The constraint-aware compiler generates latency, energy consumption, and memory cost tables based on compile-time operator cycle estimates and target device calibration power consumption tables; for lightweight model versions... Its inference delay estimation The inference energy consumption estimate is obtained by dividing each operator cycle by the effective frequency of the corresponding execution position and then adding the data transfer and segmented loading delays, in milliseconds. The result is obtained by multiplying the rated power (mW) of each execution stage by the duration (s) and summing the results, which are then written into the cost table in mJ.

[0091] The cost table uses the lightweight model version number, branch exit identifier, input precision, and loading segment set as a joint key to store latency cost, energy cost, memory cost, and cold start loading cost. The memory cost is obtained by adding the peak memory of the static memory reuse plan, the runtime scheduling buffer, and the size of the resident segment set. The cold start loading cost is obtained by summing the segment loading time calibrated by the segment loader according to the non-resident segment set. Each cost item stores the unit, calibration source, effective voltage level, and table version.

[0092] The deployment manifest is generated by the constraint-aware compiler after all feasible lightweight model versions have been compiled. The manifest fields include task identifier, device profile version, lightweight model version number, baseline model version number, baseline loading segment set, baseline input precision, baseline branch exit, static memory reuse plan summary, operator placement summary, flash segment layout summary, latency cost, energy cost, memory cost, cold start loading cost, verification summary, and action mapping key. Lightweight model versions that violate task execution constraints are retained in the diagnostic area and not written to the executable version list. The executable version list in the deployment manifest is used as the deployment object written by S3 to the target device.

[0093] Before outputting the deployment manifest, the constraint-aware compiler performs a manifest consistency check. The check includes whether the peak memory of the static memory reuse plan is consistent with the memory cost table, whether the hardware acceleration units in the operator placement results exist in the target device resource profile, whether the address ranges of the flash segment layout do not overlap, whether the baseline model version number exists in the executable version list, and whether each action mapping key can be matched in the cost table. Deployment manifests that fail the check will not be written to the target device. The compiler writes the failure field, candidate configuration identifier, and compilation task status to the diagnostic area. Only after the check passes will a manifest file be generated for S3 to write to the target device. The manifest file header contains the manifest version number, build time, compiler version, profile version, constraint version, and full check summary.

[0094] The deployment manifest also stores the edge-side parsing index. The edge-side parsing index uses the lightweight model version number and the loaded segment identifier as keys to point to the address range of the corresponding parameter segment, resident operator segment, on-demand loaded operator segment, and branch exit segment. The index record includes segment offset, segment size, checksum, whether resident is allowed, recovery priority, and mismatch handling method. After the target device is written in S3, it loads segment by segment according to the index. If the index is not hit, the baseline loaded segment set is used directly and the manifest parsing failure flag is written to the policy cache. The edge-side parsing index also stores the index generation time, index version number, applicable voltage level, and loading interface identifier, so that the target device can select a consistent segmented loading path under different power levels, and can read the segment size, segment type, recovery priority, on-demand loading flag, loading failure rollback flag, and mirror recovery flag for subsequent hot zone segment resident control.

[0095] In this specific embodiment, S3 includes:

[0096] The segmented loading execution module of the target device receives the deployment manifest generated by S2 and writes the deployment manifest, lightweight model version file, flash segment layout, cost table, action mapping key and baseline model version information into the non-volatile storage area of ​​the target device through a secure write process. After writing is completed, the device bootstrap reads the verification digest in the header of the manifest and verifies the parameter segment, resident operator segment, on-demand loading operator segment and branch exit segment segment by segment. If the verification fails, the deployment status is written to the manifest exception flag and the baseline model version is called to execute inference.

[0097] During the operation of the target device, the edge status acquisition module reads the operation monitoring data in one inference task or one scheduling cycle as the sampling unit, and writes the sampling unit number into the status vector primary key. Within a sampling unit, the module reads the random access memory usage from the memory allocator peak register, reads the number of real-time task preemptions and the preemption duration from the scheduler log, reads the input data size, number of effective channels and sparsity of the previous stage features from the input preprocessing buffer, reads the power consumption from the power management unit, reads the temperature from the temperature sampling interface, reads the inference delay from the inference timer, reads the inference energy consumption from the current integrator or power consumption estimation table, and reads the cold start loading duration from the segmented loader.

[0098] The edge-side status acquisition module records the peak memory usage as the peak reading of the random access memory usage within the sampling unit. The peak reading is obtained by the memory allocator clearing it to zero at the beginning of the sampling unit, updating the maximum usage value after each allocation and release, and latching it at the end of the sampling unit. The unit is KB. When the peak register overflows, the module sets the peak memory usage to the random access memory capacity in the resource profile and writes the overflow flag into the status vector, so that S4 handles the sampling unit according to the hard constraint risk.

[0099] The edge-side status acquisition module records the task preemption marker as the real-time task preemption count and preemption duration within the sampling unit. The real-time task preemption count is read from the scheduler preemption counter, and the preemption duration is obtained by subtracting the preemption start timestamp and the resumption execution timestamp and then summing them up, with the unit being ms. When an inference task spans multiple scheduling cycles, the module records the preemption count and preemption duration according to the scheduling cycle, and associates the corresponding sampling units with the same inference task identifier.

[0100] The edge-side status acquisition module performs alignment and anomaly handling on the acquired operation monitoring data. All timing fields are unified to ms, energy consumption fields to mJ, memory fields to KB, and temperature fields to degrees Celsius. When a temperature reading is missing, the temperature reading of the previous sampling unit is used and the missing flag is set to 1. When the inference task is preempted by a real-time task, causing a timing interruption, the module records the preemption duration separately and does not include it in the pure inference latency, so that S4 can distinguish between model cost and scheduling interference.

[0101] The input complexity is generated by the edge-side state acquisition module using a normalized combination value based on the input data size, the number of effective channels, and the sparsity of the preceding features. The calculation rule is as follows: ,in The normalized value truncated to [0,1] is the input data size of the current sampling unit divided by the baseline input size in the deployment list. The normalized value is the number of effective channels divided by the baseline number of effective channels, truncated to [0,1]. , The proportion of zero-value elements in the preceding features. This represents the effective non-zero feature density with values ​​ranging from [0,1]. , and The input complexity weights are derived from the input complexity weight table and the sum of the three is 1.

[0102] The edge-side status acquisition module aligns power consumption, temperature, inference latency, inference power consumption, and cold start loading time with the corresponding budget items in the deployment manifest to generate an edge-side running status vector. The end-side operating state vector The fields include sampling unit number, inventory version number, current model version number, current branch exit, current input precision, current loading segment set, peak memory usage, task preemption flag, input complexity, power consumption, temperature, inference latency, inference energy consumption, cold start loading time, budget alignment flag, temperature missing flag, memory overflow flag, and inventory exception flag, and are written to the circular state cache;

[0103] To ensure the continuity of the S4 read operation status, the edge status acquisition module sets up a short window cache and a long window statistical cache for the ring status buffer. The short window cache stores the edge operation status vector of the current sampling unit and the last three sampling units, while the long window statistical cache stores the average, maximum, number of over-limits, and number of consecutive triggers of each budget item in the last sixty scheduling cycles. Before the device enters sleep mode, the module writes the current list version number, the current set of loaded segments, and the verification digest of the last edge operation status vector into the low-power holding area. After waking up, if the digests are consistent, sampling continues to be added. If the digests are inconsistent, a new sampling sequence is created and a recovery flag is written into the status vector.

[0104] At the end of each sampling unit, the end-side status acquisition module outputs an end-side operating status vector. The module also notifies the policy control module, sending a notification message that includes the state vector primary key, inventory version number, sampling unit end time, and budget alignment identifier. If the deployment inventory is in an abnormal state, the module still outputs the end-side running state vector. However, by filling the current model version number with the base model version number and setting the adjustable action flag to 0, S4 can trigger a fallback path when generating an action set without using the corrupted lightweight model version.

[0105] In this specific embodiment, S4 includes:

[0106] The strategy control module reads the end-side running state vector generated by S3. The deployment list formed by S2 is checked by the list version number to see if they belong to the same deployment. When the list version numbers are inconsistent, the policy control module writes the current sampling unit into the list mismatch record and outputs only the candidate actions that fall back to the baseline model version. When the list version numbers are consistent, the module continues to read the peak memory usage, inference latency, inference energy consumption, temperature, cold start loading time, power consumption, input complexity and task preemption flag.

[0107] The strategy control module first calculates the remaining resources, including memory. Delay margin Energy consumption margin Temperature margin Cold start margin ;in , , , and These come from the corresponding budget items in the deployment list. , , , and They come from the end-side running state vectors respectively The units for each balance are KB, ms, mJ, degrees Celsius, and ms, respectively.

[0108] The strategy control module divides each surplus by the corresponding budget item and truncates it to a preset range to obtain the normalized surplus value. ,in These constraints fall into five categories: memory, latency, power consumption, temperature, and cold start. For the corresponding budget item, This represents a cutoff function that takes 0 when the value is less than 0 and 1 when the value is greater than 1; The value is a dimensionless value in the range of [0,1]. The larger the value, the more sufficient the resource margin of the corresponding constraint item. The smaller the value, the closer the corresponding constraint item is to the budget boundary or has exceeded the budget.

[0109] Resource surplus risk scoring Calculation, where , , , and These are the normalized margin values ​​for memory, latency, power consumption, temperature, and cold start constraints, respectively. , , , and The constraint priority weights were defined before deployment, and the sum of the five weights is 1. The range is [0,1]. A larger value indicates more abundant resources and lower risk, while a smaller value indicates insufficient resources and higher risk. The strategy control module will... Write the status score table;

[0110] The constraint priority table used to calculate weights and thresholds for each action is generated from device resource load test samples before deployment. The samples include running records under different input complexities, power consumption, ambient temperature, and scheduling loads. The table fields include task identifier, device profile version, constraint item, weight, mild risk scoring threshold, severe risk scoring threshold, recovery scoring threshold, effective time, and table version. The weights are obtained by normalizing the percentage of constraint violations and the severity of task failures. Before being written to the target device, the table version is replayed using a validation set. During the replay, any table version whose hard constraint violation rate exceeds the validation limit is marked as invalid and rolled back to the previous version.

[0111] The strategy control module scores risks based on resource availability. A preset action mapping table is queried to generate candidate actions. This action mapping table uses scoring range, trigger constraints, current model version number, current branch exit, current input precision, and battery range as keys, and outputs the candidate action type and action parameter range. When the score is below the severe risk threshold, candidate actions such as load reduction, extended sleep, adjustment of the load segment set, or rollback to the baseline model version are prioritized. When the score is between the mild risk threshold and the recovery score threshold, candidate actions are generated to switch to the lightweight model version, enable branch exit, or reduce input precision. If the action map is not hit, the module sets the candidate action to fall back to the baseline model version and writes a miss flag.

[0112] For candidate actions The policy control module predicts the resource consumption after performing the action based on the cost table of each lightweight model version in the deployment list, including the predicted memory consumption. The inference latency is predicted by summing the peak memory of the candidate version, the current running buffer, and the memory of the resident segment set. Add the latency cost of candidate versions to the segmented loading latency and preemption correction amount to calculate the predictive inference energy consumption. The cold start loading time is predicted by multiplying the energy consumption cost of the candidate version by the correction factor of the current power consumption level. Take the sum of the loading times of non-resident segments in the candidate loading segment set;

[0113] Predicted temperature It is obtained from the current temperature, predictive inference energy consumption, and sleep heat dissipation correction, and the calculation rule is as follows: ,in The current temperature. The value is the temperature rise coefficient, expressed in degrees Celsius per mJ, and is read from the target device's thermal calibration table according to the power consumption setting. For action The corresponding sleep duration, in milliseconds (ms). The heat dissipation coefficient is expressed in degrees Celsius per millisecond. The candidate action evaluation record is written in degrees Celsius.

[0114] The strategy control module uses hard constraint predicates to filter candidate actions, and the filtering rules are as follows: , , , and Simultaneously, candidate actions that satisfy the corresponding budget item are written into the set of feasible actions, and candidate actions that do not satisfy any budget item are written into the record of infeasible actions and marked with the violated constraint. If the candidate action evaluation record is missing any prediction field, the candidate action is considered infeasible.

[0115] When multiple candidate actions all satisfy the hard constraints, the strategy control module first calculates the predicted resource surplus risk score after executing the action and uses it as the predicted safety score. The predicted safety score is calculated according to the surplus normalization direction of claim 5, and the larger the value, the more sufficient the predicted surplus after executing the action. The strategy control module then calculates the cost of insufficient predicted surplus. The cost of insufficient predicted surplus is obtained by subtracting the predicted safety score from 1 and truncating to [0,1]. The larger the value, the higher the degree of insufficient predicted surplus. The corresponding formula is: ; then according to the value of the action Sort by size from smallest to largest, among which Penalty for loss of model accuracy For switching costs, predict insufficient margin costs, and All are cost terms, and the larger the value, the higher the cost. , and It comes from the action ranking weight table and satisfies that the sum of the three is 1; Smaller values ​​take precedence; when items are listed side-by-side, the switching cost is prioritized. The action with the smallest value;

[0116] The action ranking weight table uses task identifier, device profile version, battery range, and temperature range as keys. Fields include risk weight, accuracy loss weight, switching cost weight, tie-in rules, effective time, and table version. Table records are generated by statistics from pre-deployment replay logs. The replay logs save the state vector before the action, the resource remaining risk score after the action, the change in task accuracy, and the switching time. When the current battery range is not hit, the conservative record with the highest risk weight under the same task identifier is read, and the miss flag is written into the set of actionable actions.

[0117] When candidate actions include adjusting the loaded segment set, the policy control module performs cold start hot segment residency control; the module counts the number of times each flash segment is called within the historical window. and cumulative loading time When the number of calls to a flash segment is not less than the call count threshold and the loading time is not less than the loading time threshold, the flash segment is marked as a hot segment. The call count threshold and the loading time threshold are derived from the segment residency threshold table, and the table fields include task identifier, segment type, call count threshold, loading time threshold, history window length, and table version.

[0118] The policy control module determines the set of resident segments based on the remaining memory budget and the segment size of hot segments. The selection rule is to add segments to the resident segment set according to the recovery priority of hot segments and the segment size from smallest to largest, until the total size of the resident segment set reaches the remaining memory budget. Before entering sleep mode, the module retains the memory image or verification digest corresponding to the resident segment set. After waking up, the module prioritizes the recovery of the resident segment set and writes the recovery success flag, recovery time, and unrecovered segment identifier into the candidate action evaluation record for updating the predicted cold start loading time.

[0119] When the action set is empty, or the target device's battery level is less than the battery threshold, or the temperature is not less than the temperature threshold, the strategy control module generates an action record to fall back to the baseline model version. The baseline model version number, baseline loading segment set, baseline input precision, and baseline branch exit are from the baseline field of the deployment list, and the battery threshold and temperature threshold are from the device protection threshold table. The fallback action replaces the current model version number with the baseline model version number and replaces the current loading segment set with the baseline loading segment set.

[0120] Resource reserve risk assessment during continuous recovery cycle When all values ​​are not less than the recovery score threshold, the strategy control module releases the rollback lock and allows S5 to re-update the model version, branch exit, input precision, and loaded segment set according to the short window; the recovery score threshold and the number of consecutive recovery cycles are derived from the constraint priority table, and the recovery cycle count is determined when any hard constraint is violated again or The score is reset to zero when it falls below the mild risk score threshold, thus avoiding frequent switching of the lightweight model version when the device has insufficient resources.

[0121] The policy control module ultimately outputs a set of actionable actions, which includes action identifier, action type, target model version number, target branch exit, target input precision, target inference frequency, target power consumption level, target sleep duration, target loading segment set, predicted memory usage, predicted inference latency, predicted inference energy consumption, predicted temperature, predicted cold start loading duration, action cost, violated constraint items, and rollback lock flag. This set is written to the policy cache and carries the current edge-side running state vector primary key to the S5 dual-window policy controller.

[0122] In this specific embodiment, S5 includes:

[0123] The dual-window policy controller reads the action set output by S4, the short window cache and the long window statistics cache maintained by S3, and executes policy control with the scheduling cycle as the control beat. In this specific embodiment, the short window length is set to four scheduling cycles, and the long window length is set to sixty scheduling cycles. The length of the long window is greater than the length of the short window. The short window is used to handle the rapid fluctuations of current memory, latency, power consumption, temperature and cold start loading time constraints. The long window is used to handle the impact of power loss, temperature accumulation and task load trends on inference frequency, sleep mode and power consumption level.

[0124] The short window controller reads the current end-side running state vector in each scheduling cycle. And a set of possible actions, and organize the action fields into short window controls. The short window control quantity This includes the target model version number, target branch exit, target input precision, and target loading segment set; the short window controller removes actions that violate the rollback lock flag and actions with mismatched target loading segment set verification summaries from the set of actionable actions, and assigns the remaining actions based on their action cost. Sort by size from smallest to largest, select The smallest action is taken as the action for the current scheduling cycle.

[0125] When the short window controller executes the selected action, it first sends the target loading segment set to the segmented loading execution module. The segmented loading execution module verifies the target segment summary according to the flash segment layout and loads the missing segments. Then, it writes the target model version number, target branch exit, and target input precision to the inference runtime. When the target loading segment set fails to load or the verification summary is inconsistent after loading, the short window controller does not enable the action. Instead, it calls the next action in the set of available actions. When all actions fail, it executes the fallback to the baseline model version action generated by S4 and writes the failure reason.

[0126] The long window controller updates the inference frequency, sleep mode, and power consumption level using statistical values ​​from multiple scheduling cycles as input. These statistical values ​​include the average resource reserve risk score within the long window, the maximum temperature, the average energy consumption, the power consumption rate, the average number of real-time task preemptions, and the maximum cold start loading time. The average resource reserve risk score within the long window follows the reserve normalization direction of claim 5; a higher score indicates more sufficient resource reserve within the long window. The long window controller first calculates the long window reserve insufficiency pressure term, which is obtained by subtracting the average resource reserve risk score within the long window from 1. A higher value indicates higher long window reserve insufficiency pressure, as shown in the formula below. Then, the long window control evaluation value is calculated. ,in This is the normalized value truncated to [0,1] after dividing the average energy consumption by the energy consumption budget. This is the normalized value truncated to [0,1] after dividing the maximum temperature by the upper temperature limit. The normalized value is the average number of preemptions divided by the upper limit of the scheduling period, truncated to [0,1]. The four weights are from the long window weight table and their sum is 1.

[0127] When long window control evaluation value When the load is not less than the long window descent threshold, the long window controller will reduce the inference frequency by one level, switch the sleep mode to deep sleep or extend the sleep duration, and reduce the power consumption level to the adjacent low power consumption level allowed by the deployment list; when When the number of consecutive recovery cycles is not greater than the long window recovery threshold and reaches the recovery cycle threshold, the long window controller will gradually restore the inference frequency, sleep mode, and power consumption level to the default state allowed by the task constraints; when the long window de-load threshold is greater than the long window recovery threshold, the two form a hysteresis interval, making... When the current inference frequency, sleep mode, and power consumption level are between two thresholds, the controller retains the current sleep mode and only adjusts the inference frequency and power consumption level. If the unload or recovery command causes the cold start loading time to exceed the upper limit of the cold start loading time, the controller retains the current sleep mode and only adjusts the inference frequency and power consumption level.

[0128] When the number of times the same constraint is triggered consecutively in the short window is not less than the trigger count threshold, the dual-window strategy controller writes the constraint into the long window input, causing the long window to adjust its sleep mode or power consumption level. The number of triggers is counted from the short window trigger records, and the trigger count threshold comes from the window control threshold table. The table fields include task identifier, device profile version, constraint, trigger count threshold, long window load reduction threshold, long window recovery threshold, recovery cycle threshold, lookup key, and table version. When the current constraint is not matched, the conservative record with the smallest trigger count threshold under the same task identifier is read.

[0129] The dual-window strategy controller arbitrates the consistency of long-window and short-window outputs. The arbitration rules are: hard constraints take precedence over minimum cost, rollback locking takes precedence over version switching, and temperature protection takes precedence over inference frequency recovery. If the long window requires a reduction in power consumption level while the short window requires switching to a high-performance model version, the controller retains the low-power level of the long window and reselects an action that satisfies the power consumption level in the short window. If the long window allows recovery while the short window still has the same constraint term triggered continuously, the controller delays recovery and maintains the current sleep mode and power consumption level.

[0130] After arbitration, the dual-window strategy controller sends power level commands and sleep mode commands to the power management unit, inference frequency commands to the inference scheduler, target loading segment set commands to the segmented loading execution module, and target model version number, target branch exit, and target input precision to the inference runtime. Each command carries the primary key of the end-side running status vector, action identifier, and list version number. The execution module returns a success flag, a failure flag, or a rollback flag. The controller writes the returned results to the strategy execution log as feedback input for the next scheduling cycle S3 acquisition and S4 action generation.

[0131] During operation, the dual-window policy controller continuously verifies whether the peak memory usage, inference latency, inference energy consumption, temperature, and cold start loading time after execution meet the preset memory budget, latency budget, energy consumption budget, temperature limit, and cold start loading time limit, respectively. If any hard constraint is still violated after execution, the controller immediately freezes the current non-baseline action and triggers a rollback to the base model version. When the resource margin risk score is not less than the recovery score threshold within the continuous recovery cycle and there is no loading failure flag in the execution log, the controller unfreezes and reselects short-window actions from the set of available actions. The policy execution log saves long-window inputs, short-window actions, arbitration results, execution return flags, and budget verification results according to the scheduling cycle, and serves as a sample source for updating the window control threshold table and action mapping table before the next deployment. This ensures that the target device maintains a closed-loop consistency between the lightweight model deployment results and the resource budget during operation.

[0132] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0133] This invention establishes a correspondence between the model computation graph, device resource profile, deployment list, and runtime status feedback by using a target device constraint-aware compilation and edge-side closed-loop elastic reasoning mechanism. This enables model deployment to no longer rely solely on offline compression results, but to form a continuous technical control link around memory budget, latency budget, energy consumption budget, temperature limit, and cold start loading time limit.

[0134] This invention introduces resource margin risk scoring, cold start and hot phased residence, and dual time scale strategy control into the edge-side policy controller, which changes the edge-side inference control structure and loading segment management method, so that short-term state fluctuations and long-term energy consumption temperature rise trends are handled separately, thereby better realizing the lightweight deployment and stable operation of models on low-power devices.

Claims

1. A lightweight deployment method for edge artificial intelligence models for low-power devices, characterized in that, include: S1. Obtain the computation graph, target device resource profile, and task execution constraints of the model to be deployed. Match the operators, tensors, and branch exits in the computation graph with the target device resource profile to generate a set of candidate model configurations. S2. Perform constraint-aware compilation on the candidate model configuration set to generate multiple lightweight model versions. For each lightweight model version, generate a static memory reuse plan, operator placement results, flash memory segmentation layout, and tables of latency, energy consumption, and memory costs to form a deployment list. S3. Write the deployment list to the target device. During the operation of the target device, collect peak memory usage, task preemption flag, input complexity, power consumption, temperature, inference latency, inference energy consumption and cold start loading time to generate the edge running status vector. S4. Calculate the normalized weighted values ​​of memory margin, latency margin, energy consumption margin, temperature margin and cold start margin based on the terminal side running status vector to obtain the resource margin risk score, and generate a set of actionable actions based on the resource margin risk score and deployment list. S5. Execute strategy control according to long window and short window. The long window controls the inference frequency, sleep mode and power consumption level. The short window selects the model version, branch exit, input precision and loading segment set from the set of available actions to ensure that the target device meets the preset memory budget, latency budget, energy consumption budget, temperature limit and cold start loading time limit during operation.

2. The lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 1, characterized in that, S1 includes: The target device resource profile includes random access memory capacity, flash memory capacity, processor instruction set, hardware acceleration unit, operating voltage level, sleep / wake duration, temperature sampling interface, and real-time task scheduling cycle; the task operation constraints include memory budget, latency budget, energy consumption budget, temperature limit, and cold start loading time limit. The computational complexity, tensor size, parameter size, and branch exit position of the computation graph are mapped to the target device resource profile and the task execution constraints, respectively. Configurations exceeding the random access memory capacity or flash memory capacity are eliminated to obtain the candidate model configuration set.

3. The lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 1, characterized in that, S2 includes: For each candidate model, perform weight quantization, activation quantization, channel pruning, and branch exit insertion to obtain the corresponding lightweight model version; The static memory reuse plan is obtained by allocating reusable memory blocks to intermediate tensors based on tensor lifecycles; the operator placement result is determined based on operator type, tensor size, and hardware acceleration unit support table; the flash memory address range is divided according to parameter segment, resident operator segment, on-demand loaded operator segment, and branch exit segment to obtain the flash memory segment layout; the latency, energy consumption, and memory cost table are calculated based on compile-time operator cycle estimation and target device calibration power consumption table.

4. The lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 1, characterized in that, S3 includes: reading operation monitoring data in sampling units of one inference task or one scheduling cycle; recording the peak memory usage as the peak reading of random access memory usage within the sampling unit; recording the task preemption flag as the number of real-time task preemptions and the duration of preemption within the sampling unit; recording the input complexity as a normalized combination of the input data size, the number of effective channels, and the sparsity of the preceding features; and aligning the power consumption, temperature, inference latency, inference energy consumption, and cold start loading duration with the corresponding budget items in the deployment list to obtain the edge-side operation state vector.

5. A lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 1, characterized in that, S4 includes: The memory margin is obtained by subtracting the peak memory usage from the memory budget, the latency margin by subtracting the inference latency from the latency budget, the energy margin by subtracting the inference energy consumption from the energy consumption budget, the temperature margin by subtracting the temperature limit from the temperature limit, and the cold start margin by subtracting the cold start loading time limit from the cold start loading time limit. Each margin is divided by the corresponding budget item and truncated to a preset value range to obtain a normalized margin value. Each normalized margin value is multiplied by its corresponding weight and summed to obtain the resource margin risk score, wherein the corresponding weight is determined by the memory, latency, energy consumption, temperature, and cold start constraint priority specified before deployment.

6. A lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 5, characterized in that, The generation of the action set includes: matching the resource margin risk score with a preset action mapping table, and combining it with the cost table of each lightweight model version in the deployment list to generate candidate actions; the candidate actions include switching lightweight model versions, enabling branch exits, reducing input precision, reducing inference frequency, switching power consumption levels, extending sleep duration, adjusting loading segment sets, and rolling back to the baseline model version; for each candidate action, verifying whether its predicted memory usage, predicted inference latency, predicted inference energy consumption, predicted temperature, and predicted cold start loading duration meet the corresponding budget items, and writing the candidate actions that meet the corresponding budget items into the action set.

7. A lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 6, characterized in that, The adjusted loading segment set includes cold start hot segment residency control: counting the number of calls and loading time of each flash segment within the historical window; when the number of calls of a flash segment is not less than the number of calls threshold and the loading time is not less than the loading time threshold, the flash segment is marked as a hot segment; determining the residency segment set based on the remaining memory budget and the segment size of the hot segment; retaining the memory image or checksum corresponding to the residency segment set before entering sleep mode, and prioritizing the restoration of the residency segment set after waking up to reduce cold start loading time.

8. A lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 6, characterized in that, S5 includes: The length of the long window is greater than the length of the short window. The long window takes statistical values ​​from multiple scheduling cycles as input and updates the inference frequency, sleep mode, and power consumption level. The short window takes the edge running state vector of the current scheduling cycle and the set of possible actions as input and updates the model version, branch exit, input precision, and loading segment set. When the number of times the short window triggers the same constraint term is not less than the trigger count threshold, the constraint term is written into the long window input, causing the long window to adjust the sleep mode or power consumption level.

9. A lightweight deployment method for edge artificial intelligence models for low-power devices according to claim 6, characterized in that, The rollback to the baseline model version includes: saving the baseline model version number, baseline loading segment set, baseline input precision, and baseline branch exit for each task in the deployment manifest; when the actionable set is empty, or the target device's battery level is less than the battery threshold, or the temperature is not less than the temperature threshold, calling the corresponding baseline loading segment set according to the baseline model version number, and performing inference using the baseline input precision and the baseline branch exit; after the resource remaining risk score meets the recovery threshold within the continuous recovery cycle, updating the model version, branch exit, input precision, and loading segment set again according to the short window.

10. A lightweight deployment system for edge AI models for low-power devices, used to execute the lightweight deployment method for edge AI models for low-power devices as described in any one of claims 1 to 9, characterized in that, include: The configuration generation module is used to obtain the computation graph of the model to be deployed, the target device resource profile, and the task execution constraints, and to generate a set of candidate model configurations; The constraint-aware compilation module is used to generate multiple lightweight model versions, static memory reuse plans, operator placement results, flash memory segmentation layout, and tables of latency, energy consumption, and memory costs, forming a deployment list; The edge-side status acquisition module is used to collect the peak memory usage, task preemption flag, input complexity, power consumption, temperature, inference latency, inference energy consumption, and cold start loading time of the target device during operation, and generate an edge-side running status vector. The strategy control module is used to calculate the resource reserve risk score, generate a set of actionable actions, and select the model version, branch exit, input precision, power consumption level, and loading segment set according to long window and short window. The segmented loading execution module is used to perform flash segmented loading, hot segment dwell, sleep wake-up recovery, and model inference according to the selection result of the strategy control module.

Images