Edge generation type AI adaptive reasoning method

Abstract

The invention discloses an edge generation type AI self-adaptive reasoning method which is characterized by comprising the following steps: S1, establishing a resource state dynamic data acquiring computing power load, storage resource, power consumption state and network bandwidth core resource parameters of equipment in real time through a built-in resource monitoring unit of the edge equipment, establishing a resource state dynamic database; s2, forming a task demand feature vector: receiving a generated task input by a user, analyzing a reasoning precision requirement, a real-time requirement, content complexity and task priority core demand parameters of the task, and forming the task demand feature vector; s3, adjusting a reasoning strategy of the generative AI model; s4, executing generation task reasoning; and S5, performing adaptive adjustment. According to the method, the scale of the generative AI model can be flexibly adjusted according to the resource fluctuation of the edge equipment through dynamic adaptation of real-time resource perception and the model architecture.

Description

Technical Field

[0001] This invention relates to the field of AI, and in particular to an edge-generative AI adaptive reasoning method. Background Technology

[0002] With the rapid development of edge computing technology, edge devices (such as smart terminals, industrial sensors, vehicle terminals, and IoT gateways) have been widely used in various fields such as consumer electronics, smart manufacturing, autonomous driving, and smart healthcare. Generative AI technologies (such as text generation, image generation, speech synthesis, and 3D model generation), with their powerful content creation and data processing capabilities, are gradually penetrating the edge, aiming to achieve "local data local processing," reduce data transmission latency, and protect data privacy and security.

[0003] However, existing edge generative AI inference technologies face numerous unresolved issues that severely restrict their large-scale application: The contradiction between resources and model adaptation is prominent: generative AI models (such as large language models and diffusion models) typically have a large number of parameters and high computational complexity, while edge devices generally suffer from limited computing power, scarce storage resources, and strict power consumption constraints. Existing technologies mostly adopt a "fixed model deployment" model, directly trimming and porting models trained in the cloud to edge devices. This fails to dynamically adjust according to the real-time resource status of the device, leading to either computational overload and excessive inference latency due to overly large models, or severe degradation of generation results due to excessive model trimming.

[0004] The inference strategy lacks dynamic adaptability: the resource status of edge devices (computing load, remaining storage, real-time power consumption) changes dynamically with the operating scenario (e.g., the computing load of industrial sensors surges during peak acquisition periods, and resource competition intensifies when navigation and multimedia are running simultaneously in vehicle terminals). Meanwhile, the requirements of the generated tasks (accuracy requirements, real-time requirements, content complexity) also vary (e.g., "summary generation" in text generation requires rapid response, while "creative writing" requires higher accuracy). Existing technologies employ a uniform inference strategy, which cannot be dynamically adjusted according to resource fluctuations and task requirements, leading to low inference efficiency, resource waste, or failure to meet task demands.

[0005] Poor adaptability to multimodal tasks: Current edge generative AI inference is mostly designed for single-modal tasks (such as supporting only text generation or only image generation), lacking the ability to adaptively process multimodal inputs (text + image, speech + text). When edge devices need to handle complex multimodal generation tasks, multiple independent models need to be deployed, further consuming limited resources, and the collaboration efficiency between models is low, making it impossible to achieve optimized resource allocation for cross-modal inference.

[0006] Balancing energy consumption and inference performance is challenging: Edge devices (especially mobile edge devices and battery-powered devices) are sensitive to power consumption, and existing generative AI inference technologies have not established a dynamic balance mechanism between energy consumption and performance. In high-power scenarios, failure to reduce inference complexity in a timely manner leads to a sharp drop in device battery life; in low-power scenarios, failure to fully utilize remaining resources to improve generation accuracy results in wasted performance.

[0007] Lack of closed-loop optimization mechanism: The inference process of existing technologies is "one-way execution" without real-time monitoring and feedback of inference results (generation accuracy, latency, energy consumption). It is impossible to adjust model parameters and inference strategies according to actual inference effects, resulting in inference performance remaining suboptimal for a long time and making it difficult to adapt to long-term dynamic factors such as hardware aging of edge devices and environmental changes.

[0008] Therefore, there is an urgent need for a generative AI inference method that can dynamically adapt to the resource status and task requirements of edge devices, solve problems such as resource adaptation contradictions, rigid inference strategies, poor multimodal adaptation, and energy consumption and performance imbalance in existing technologies, and improve the efficiency, stability and versatility of edge generative AI inference. Summary of the Invention

[0009] In view of the problems mentioned in the background art, the present invention aims to provide an edge-generative AI adaptive inference method to solve the problems raised in the background art.

[0010] The above-mentioned technical objective of the present invention is achieved through the following technical solution: an edge-generated AI adaptive inference method, comprising the following steps: S1, establishing a dynamic database of resource status: through the resource monitoring unit built into the edge device, the computing load, storage resources, power consumption status, network bandwidth and other core resource parameters of the device are collected in real time to establish a dynamic database of resource status.

[0011] S2. Forming a task requirement feature vector: Receive the generated task input by the user, analyze the core requirement parameters of the task, such as inference accuracy requirements, real-time requirements, content complexity, and task priority, and form a task requirement feature vector.

[0012] S3. Adjust the inference strategy of the generative AI model: Based on the real-time resource parameters and task requirement feature vectors of the dynamic resource status database, the inference strategy of the generative AI model is dynamically adjusted through an adaptive decision engine.

[0013] S4. Perform generative task inference: According to the adjusted inference strategy, load the adapted generative AI model and perform the generative task inference process.

[0014] S5. Adaptive Adjustment: Collect data on inference result quality, inference latency, actual power consumption, and resource utilization, compare and analyze them with task requirement parameters and resource status parameters, and provide feedback to optimize the adjustment strategy of the adaptive decision engine, forming a closed loop.

[0015] Preferably, in step S1, the update cycle of the dynamic resource status database is 50-100ms, and the core resource parameters include CPU / GPU utilization, computing throughput, remaining storage space, model storage usage, real-time power consumption, remaining power, uplink / downlink speed, and network latency.

[0016] Preferably, in S2, the generation task includes text generation, image generation, speech synthesis, and multimodal content generation types, and the inference accuracy requirements include text syntax accuracy, image resolution / clarity, and speech synthesis naturalness evaluation indicators.

[0017] Preferably, in S3, the inference strategy adjustment includes dynamic model architecture adaptation, inference process optimization, and multimodal task adaptation, wherein dynamic model architecture adaptation includes adaptive selection of model pruning intensity, quantization accuracy, and model distillation level.

[0018] Preferably, the inference process optimization includes adjusting the inference parallelism, inference step size, and feature extraction depth, wherein the inference parallelism supports switching between single-threaded and multi-threaded inference.

[0019] Preferably, the multimodal task adaptation includes dynamically allocating the proportion of model resources for different modalities and selecting serial or parallel inference modes.

[0020] Preferably, in step S4, the data from the resource monitoring unit is called in real time during the inference process. If a sudden change in the resource status causes the computing load to exceed a preset threshold, an emergency adjustment of the inference strategy is triggered.

[0021] Preferably, in step S5, the quality of the inference result is quantified by a preset evaluation index, including text BLEU value, image PSNR value, and speech MOS value.

[0022] Preferably, in S5, the feedback optimization includes: if the quality of the inference result does not meet the task requirements and the device has remaining resources, improving the model inference accuracy; if the inference delay or power consumption exceeds the threshold, further optimizing the model architecture; and updating the adjustment strategy parameters of the adaptive decision engine based on the feedback data.

[0023] This invention resolves resource compatibility issues and improves inference stability: through real-time resource awareness and dynamic model architecture adaptation, generative AI models can flexibly adjust their scale according to fluctuations in edge device resources, avoiding computational overload and crashes caused by excessively large models, or degradation of generation effects due to excessive model pruning. Experimental verification shows that this method enables stable inference operation on edge devices with 30%-50% fluctuations in computational load, without interruptions or stuttering, and is compatible with over 80% of mainstream edge devices (such as smart cameras, vehicle terminals, and industrial gateways).

[0024] This invention balances inference accuracy and real-time performance to optimize user experience: based on task requirements, it intelligently analyzes and optimizes the inference process, achieving a dynamic balance between prioritizing quality for high-accuracy requirements and improving speed for high-real-time requirements. For example, in text summarization (high real-time requirement) scenarios, inference latency is reduced by 30%-50%, meeting millisecond-level response requirements; in creative writing (high-accuracy requirement) scenarios, text grammatical accuracy is improved to over 95%, and content coherence is improved by 40%, taking into account the core requirements of different tasks.

[0025] This invention supports multimodal generation tasks and improves the versatility of the method: through a multimodal task adaptation mechanism, it can handle text, image, speech and multimodal combination generation tasks without deploying multiple independent models, reducing model storage usage by 30%-45%, while improving the collaborative efficiency of cross-modal inference. The resource utilization rate of multimodal generation tasks is increased by 25%-35%, making it suitable for applications in multiple scenarios such as smart office, intelligent monitoring, and in-vehicle entertainment.

[0026] This invention reduces power consumption and extends battery life of edge devices: through a dynamic balance mechanism between power consumption and performance, it adaptively reduces ineffective power consumption during inference while ensuring task requirements are met. Experimental data shows that compared to existing fixed inference methods, this method reduces power consumption for edge device inference by 25%-40% and extends the battery life of battery-powered edge devices (such as mobile terminals and portable testing devices) by 20%-30%, thus solving the power consumption constraint problem.

[0027] This invention utilizes closed-loop optimization to improve inference performance and adapt to long-term dynamic changes: through dual-dimensional feedback of inference results and resource consumption, it continuously optimizes the model adjustment strategy, enabling inference performance to gradually improve over time while adapting to long-term dynamic factors such as edge device hardware aging and environmental changes. After one month of use, inference accuracy improves by an average of 10%-15%, resource utilization improves by 15%-20%, and the accuracy of inference strategy adaptation reaches over 90%.

[0028] This invention simplifies the edge deployment process and lowers the application threshold: This method does not require manual adjustment of model parameters or replacement of model versions, and can adapt to edge devices with different configurations and different types of generation tasks. It simplifies the deployment process of generative AI models on edge devices, lowers the technical application threshold, and helps to promote the large-scale application of generative AI in edge computing scenarios. Attached Figure Description

[0029] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1

[0031] refer to Figure 1 An edge-generative AI adaptive inference method includes the following steps: S1, establishing a dynamic database of resource status: through the resource monitoring unit built into the edge device, the computing load, storage resources, power consumption status, network bandwidth and other core resource parameters of the device are collected in real time to establish a dynamic database of resource status.

[0032] S2. Forming a task requirement feature vector: Receive the generated task input by the user, analyze the core requirement parameters of the task, such as inference accuracy requirements, real-time requirements, content complexity, and task priority, and form a task requirement feature vector.

[0033] S3. Adjust the inference strategy of the generative AI model: Based on the real-time resource parameters and task requirement feature vectors of the dynamic resource status database, the inference strategy of the generative AI model is dynamically adjusted through an adaptive decision engine.

[0034] S4. Perform generative task inference: According to the adjusted inference strategy, load the adapted generative AI model and perform the generative task inference process.

[0035] S5. Adaptive Adjustment: Collect data on inference result quality, inference latency, actual power consumption, and resource utilization, compare and analyze them with task requirement parameters and resource status parameters, and provide feedback to optimize the adjustment strategy of the adaptive decision engine, forming a closed loop.

[0036] In S1, the update cycle of the dynamic database of resource status is 50-100ms. The core resource parameters include CPU / GPU utilization, computing throughput, remaining storage space, model storage usage, real-time power consumption, remaining power, uplink / downlink speed, and network latency.

[0037] In S2, the generation tasks include text generation, image generation, speech synthesis, and multimodal content generation types. The inference accuracy requirements include text grammar accuracy, image resolution / clarity, and speech synthesis naturalness evaluation indicators.

[0038] In S3, inference strategy adjustment includes dynamic model architecture adaptation, inference process optimization, and multimodal task adaptation. Dynamic model architecture adaptation includes adaptive selection of model pruning intensity, quantization accuracy, and model distillation level.

[0039] The inference process optimization includes adjusting the inference parallelism, inference step size, and feature extraction depth. The inference parallelism supports switching between single-threaded and multi-threaded inference.

[0040] Among them, multimodal task adaptation includes dynamically allocating the proportion of model resources for different modalities and selecting serial or parallel inference modes.

[0041] In S4, data from the resource monitoring unit is called in real time during the inference process. If a sudden change in the resource status causes the computing load to exceed the preset threshold, the inference strategy is adjusted urgently.

[0042] In S5, the quality of the inference results is quantified by preset evaluation indicators, including text BLEU value, image PSNR value, and speech MOS value.

[0043] In S5, feedback optimization includes: improving model inference accuracy if the inference result quality does not meet the task requirements and the device has remaining resources; further optimizing the model architecture if the inference latency or power consumption exceeds the threshold; and updating the adjustment strategy parameters of the adaptive decision engine based on feedback data.

[0044] This invention addresses the resource adaptation conflict and improves inference stability: through real-time resource awareness and dynamic model architecture adaptation, generative AI models can flexibly adjust their scale according to fluctuations in edge device resources, avoiding computational overload and crashes caused by excessively large models, or degradation of generation effects due to excessive model pruning. Experiments verify that this method enables stable inference operation on edge devices with computational load fluctuations of 30%-50%, without interruption or lag, and is compatible with over 80% of mainstream edge devices (such as smart cameras, vehicle terminals, and industrial gateways).

[0045] This invention balances inference accuracy and real-time performance while optimizing user experience: based on task requirements, it intelligently analyzes and optimizes the inference process, achieving a dynamic balance between prioritizing quality for high-accuracy requirements and improving speed for high-real-time requirements. For example, in text summarization (high real-time requirement) scenarios, inference latency is reduced by 30%-50%, meeting millisecond-level response requirements; in creative writing (high-accuracy requirement) scenarios, text grammatical accuracy is improved to over 95%, and content coherence is improved by 40%, taking into account the core requirements of different tasks.

[0046] This invention supports multimodal generation tasks and improves the versatility of the method: through a multimodal task adaptation mechanism, it can handle text, image, speech and multimodal combination generation tasks without deploying multiple independent models, reducing model storage usage by 30%-45%, while improving the collaborative efficiency of cross-modal inference. The resource utilization rate of multimodal generation tasks is increased by 25%-35%, making it suitable for applications in multiple scenarios such as smart office, intelligent monitoring, and in-vehicle entertainment.

[0047] This invention reduces power consumption and extends battery life of edge devices by employing a dynamic balance mechanism between power consumption and performance to adaptively reduce ineffective power consumption during inference while ensuring task requirements are met. Experimental data shows that compared to existing fixed inference methods, this method reduces power consumption for edge device inference by 25%-40% and extends the battery life of battery-powered edge devices (such as mobile terminals and portable testing devices) by 20%-30%, thus solving the power consumption constraint problem.

[0048] This invention features closed-loop optimization to improve inference performance and adapt to long-term dynamic changes: through dual-dimensional feedback of inference results and resource consumption, it continuously optimizes the model adjustment strategy, enabling inference performance to gradually improve over time, while adapting to long-term dynamic factors such as edge device hardware aging and environmental changes. After one month of use, inference accuracy improves by an average of 10%-15%, resource utilization improves by 15%-20%, and the accuracy of inference strategy adaptation reaches over 90%.

[0049] This invention simplifies the edge deployment process and lowers the application threshold: This method does not require manual adjustment of model parameters or replacement of model versions, and can adapt to edge devices with different configurations and different types of generation tasks. It simplifies the deployment process of generative AI models on edge devices, lowers the technical application threshold, and helps to promote the large-scale application of generative AI in edge computing scenarios.

[0050] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An edge-generative AI adaptive inference method, characterized in that: Includes the following steps: S1. Establish a dynamic database of resource status: Through the resource monitoring unit built into the edge device, collect the device's computing load, storage resources, power consumption status, network bandwidth and other core resource parameters in real time to establish a dynamic database of resource status. S2. Forming a task requirement feature vector: Receive the generated task input by the user, analyze the core requirement parameters of the task, such as inference accuracy requirements, real-time requirements, content complexity, and task priority, and form a task requirement feature vector. S3. Adjust the inference strategy of the generative AI model: Based on the real-time resource parameters and task requirement feature vectors of the dynamic resource status database, the inference strategy of the generative AI model is dynamically adjusted through an adaptive decision engine. S4. Perform generative task inference: According to the adjusted inference strategy, load the adapted generative AI model and perform the generative task inference process. S5. Adaptive Adjustment: Collect data on inference result quality, inference latency, actual power consumption, and resource utilization, compare and analyze them with task requirement parameters and resource status parameters, and provide feedback to optimize the adjustment strategy of the adaptive decision engine, forming a closed loop.

2. The edge-generative AI adaptive inference method according to claim 1, characterized in that: In step S1, the update cycle of the dynamic database of resource status is 50-100ms, and the core resource parameters include CPU / GPU utilization, computing throughput, remaining storage space, model storage usage, real-time power consumption, remaining power, uplink / downlink speed, and network latency.

3. The edge-generative AI adaptive inference method according to claim 1, characterized in that: In S2, the generation tasks include text generation, image generation, speech synthesis, and multimodal content generation types. The inference accuracy requirements include text grammar accuracy, image resolution / clarity, and speech synthesis naturalness evaluation indicators.

4. The edge-generative AI adaptive inference method according to claim 1, characterized in that: In S3, the inference strategy adjustment includes dynamic model architecture adaptation, inference process optimization, and multimodal task adaptation. Dynamic model architecture adaptation includes adaptive selection of model pruning intensity, quantization accuracy, and model distillation level.

5. The edge-generative AI adaptive inference method according to claim 1, characterized in that: The inference process optimization includes adjusting the inference parallelism, inference step size, and feature extraction depth, wherein the inference parallelism supports switching between single-threaded and multi-threaded inference.

6. The edge-generative AI adaptive inference method according to claim 1, characterized in that: The multimodal task adaptation includes dynamically allocating the proportion of model resources for different modalities and selecting serial or parallel inference modes.

7. The edge-generative AI adaptive inference method according to claim 1, characterized in that: In S4, data from the resource monitoring unit is called in real time during the inference process. If a sudden change in the resource status causes the computing load to exceed a preset threshold, an emergency adjustment of the inference strategy is triggered.

8. The edge-generative AI adaptive inference method according to claim 1, characterized in that: In S5, the quality of the inference result is quantified by preset evaluation indicators, including text BLEU value, image PSNR value, and speech MOS value.

9. The edge-generative AI adaptive inference method according to claim 1, characterized in that: In S5, the feedback optimization includes: if the quality of the inference result does not meet the task requirements and the device has remaining resources, improving the model inference accuracy; if the inference delay or power consumption exceeds the threshold, further optimizing the model architecture; and updating the adjustment strategy parameters of the adaptive decision engine based on the feedback data.

Images