AI algorithm lightweight loading and cross-layer resource dynamic scheduling method and system

Abstract

The invention provides an AI algorithm lightweight loading and cross-layer resource dynamic scheduling method and system, and the method comprises the steps: carrying out the model preprocessing of an original AI model at a central cloud side, generating a lightweight model, and meeting the volume compression ratio and precision loss constraints; resource states of all levels in the end side device, the edge computing node and the central holder are collected in real time, and the resource states comprise computing power, memory, network bandwidth and task load information; acquiring a current analysis task, and based on the resource state and the task demand of the current analysis task, calling the resource scheduling optimization model to solve the distribution proportion of an optimal task by adopting a genetic algorithm so as to minimize the processing time delay; and issuing the lightweight model to edge side and end side equipment, and allocating the current analysis task to three layers of end side, edge and cloud for execution according to the allocation proportion. The method has the effects of lightweight algorithm loading, efficient resource scheduling, high dynamic adaptability, high platform universality and low deployment cost.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for lightweight loading of AI algorithms and dynamic scheduling of cross-layer resources. Background Technology

[0002] In large-scale, heterogeneous IoT systems, massive amounts of data generated by numerous devices from multiple sources urgently require efficient and intelligent processing. With the deep application of artificial intelligence (AI) technology in video surveillance, AI algorithms based on deep learning, such as object detection, behavior recognition, and anomaly analysis, have become core capabilities of modern intelligent security systems. However, in a typical "edge-cloud" three-tier architecture, the deployment and resource scheduling of AI algorithms still face severe challenges, significantly limiting the system's real-time performance, scalability, and energy efficiency.

[0003] Current AI algorithms suffer from large file sizes and high edge loading latency. Mainstream AI models (such as YOLOv5, Faster R-CNN, and ST-GCN behavior recognition networks) have massive parameter counts, with model files typically ranging from 200-500MB. Edge devices (such as embedded edge computing gateways and smart cameras), limited by embedded hardware platforms, usually only have 2-4 TOPS of computing power and limited memory (4-8GB), making it difficult to directly load the full model. Real-world testing shows that a 350MB YOLOv5 model can take 12-15 seconds to load on a typical edge microserver, far exceeding the "fast response" requirements of intelligent monitoring systems (typically <3 seconds). Furthermore, frequent model loading and switching keep the CPU and memory resources of edge devices under high load for extended periods, impacting the execution of other real-time tasks.

[0004] Traditional scheduling methods are mostly "one-time allocations," lacking monitoring and feedback on the task execution process. For example, when the edge side experiences computing power shortages due to sudden tasks, the system cannot promptly migrate some tasks to the central cloud; conversely, when the central cloud load decreases, it cannot proactively decentralize tasks to the edge, leading to resource mismatch.

[0005] Although some manufacturers have launched their own edge intelligence platforms, their scheduling mechanisms are still limited to specific hardware ecosystems and lack universality. Existing technologies mostly focus on optimizing a single aspect (such as only pruning models or only unloading tasks), failing to systematically coordinate model lightweighting with cross-layer resource scheduling, resulting in limited optimization effects.

[0006] In summary, existing technologies have significant shortcomings in terms of AI model deployment efficiency, resource scheduling coordination, system dynamic adaptability, and platform versatility. There is an urgent need for a universal collaborative scheduling method that integrates lightweight models, cross-layer resource awareness, dynamic task allocation, and closed-loop control to support the sustainable development of diverse intelligent video surveillance systems. Summary of the Invention

[0007] Based on this, the purpose of this invention is to provide a method and system for lightweight loading and dynamic scheduling of cross-layer resources for AI algorithms, so as to at least solve the shortcomings of the above-mentioned technologies.

[0008] This invention proposes a lightweight loading and cross-layer resource dynamic scheduling method for AI algorithms, including: The original AI model is preprocessed on the central cloud side to generate a corresponding lightweight model that meets the constraints of volume compression rate and accuracy loss. Real-time acquisition of resource status at each level in edge devices, edge computing nodes, and central cloud platform, wherein the resource status includes computing power, memory, network bandwidth, and task load information; The current analysis task is obtained, and based on the resource status and the task requirements of the current analysis task, the resource scheduling optimization model is invoked to use a genetic algorithm to solve the optimal task allocation ratio in order to minimize processing latency. The lightweight model is distributed to edge and endpoint devices, and the current analysis task is allocated to the endpoint, edge, and cloud for execution according to the allocation ratio.

[0009] Furthermore, the steps of preprocessing the original AI model on the central cloud side to generate a corresponding lightweight model that meets the constraints of volume compression ratio and accuracy loss include: Based on the weighted importance score, the original AI model is structurally pruned on the central cloud side to remove redundant convolution channels; The original AI model after structured pruning is quantized and knowledge distilled to output a corresponding lightweight model, wherein the lightweight model satisfies: ; In the formula, The original AI model, The pruning rate, As a quantification factor, the model accuracy loss does not exceed 3% of the original AI model, and the loading time of the edge device is ≤3 seconds.

[0010] Furthermore, the resource scheduling optimization model satisfies the following constraints: Normalization constraints: and , In the formula, This indicates the allocation ratio of tasks across edge devices, edge computing nodes, and the central gimbal; Resource utilization constraints: End side: In the formula, This indicates the total computing power requirement for the current analysis task. This indicates the current available computing power of the edge device. This indicates the upper limit of resource utilization for edge devices; Edge side: In the formula, This indicates the current available computing power of the edge devices. This indicates the upper limit of resource utilization on the edge side; Central Cloud: , This indicates the available computing power of the central cloud. This indicates the upper limit of central cloud resource utilization; Total computing power coverage constraints: ; Objective function: Minimize parallel processing latency .

[0011] Furthermore, the method also includes: Continuously monitor the execution effect and resource changes of the current analysis task; When resource overload or task delay bidding is detected, resource reallocation and task migration are triggered, forming a closed-loop adaptive scheduling.

[0012] Furthermore, when resource overload or task delay bidding is detected, the steps to trigger resource reallocation and task migration, forming a closed-loop adaptive scheduling mechanism, include: When the resource utilization rate of any layer continuously exceeds the threshold More than 5 seconds, or end-to-end delay of the task When this happens, a dynamic adjustment mechanism is triggered; The scheduling engine re-invokes the resource scheduling optimization model for real-time re-optimization, updates the task allocation ratio, and performs task migration or service instance elastic scaling to form a closed-loop adaptive scheduling.

[0013] This invention also proposes a lightweight loading and cross-layer resource dynamic scheduling system for AI algorithms, comprising: The model generation module is used to preprocess the original AI model on the central cloud side to generate a corresponding lightweight model that meets the constraints of volume compression rate and accuracy loss. The resource acquisition module is used to collect the resource status of each level in the terminal devices, edge computing nodes and central cloud platform in real time. The resource status includes computing power, memory, network bandwidth and task load information. The task processing module is used to obtain the current analysis task and, based on the resource status and the task requirements of the current analysis task, call the resource scheduling optimization model to use a genetic algorithm to solve the optimal task allocation ratio in order to minimize processing latency. The task allocation module is used to distribute the lightweight model to edge and end devices, and allocate the current analysis task to the three layers of end, edge and cloud for execution according to the allocation ratio.

[0014] Furthermore, the model generation module is specifically used for: Based on the weighted importance score, the original AI model is structurally pruned on the central cloud side to remove redundant convolution channels; The original AI model after structured pruning is quantized and knowledge distilled to output a corresponding lightweight model, wherein the lightweight model satisfies: ; In the formula, The original AI model, The pruning rate, As a quantification factor, the model accuracy loss does not exceed 3% of the original AI model, and the loading time of the edge device is ≤3 seconds.

[0015] Furthermore, the system also includes a resource scheduling module, which is specifically used for: Continuously monitor the execution effect and resource changes of the current analysis task; When resource overload or task delay bidding is detected, resource reallocation and task migration are triggered, forming a closed-loop adaptive scheduling.

[0016] Furthermore, the resource scheduling module also includes: When the resource utilization rate of any layer continuously exceeds the threshold More than 5 seconds, or end-to-end delay of the task When this happens, a dynamic adjustment mechanism is triggered; The scheduling engine re-invokes the resource scheduling optimization model for real-time re-optimization, updates the task allocation ratio, and performs task migration or service instance elastic scaling to form a closed-loop adaptive scheduling.

[0017] The present invention also proposes a storage medium on which a computer program is stored, which, when executed by a processor, implements the above-mentioned method for lightweight loading of AI algorithms and dynamic scheduling of cross-layer resources.

[0018] The present invention also proposes a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for lightweight loading and dynamic scheduling of cross-layer resources for AI algorithms.

[0019] The lightweight loading and cross-layer resource dynamic scheduling method and system for AI algorithms in this invention have the following beneficial effects: 1. Lightweight AI algorithm loading: Model size is reduced by 60%–75%, and edge loading time is shortened to 2–3 seconds, significantly improving the rapid adaptability of heavy-load applications.

[0020] 2. Efficient resource scheduling: The difference in resource utilization between the end, edge, and cloud layers is reduced to less than 10%, and the overall task processing latency is reduced by 30%-40%, meeting the real-time requirements of massive video analysis.

[0021] 3. Strong dynamic adaptability of the system: Through a closed-loop feedback mechanism, the system can automatically respond to load fluctuations and avoid resource waste or overload.

[0022] 4. Strong platform versatility: It does not depend on a specific vendor's platform and can be integrated into various video surveillance systems through standard interfaces, making it widely applicable.

[0023] 5. Low deployment cost: Improves edge resource utilization, reduces reliance on central cloud, and lowers hardware investment and maintenance costs. Attached Figure Description

[0024] Figure 1 This is a flowchart of the AI ​​algorithm lightweight loading and cross-layer resource dynamic scheduling method in the first embodiment of the present invention; Figure 2 This is a block diagram of the lightweight loading and cross-layer resource dynamic scheduling method for AI algorithms in the first embodiment of the present invention; Figure 3 This is a structural block diagram of the AI ​​algorithm lightweight loading and cross-layer resource dynamic scheduling system in the second embodiment of the present invention; Figure 4 This is a structural block diagram of the computer in the third embodiment of the present invention.

[0025] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0026] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0028] Example 1 Please see Figure 1 The figure shows a lightweight loading and cross-layer resource dynamic scheduling method for AI algorithms in the first embodiment of the present invention. The method specifically includes steps S101 to S105: S101 performs model preprocessing on the original AI model on the central cloud side to generate a corresponding lightweight model that meets the constraints of volume compression rate and accuracy loss. Furthermore, step S101 specifically includes steps S1011 to S1012: S1011 performs structured pruning on the original AI model on the central cloud side based on weight importance scores to remove redundant convolution channels; S1012, quantize and distill the original AI model after structured pruning to output a corresponding lightweight model, wherein the lightweight model satisfies: ; In the formula, The original AI model, The pruning rate, As a quantification factor, the model accuracy loss does not exceed 3% of the original AI model, and the loading time of the edge device is ≤3 seconds.

[0029] Please see Figure 2 The diagram shown is a block diagram of lightweight loading of AI algorithms and dynamic scheduling of cross-layer resources in this embodiment.

[0030] In practice, the central cloud platform activates the model optimization module to perform the following operations on the target AI model (such as the YOLOv5 object detection model): 1. Structured pruning: Based on weight importance scores (such as L1 norm), redundant convolutional channels are removed, and the pruning rate is [not specified]. The sensitivity is adaptively set based on the model layer sensitivity. 2. Quantization Compression: Convert the 32-bit floating-point weights to 8-bit integers (INT8) for calculation. ; 3. Accuracy Verification: Test the accuracy of the lightweight model on the standard validation set to ensure accuracy. ; 4. Output a lightweight model (Volume ≤ 100MB).

[0031] Specifically, let the volume of the original AI algorithm model be... (Unit: MB), model pruning rate (Automatically assessed based on the importance of each layer's weights, with higher pruning rates for non-critical layers), quantization bit-width compression factor is: (Defined as the ratio of the quantized bit width to the original bit width, such as τ=8 / 32=0.25 when 32-bit floating point is 8-bit integer), then the volume of the lightweight model is... for: ; At the same time, the model inference accuracy loss constraint must be met: ; in, This represents the accuracy of the original model on the validation set. To improve the accuracy of lightweight models, This is the threshold for the maximum permissible accuracy loss.

[0032] Understandably, the pruning rate With quantification factor The coupling is performed in a multiplicative manner to accurately reflect the superposition effect of dual compression, and precision constraints are introduced to ensure the controllability and reliability of the lightweighting process.

[0033] S102, Real-time collection of resource status at each level in end-side devices, edge computing nodes and central cloud platform, wherein the resource status includes computing power, memory, network bandwidth and task load information; In practice, monitoring agents deployed on edge devices, edge computing nodes, and the central cloud platform are used to collect real-time information on computing power, memory, network bandwidth, and task load at each level.

[0034] Specifically, the resource monitoring module collects data in real time: 1. Edge: Camera computing power Current load; 2. Edge side: Edge gateway + microserver total Memory usage; 3. Central Cloud: Total computing power of the GPU cluster Queue length; 4. Current task: Number of video streams, analysis type, total computing power requirement .

[0035] The data sampling period is 1 second, and it is reported through standard protocols (such as MQTT, gRPC).

[0036] S103, obtain the current analysis task, and based on the resource status and the task requirements of the current analysis task, call the resource scheduling optimization model to use a genetic algorithm to solve the optimal task allocation ratio in order to minimize processing latency; In practical implementation, to achieve efficient scheduling of intelligent video tasks under the edge-cloud collaborative architecture, a cross-layer resource scheduling optimization model is proposed. This model takes minimizing the overall system processing latency as its core objective, comprehensively considering the computing power constraints, stability requirements, and parallel execution characteristics of tasks at each level of the device, ensuring that the scheduling strategy meets both real-time requirements and guarantees long-term stable system operation.

[0037] Specifically, the scheduling engine collects real-time data on edge computing power, central cloud computing power, and terminal computing power. (Unit: TOPS) and the total computing power requirements of the current intelligent analysis task. , (unit: TOPS), are passed as input parameters to the cross-layer resource scheduling optimization model.

[0038] The genetic algorithm is used to solve this cross-layer resource scheduling optimization model: 1. Encoding method: Real number encoding is used, with each individual representing a set of task allocation ratios ( The allocation ratio of tasks on the edge, device, and central cloud. ); 2. Fitness function: defined as follows ,in, For target latency, Penalties for violating the constraints This is the penalty coefficient; 3. Constraint handling: Constraints are incorporated into the fitness function using the penalty function method; 4. Parameter settings: Population size is 50, maximum number of iterations is 100 generations, crossover probability is 0.8, and mutation probability is 0.1.

[0039] The solution process is completed in the edge collaborative scheduler, with a single solution taking less than 150ms, meeting the system's real-time requirements. The final output is the optimal allocation ratio that satisfies all constraints. 、 It is used to guide the scheduling and execution of subsequent tasks.

[0040] In this embodiment, the task allocation ratio must meet the normalization condition: ; To ensure the stable operation of equipment at all levels and prevent equipment overheating or downtime due to resource overload, a resource utilization upper limit constraint is set: End side: Preferred ; Edge side: Preferred ; Central Cloud: Preferred ; At the same time, the total computing power of the system must meet the task requirements, that is, meet the total computing power coverage constraint: ; The goal is to minimize the overall system processing latency. Since the task is executed in parallel across the edge, cloud, and endpoint layers, the overall latency is determined by the layer with the longest execution time. Therefore, we define: ; The edge-cloud cross-layer resource scheduling optimization problem constructed in this embodiment is as follows: ; The following conditions (st) must be met: ; ; ; ; ; ; This optimization problem can also be solved using intelligent optimization algorithms such as particle swarm optimization (PSO) or interior point methods, ensuring that the optimal allocation ratio is obtained while satisfying all constraints. and achieve The goal is real-time performance.

[0041] Understandably, using "maximizing parallel latency" as the objective function truly reflects the physical characteristics of parallel task execution in the edge-cloud collaborative system, avoiding misjudgments by the traditional summation model; and introducing hierarchical differentiated resource utilization constraints ( This reflects the higher stability requirements of edge devices; the total computing power coverage constraint ensures scheduling feasibility and prevents task failure due to insufficient local computing power; the model structure is clear, the solvability is strong, and it has good engineering practicality and promotion value.

[0042] S104, the lightweight model is distributed to the edge and end devices, and the current analysis task is allocated to the three layers of end, edge and cloud for execution according to the allocation ratio.

[0043] In practical implementation, the central cloud will Distribute tasks to edge and endpoint devices; each device's model loading time is ≤3 seconds; schedule tasks according to the allocation ratio: 1. Endpoint: perform lightweight tasks such as face capture and license plate recognition; 2. Edge: perform vehicle behavior analysis and crowd density statistics; 3. Central cloud: perform cross-camera trajectory tracking and historical video retrospective analysis.

[0044] In some alternative embodiments, the method further includes: Continuously monitor the execution effect and resource changes of the current analysis task; When resource overload or task delay bidding is detected, resource reallocation and task migration are triggered, forming a closed-loop adaptive scheduling.

[0045] Furthermore, when resource overload or task delay bidding is detected, the steps to trigger resource reallocation and task migration, forming a closed-loop adaptive scheduling mechanism, include: When the resource utilization rate of any layer continuously exceeds the threshold More than 5 seconds, or end-to-end delay of the task When this happens, a dynamic adjustment mechanism is triggered; The scheduling engine re-invokes the resource scheduling optimization model for real-time re-optimization, updates the task allocation ratio, and performs task migration or service instance elastic scaling to form a closed-loop adaptive scheduling.

[0046] In practice, continuous monitoring of computing power utilization at each layer is essential. 1. If the edge utilization rate is <60% (idle rate >20%), then gradually increase... Migrate some cloud tasks to the edge; 2. If the length of the central cloud queue is greater than the threshold, then reduce... Improve edge and end-side load; 3. The adjustment cycle is 10 seconds, forming a closed-loop optimization.

[0047] In summary, the lightweight loading and cross-layer resource dynamic scheduling method for AI algorithms in the above embodiments of the present invention has the following beneficial effects: 1. Lightweight AI algorithm loading: Model size is reduced by 60%–75%, and edge loading time is shortened to 2–3 seconds, significantly improving the rapid adaptability of heavy-load applications.

[0048] 2. Efficient resource scheduling: The difference in resource utilization between the end, edge, and cloud layers is reduced to less than 10%, and the overall task processing latency is reduced by 30%-40%, meeting the real-time requirements of massive video analysis.

[0049] 3. Strong dynamic adaptability of the system: Through a closed-loop feedback mechanism, the system can automatically respond to load fluctuations and avoid resource waste or overload.

[0050] 4. Strong platform versatility: It does not depend on a specific vendor's platform and can be integrated into various video surveillance systems through standard interfaces, making it widely applicable.

[0051] 5. Low deployment cost: Improves edge resource utilization, reduces reliance on central cloud, and lowers hardware investment and maintenance costs.

[0052] Example 2 In another aspect, this invention proposes a lightweight loading and cross-layer resource dynamic scheduling system for AI algorithms. Please refer to [link / reference]. Figure 3 The figure shows a lightweight loading and cross-layer resource dynamic scheduling system for AI algorithms according to a second embodiment of the present invention. The system includes: The model generation module 11 is used to preprocess the original AI model on the central cloud side to generate a corresponding lightweight model that meets the constraints of volume compression rate and accuracy loss. The resource acquisition module 12 is used to collect the resource status of each level in the terminal device, edge computing node and central cloud platform in real time. The resource status includes computing power, memory, network bandwidth and task load information. The task processing module 13 is used to obtain the current analysis task, and based on the resource status and the task requirements of the current analysis task, call the resource scheduling optimization model to use a genetic algorithm to solve the optimal task allocation ratio in order to minimize processing latency. The task allocation module 14 is used to distribute the lightweight model to edge and end devices, and allocate the current analysis task to the three layers of end, edge and cloud for execution according to the allocation ratio.

[0053] Furthermore, the model generation module 11 is specifically used for: Based on the weighted importance score, the original AI model is structurally pruned on the central cloud side to remove redundant convolution channels; The original AI model after structured pruning is quantized and knowledge distilled to output a corresponding lightweight model, wherein the lightweight model satisfies: ; In the formula, The original AI model, The pruning rate, As a quantification factor, the model accuracy loss does not exceed 3% of the original AI model, and the loading time of the edge device is ≤3 seconds.

[0054] Furthermore, the system also includes a resource scheduling module, which is specifically used for: Continuously monitor the execution effect and resource changes of the current analysis task; When resource overload or task delay bidding is detected, resource reallocation and task migration are triggered, forming a closed-loop adaptive scheduling.

[0055] Furthermore, the resource scheduling module also includes: When the resource utilization rate of any layer continuously exceeds the threshold More than 5 seconds, or end-to-end delay of the task When this happens, a dynamic adjustment mechanism is triggered; The scheduling engine re-invokes the resource scheduling optimization model for real-time re-optimization, updates the task allocation ratio, and performs task migration or service instance elastic scaling to form a closed-loop adaptive scheduling.

[0056] The functions or operation steps implemented by the above modules and units are largely the same as those in the above method embodiments, and will not be repeated here.

[0057] The AI ​​algorithm lightweight loading and cross-layer resource dynamic scheduling system provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0058] Example 3 This invention also proposes a computer, please refer to [link / reference]. Figure 4 The computer shown in the third embodiment of the present invention includes a memory 10, a processor 20, and a computer program 30 stored on the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, it implements the above-mentioned AI algorithm lightweight loading and cross-layer resource dynamic scheduling method.

[0059] The memory 10 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10 can be an internal storage unit of a computer, such as the computer's hard disk. In other embodiments, the memory 10 can be an external storage device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 10 can include both internal and external storage units of the computer. The memory 10 can be used not only to store application software and various types of data installed on the computer, but also to temporarily store data that has been output or will be output.

[0060] In some embodiments, the processor 20 may be an electronic control unit (ECU), a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program code stored in the memory 10 or process data, such as executing access restriction programs.

[0061] It should be pointed out that, Figure 4 The structure shown does not constitute a limitation on the computer. In other embodiments, the computer may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0062] This invention also proposes a readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for lightweight loading of AI algorithms and dynamic scheduling of cross-layer resources.

[0063] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0064] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0065] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0066] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An AI algorithm lightweight loading and cross-layer resource dynamic scheduling method, characterized in that, The method comprises the following steps: model preprocessing is performed on the original AI model at the center cloud side to generate a corresponding lightweight model, so as to meet the volume compression rate and accuracy loss constraint; real-time collection of resource states of each level in the terminal side device, the edge computing node and the center cloud station, wherein the resource states comprise computing power, memory, network bandwidth and task load information; obtaining a current analysis task, and based on the resource states and task requirements of the current analysis task, calling a resource scheduling optimization model to solve an optimal task distribution ratio by using a genetic algorithm, so as to minimize the processing delay; distributing the lightweight model to the edge side and the terminal side device, and distributing the current analysis task to the terminal side, the edge and the cloud end according to the distribution ratio. 2.The AI algorithm lightweight loading and cross-layer resource dynamic scheduling method of claim 1, wherein, The step of performing model preprocessing on the original AI model at the center cloud side to generate a corresponding lightweight model to meet the volume compression rate and accuracy loss constraint comprises: performing structural pruning on the original AI model at the center cloud side based on weight importance scores to remove redundant convolution channels; quantizing and knowledge distilling the original AI model processed by the structural pruning to output a corresponding lightweight model, wherein the lightweight model meets: ； In the formula, is the original AI model, is the pruning rate, is the quantization factor, the model accuracy loss is not more than 3% of the original AI model, and the end-side device loading time is ≤3 seconds. 3.The AI algorithm lightweight loading and cross-layer resource dynamic scheduling method of claim 1, wherein, The resource scheduling optimization model meets the following constraint conditions: Normalization constraints: and , , wherein, denotes the distribution ratio of the task at the end-side device, the edge computing node, and the central cloud platform. resource utilization constraint: End side: , wherein, represents the total computing power demand of the current analysis task, represents the currently available computing power of the end side device, represents the upper limit of the resource utilization rate of the end side device; Edge side: , wherein, represents the current available computing power of the edge side device, represents the upper limit of the edge side resource utilization rate; Central cloud: , represents the central cloud available computing power, represents the central cloud resource utilization upper limit; Total computational effort coverage constraint: ; Objective function: Minimize parallel processing latency . 4.The AI algorithm lightweight loading and cross-layer resource dynamic scheduling method of claim 1, wherein, The method further comprises: continuously monitoring the execution effect of the current analysis task and the resource change; when resource overload or task delay is detected, triggering resource reallocation and task migration to form a closed-loop adaptive scheduling. 5.The AI algorithm lightweight loading and cross-layer resource dynamic scheduling method of claim 4, wherein, The step of triggering resource reallocation and task migration to form a closed-loop adaptive scheduling when resource overload or task delay is detected comprises: When any tier resource utilization continues to exceed the threshold for more than 5 seconds, or task end-to-end latency a dynamic adjustment mechanism is triggered; the scheduling engine re-calls the resource scheduling optimization model for real-time re-optimization, updates the task distribution ratio, and performs task migration or service instance elastic scaling to form a closed-loop adaptive scheduling.

6. An AI algorithm lightweight loading and cross-layer resource dynamic scheduling system, characterized in that, The method comprises the following steps: a model generation module is configured to perform model preprocessing on the original AI model at the center cloud side to generate a corresponding lightweight model to meet the volume compression rate and accuracy loss constraint; a resource collection module is configured to collect resource states of each level in the terminal side device, the edge computing node and the center cloud station in real time, wherein the resource states comprise computing power, memory, network bandwidth and task load information; a task processing module is configured to obtain a current analysis task, and based on the resource states and task requirements of the current analysis task, call a resource scheduling optimization model to solve an optimal task distribution ratio by using a genetic algorithm, so as to minimize the processing delay; a task distribution module is configured to distribute the lightweight model to the edge side and the terminal side device, and distribute the current analysis task to the terminal side, the edge and the cloud end according to the distribution ratio. 7.The AI algorithm lightweight loading and cross-layer resource dynamic scheduling system of claim 6, wherein, The model generation module is specifically configured to: perform structural pruning on the original AI model at the center cloud side based on weight importance scores to remove redundant convolution channels; quantize and knowledge distill the original AI model processed by the structural pruning to output a corresponding lightweight model, wherein the lightweight model meets: ； In the formula, is the original AI model, is the pruning rate, is the quantization factor, the model accuracy loss is not more than 3% of the original AI model, and the end-side device loading time is ≤3 seconds. 8.The AI algorithm lightweight loading and cross-layer resource dynamic scheduling system of claim 6, wherein, The system further comprises a resource scheduling module, and the resource scheduling module is specifically configured to: continuously monitor the execution effect and resource change of the current analysis task; trigger resource reallocation and task migration to form a closed-loop adaptive scheduling when resource overload or task delay bidding is detected.

9. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to realize the AI algorithm lightweight loading and cross-layer resource dynamic scheduling method of any one of claims 1-5.

10. A computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the AI algorithm lightweight loading and cross-layer resource dynamic scheduling method of any one of claims 1-5.

Images