Resource adaptive edge intelligent reasoning framework

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a resource adaptive edge intelligent reasoning framework. The method comprises the following steps: S1, accessing a plurality of input sources through an input source identification module, and intelligently identifying the formats of the input sources; s2, recoding the decoded video data into video data in a specified format according to a recognition result of the input source module; s3, according to a decoded video data result, converting the video data into an AI model reasoning format through a data preprocessing module; s4, outputting an AI model reasoning result in combination with a video data preprocessing result; and S5, converting the information into available information in combination with a service rule. The invention aims to provide a resource adaptive edge intelligent reasoning framework, so as to realize an edge AI reasoning framework with high portability, unified management capability, resource adaptability, safety and reliability, and meet the requirements of edge intelligent calculation complex scenes.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a resource-adaptive edge intelligent reasoning framework. Background Technology

[0002] With the rapid evolution of IoT and 5G technologies, edge computing, with its advantages of low latency and high real-time performance, has been widely applied in fields such as smart security, industrial inspection, smart cities, and operational safety management. In these scenarios, edge devices such as industrial cameras, smart safety helmets, and mobile inspection terminals need to process massive amounts of video and sensor data in real time, placing stringent demands on the timeliness, stability, portability, and resource adaptability of AI model inference—it must adapt to heterogeneous hardware environments, cope with dynamic resource changes such as device load and temperature fluctuations, and support efficient model updates and collaborative management of multiple devices.

[0003] Current mainstream AI inference frameworks mostly rely on interpreted runtime environments and specific hardware acceleration libraries. While they perform excellently on high-performance servers, in edge computing scenarios, due to the heterogeneity of terminal devices and limited resources, they are gradually revealing many adaptation shortcomings, making it difficult to meet the application needs of complex scenarios. The following are some of their deficiencies:

[0004] 1. Edge terminals cover multiple architectures including x86 and ARM, with significant differences in CPU and GPU models. Due to the dynamic nature of interpreted languages ​​and the environmental dependence of hardware acceleration libraries, existing frameworks require extensive low-level adaptation work during porting. For example, when migrating from x86 servers to ARM embedded devices, it is necessary not only to solve the compatibility issues of interpreted language interpreters on different platforms, but also to make a lot of low-level code adjustments for performance optimization of hardware acceleration libraries on the ARM architecture. Even so, it is difficult to ensure stable and efficient operation on various devices.

[0005] 2. In scenarios such as operational safety management, multiple devices such as cameras, smart bases, and small servers are usually required. However, the existing inference framework lacks unified adaptation capabilities, making it difficult to manage the model inference of these devices in a unified manner. This results in independent inference processes for each device, poor data interaction, high system management complexity, and limited overall performance.

[0006] 3. Interpreted languages ​​have complex dependency management, prominent version compatibility issues between hardware acceleration libraries and related libraries, and are prone to system failures when updating models, resulting in high maintenance costs. At the same time, framework deployment requires relatively complex configuration, leading to increased dependency conflicts and extremely low deployment efficiency on resource-constrained edge devices.

[0007] 4. Edge device resources fluctuate dynamically with load, temperature, and power consumption. For example, when the CPU frequency is reduced due to high temperature, the existing framework cannot automatically switch computing strategies, resulting in a sharp drop in inference performance or even interruption.

[0008] 5. When converting a trained model into an inference format, an average of 3-5 intermediate steps are required. Each step may introduce performance loss and accuracy reduction, which greatly restricts the inference efficiency of edge devices.

[0009] 6. Data transmission between devices and the cloud relies on open-source libraries, which have weak encryption mechanisms and security vulnerabilities; local data lacks access control and encrypted storage capabilities, and data leakage is easily caused by device loss.

[0010] Therefore, designing a resource-adaptive edge intelligent inference framework that can solve the above-mentioned technical problems is a technical issue that needs to be addressed. Summary of the Invention

[0011] To address the aforementioned issues, the present invention aims to provide a resource-adaptive edge intelligent inference framework that combines high portability, unified management capabilities, resource adaptability, and security and reliability to meet the needs of complex edge intelligent computing scenarios.

[0012] To achieve the above objectives, the present invention adopts the following technical solution, comprising the following steps:

[0013] Step S1: Connect to multiple input sources through the input source recognition module and intelligently recognize the format of the input sources;

[0014] Step S2: Based on the recognition results of the input source module, the decoding method is intelligently switched through the video encoding and decoding module to reduce the demand on hardware resources. At the same time, during the encoding stage, the decoded video data is re-encoded into video data of the specified format.

[0015] Step S3: Based on the decoded video data, the video data is converted into the format for AI model inference through the data preprocessing module, and the video data is normalized and color space converted during the AI ​​model inference process;

[0016] Step S4: Optimize the AI ​​model parameters using a model loader and load the optimized parameters into the AI ​​model. Combine the video data preprocessing results to output the AI ​​model inference results.

[0017] Step S5: The post-processing module receives the output results of the AI ​​model and combines them with business rules to transform them into usable information; when system resources are scarce, the output format of the results is automatically simplified to reduce the amount of data transmission.

[0018] Step S6: Encrypt the information using the data encryption module to generate the final encrypted information.

[0019] Furthermore, in step S1, multiple input sources are accessed through the input source recognition module, and the format of the input sources is intelligently recognized, as follows:

[0020] This system employs input source fingerprinting technology to identify the input source type by analyzing the statistical characteristics of the data stream. These statistical characteristics include frame rate distribution, color space features, and data packet size distribution. Even when the protocol identifier of the input source is missing or incorrect, the input source type can still be accurately identified using this fingerprinting technology. Specifically, if the input source module identifies that it uses the UVC protocol, it will automatically switch to an encoding / decoding method compatible with the UVC protocol. If the system detects multiple UVC cameras accessing the system simultaneously, this framework implements a multi-camera resource collaborative scheduling algorithm. Based on the resolution, frame rate requirements, and current system load of each camera, it dynamically allocates USB bandwidth resources to avoid frame loss due to bandwidth contention.

[0021] Furthermore, the video encoding / decoding module also includes a dynamic bitrate adaptive algorithm. This algorithm adjusts video encoding parameters in real time based on the device's current CPU load and available memory. Simultaneously, when system resources are strained, it automatically reduces video resolution and frame rate, prioritizing the preservation of image quality in critical areas, including facial areas and areas prone to dangerous behavior. After acquiring raw video data at the device end, the data is decoded according to the UVC protocol recognized by the input module.

[0022] Furthermore, in step S3, the video data is converted into the format for AI model inference through the data preprocessing module, as follows: The data preprocessing module adopts an efficient image geometric transformation algorithm and uses matrix operations to quickly adjust the size of the image to achieve basic adaptation.

[0023] Further, step S4 is as follows:

[0024] During the model loading phase, a model loader is used to optimize and store the parameters of the AI ​​model and load the optimized data into the AI ​​model to reduce memory usage.

[0025] During the inference computation phase, the model inference engine intelligently selects the corresponding hardware resources for computation based on the device's hardware resource availability.

[0026] Furthermore, in step S6, the information is encrypted by the data encryption module to generate the final encrypted information, as follows: The data encryption module adopts a key management mechanism based on a hardware security module, and uses a selective encryption strategy to perform high-strength encryption on sensitive data and lightweight encryption on ordinary background data.

[0027] The present invention has the following beneficial effects:

[0028] 1. This invention leverages the direct manipulation capabilities of compiled system-level programming languages ​​to control underlying hardware, enabling the writing of specific optimized code for different hardware architectures of heterogeneous edge terminal devices, such as ARM and x86. Taking ARM architecture devices as an example, the code can be specifically optimized using its instruction set characteristics, achieving easy portability of the inference framework to various heterogeneous devices and ensuring stable and efficient operation on different devices. This significantly expands the applicability of the inference engine, eliminating the need for companies to develop separate adapted inference frameworks for different types of smart terminal devices, thus reducing development costs and maintenance complexity.

[0029] 2. This invention constructs a general AI model inference framework based on a compiled system-level programming language, encompassing functional modules such as video encoding / decoding, data preprocessing, post-processing, data encryption, and an AI model inference engine. By standardizing the interfaces and data formats of each module, different smart terminal devices can easily access the same framework for model inference. This not only enables collaborative work between devices but also improves the overall management efficiency and stability of the system.

[0030] 3. This invention is developed using a compiled system-level programming language, which offers relatively simple dependency management and stable language features, effectively reducing failures caused by version compatibility issues. Furthermore, the framework design adopts a modular approach, with each functional module operating relatively independently. This allows each module to be updated and maintained independently without significantly impacting other modules, greatly reducing the overall framework's maintenance costs. Even with continuous updates and optimizations to the AI ​​model, the system ensures long-term stable operation of the job safety management system, minimizing system downtime and risks associated with version upgrades.

[0031] 4. This invention utilizes an inference framework developed based on a compiled system-level programming language, eliminating the need for a complex interpreter environment. The compiled executable file can run directly on the target device. Code optimization reduces the need for external dependency libraries, significantly lowering the likelihood of environment conflicts during deployment. For resource-constrained edge devices, the executable file size and resource consumption have also been optimized, enabling fast and convenient deployment.

[0032] 5. Regarding data transmission, this invention employs a high-performance data encryption and decryption module developed using a compiled system-level programming language. It incorporates a strict data access control mechanism, ensuring that only authorized programs and users can access the data. Furthermore, locally stored data is encrypted, making it difficult to illegally access even if the device is lost or stolen.

[0033] 6. This invention improves data processing speed by optimizing algorithms and adopting pipelined design, and enhances inference speed by utilizing multithreading, hardware acceleration, quantization, and caching techniques, making the entire inference framework more efficient in handling AI model inference tasks. This means that under the same hardware conditions, it can process more data faster and provide more timely and accurate inference results.

[0034] 7- This invention directly converts mainstream training framework models into inference formats optimized for specific edge devices, reducing intermediate conversion steps and automatically identifying and eliminating model redundancy through a built-in model analyzer and optimizer, significantly improving model conversion efficiency and inference performance. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0037] See Figure 1 As shown, the solution includes the following steps:

[0038] Step S1: Connect to multiple input sources through the input source recognition module and intelligently recognize the format of the input sources;

[0039] Step S2: Based on the recognition results of the input source module, the decoding method is intelligently switched through the video encoding and decoding module to reduce the demand on hardware resources. At the same time, during the encoding stage, the decoded video data is re-encoded into video data of the specified format.

[0040] Step S3: Based on the decoded video data, the video data is converted into the format for AI model inference through the data preprocessing module, and normalization and color space conversion are performed on the video data during the AI ​​model inference process; specifically, it supports common AI model formats, such as Pt format, PB format, ONNX format, etc.

[0041] Step S4: Optimize the AI ​​model parameters using a model loader and load the optimized parameters into the AI ​​model. Combine the video data preprocessing results to output the AI ​​model inference results.

[0042] Step S5: The post-processing module receives the AI ​​model's output and combines it with business rules to transform it into usable information. When system resources are strained, the output format is automatically simplified to reduce data transmission. When a high-risk event is identified, complete output is guaranteed even under resource constraints. Specifically, unlike traditional fixed-frequency post-processing, full post-processing is only performed when a potential safety event is detected; otherwise, only lightweight verification is executed. For example, in high-altitude work scenarios, detailed analysis for violations is only performed when the model initially determines that high-altitude work may exist. This reduces post-processing overhead by 65% ​​under normal operating conditions, while maintaining a 100% detection rate for critical events.

[0043] Step S6: Encrypt the information using the data encryption module to generate the final encrypted information.

[0044] Furthermore, in step S1, multiple input sources are accessed through the input source recognition module, and the format of the input sources is intelligently recognized, as follows:

[0045] This system employs input source fingerprinting technology to identify the input source type by analyzing the statistical characteristics of the data stream. These statistical characteristics include frame rate distribution, color space features, and data packet size distribution. Even when the protocol identifier of the input source is missing or incorrect, the input source type can still be accurately identified using this fingerprinting technology. Specifically, if the input source module identifies that it uses the UVC protocol, it will automatically switch to an encoding / decoding method compatible with the UVC protocol. If the system detects multiple UVC cameras accessing the system simultaneously, this framework implements a multi-camera resource collaborative scheduling algorithm. Based on the resolution, frame rate requirements, and current system load of each camera, it dynamically allocates USB bandwidth resources to avoid frame loss due to bandwidth contention.

[0046] Furthermore, the video encoding / decoding module also includes a dynamic bitrate adaptive algorithm. This algorithm adjusts video encoding parameters in real time based on the device's current CPU load and available memory. When system resources are strained, it automatically reduces video resolution and frame rate, prioritizing image quality in critical areas, including facial and dangerous behavior regions. Specifically, after acquiring raw video data at the device end, data decoding is performed according to the UVC protocol identified by the input module. Depending on the video encoding format, such as H.264 or H.265, the corresponding FFmpeg decoder is invoked. For older or resource-constrained devices, lower-complexity decoding algorithms can be used to reduce hardware resource requirements while maintaining basic video quality. The encoding stage proceeds in the opposite way, re-encoding the processed video data into a specified format for storage or transmission. When system resources are strained, the encoding quality of keyframes (I-frames) is automatically increased, while the quality of non-keyframes (P / B frames) is appropriately reduced to ensure video readability even with frame loss.

[0047] Furthermore, in step S3, the video data is converted into a format suitable for AI model inference using a data preprocessing module. Specifically, the data preprocessing module employs an efficient image geometric transformation algorithm, utilizing matrix operations to quickly adjust the image size for basic adaptation. Through preliminary lightweight analysis, key regions in the image (such as the area where workers are located) are identified, and high-quality preprocessing is applied to these regions, while simplified preprocessing is used for the background areas.

[0048] Further, step S4 is as follows:

[0049] During the model loading phase, a model loader is used to optimize and store the parameters of the AI ​​model and load the optimized data into the AI ​​model to reduce memory usage.

[0050] During the inference computation phase, the model inference engine intelligently selects the corresponding hardware resources for computation based on the device's hardware resources (such as the number of CPU cores, GPU memory size, etc.). Specifically, for CPU devices, multi-threading technology is used to allocate different computational layers of the model to different threads for parallel computation. For devices that support GPU computing, CUDA programming is used to map model computation tasks to the parallel computing units of the GPU.

[0051] Further, in step S6, the information is encrypted by the data encryption module to generate the final encrypted information, specifically as follows: The data encryption module adopts a key management mechanism based on a hardware security module and employs a selective encryption strategy, using high-strength encryption for sensitive data and lightweight encryption for ordinary background data. Specifically, when storing data, a file system-level encryption method is used. First, the entire file is divided into multiple data blocks. For each data block, an AES algorithm similar to transmission encryption is used for encryption, but to improve storage efficiency and facilitate management, the encryption key for each data block can be derived from the master key. The master key is generated from the user-set password or other security seed using a key derivation function (such as PBKDF2). When storing encrypted data, the encryption parameters of each data block (such as IV, key derivation parameters, etc.) are recorded simultaneously for use during decryption. During the data reading and decryption phase, each data block is decrypted sequentially according to the records at storage time to restore the original data.

[0052] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A resource-adaptive edge intelligent inference framework, characterized in that, Includes the following steps: Step S1: Connect to multiple input sources through the input source recognition module and intelligently recognize the format of the input sources; Step S2: Based on the recognition results of the input source module, the decoding method is intelligently switched through the video encoding and decoding module to reduce the demand on hardware resources. At the same time, during the encoding stage, the decoded video data is re-encoded into video data of the specified format. Step S3: Based on the decoded video data, the video data is converted into the format for AI model inference through the data preprocessing module, and the video data is normalized and color space converted during the AI ​​model inference process; Step S4: Optimize the AI ​​model parameters using a model loader and load the optimized parameters into the AI ​​model. Combine the video data preprocessing results to output the AI ​​model inference results. Step S5: The post-processing module receives the output of the AI ​​model and combines it with business rules to transform it into usable information; Step S6: Encrypt the information using the data encryption module to generate the final encrypted information.

2. The resource-adaptive edge intelligent inference framework according to claim 1, characterized in that, In step S1, multiple input sources are accessed through the input source recognition module, and the format of the input sources is intelligently recognized, as follows: The input source feature fingerprinting technology is used to identify the input source type by analyzing the statistical characteristics of the data stream, including frame rate distribution, color space features, and data packet size distribution.

3. The resource-adaptive edge intelligent inference framework according to claim 1, characterized in that, The video encoding and decoding module also includes a dynamic bitrate adaptive algorithm, which adjusts the video encoding parameters in real time according to the current CPU load and available memory of the device. At the same time, when system resources are scarce, it automatically reduces the video resolution and frame rate, and prioritizes maintaining the image quality of key areas, including face areas and dangerous behavior areas.

4. The resource-adaptive edge intelligent inference framework according to claim 1, characterized in that, In step S3, the video data is converted into the format for AI model inference through the data preprocessing module, as follows: The data preprocessing module adopts an efficient image geometric transformation algorithm and uses matrix operations to quickly adjust the size of the image to achieve basic adaptation.

5. The resource-adaptive edge intelligent inference framework according to claim 1, characterized in that, Step S4 is as follows: During the model loading phase, a model loader is used to optimize and store the parameters of the AI ​​model and load the optimized data into the AI ​​model to reduce memory usage. During the inference computation phase, the model inference engine intelligently selects the corresponding hardware resources for computation based on the device's hardware resource availability.

6. The resource-adaptive edge intelligent inference framework according to claim 1, characterized in that, In step S6, the information is encrypted by the data encryption module to generate the final encrypted information, as follows: The data encryption module adopts a key management mechanism based on a hardware security module and uses a selective encryption strategy to perform high-strength encryption on sensitive data and lightweight encryption on ordinary background data.

Images