An edge visual autonomous system and method based on an ultra-lightweight memory safety agent runtime

Abstract

The application relates to an edge visual autonomous system and method based on an ultralight memory safety Agent runtime, and belongs to the field of edge visual autonomous systems. The system comprises: an end side, including a camera module, a sensor array and an executor, the camera module is used for collecting original images, and the executor is used for executing a decision result of an Agent; an edge side, deployed on a resource-limited edge computing node, including an ultralight memory safety Agent runtime, and the ultralight memory safety Agent runtime is realized by using a memory safety compiling language; and a cloud end layer, used for being responsible for continuous optimization of a model.

Description

Technical Field

[0001] This invention relates to the field of edge vision autonomous systems, and more particularly to an edge vision autonomous system and method based on an ultra-lightweight memory-safe agent runtime. The "Agent" described in this invention refers to an autonomous decision-making entity based on a large language model and rule engine. As the core scheduling unit on the edge side, it is responsible for perceiving the environment, invoking the vision toolchain, retrieving memories, executing inference, and outputting control commands, possessing environmental interaction capabilities and continuous learning capabilities. Background Technology

[0002] In edge visual computing fields such as intelligent security, industrial quality inspection, and smart retail, existing technologies generally adopt a three-tier "device-edge-cloud" architecture, deploying core computing tasks on cloud servers. Taking a typical intelligent security monitoring system as an example, a typical edge visual computing system in existing technologies mainly consists of three parts: the device side, the edge side, and the cloud.

[0003] Edge side: Multiple cameras (A1, A2, A3...AN) are deployed to continuously capture raw video stream B. These cameras are typically IP cameras that transmit the video stream to the edge device via wired or wireless networks.

[0004] Edge side: Deployed with edge gateways or network video recorders (NVRs), whose functions include: Video Stream Receiver D: Simultaneously receives real-time video streams from multiple cameras.

[0005] Video Compression / E: Encodes the raw video stream using H.264 or H.265 compression to reduce the amount of data.

[0006] Compressed video upload (F): Continuously uploads the compressed video stream to the cloud server.

[0007] Keyframe extraction G: Some systems perform simple motion detection at the edge to extract video keyframes containing motion.

[0008] Keyframe Upload (H): Upload the extracted keyframe images to the cloud for further analysis.

[0009] In the cloud: high-performance cloud servers or GPU clusters are deployed to handle all core computing tasks. Video Stream Decoding J: Receives and decodes compressed video streams from the edge side.

[0010] Deep learning model inference K: running large deep learning models L (such as YOLOv4 for object detection, ResNet50 for image classification, and FaceNet for face recognition).

[0011] Result Storage N: Store the inference results (such as detected targets and identified faces) into a cloud database.

[0012] Alarm Triggering / Report Generation O: Triggers alarms according to preset rules or generates periodic analysis reports, which are eventually pushed to the user's mobile APP or monitoring center P.

[0013] The core feature of this solution is that the edge side only performs data collection and forwarding, while all core AI visual analysis tasks are completed in the cloud.

[0014] Objective drawbacks of existing technologies: The aforementioned existing technical solutions have the following obvious drawbacks in actual deployment, especially in scenarios with limited resources, complex network environments, or strict requirements for real-time performance and privacy.

[0015] Disadvantage 1: Poor real-time performance and low reliability due to strong network dependence. Because the core AI inference K is entirely performed in the cloud, the system is highly dependent on a stable, high-bandwidth network connection. The inference result M needs to travel a complete round trip "edge, cloud, edge" before reaching the user. Real-world testing data shows that the end-to-end latency under this architecture is typically between 500ms and several seconds, which cannot meet the needs of industrial real-time control (requiring <100ms) or immediate security response (such as immediate alarms for unauthorized intrusion). More seriously, once the network is interrupted, the entire system immediately fails, and edge devices become "blind boxes," unable to perform any autonomous judgment or response.

[0016] Disadvantage 2: High bandwidth and cloud costs The continuous uploading of compressed video streams from multiple high-definition cameras generates enormous network bandwidth costs. For example, a medium-sized factory (with 50 1080p cameras) could incur monthly bandwidth costs of several thousand yuan just for video uploads. Simultaneously, a massive GPU server cluster needs to be deployed in the cloud to process this data, leading to a sharp increase in hardware procurement, power consumption, and maintenance costs.

[0017] Disadvantage 3: Serious risks of data security and privacy breaches Raw video data containing highly sensitive information such as faces, license plates, and people's behavioral trajectories needs to be transmitted over the network and stored in the cloud. This introduces a triple risk: the data may be intercepted during transmission, the cloud server may be compromised, and cloud service provider personnel may illegally access the data. Given the increasingly stringent regulations such as the Personal Information Protection Law, this vulnerability has become a major reason why many enterprises (such as community property management companies and medical institutions) refuse to deploy such systems.

[0018] Disadvantage 4: Severe waste of edge computing resources Increasingly powerful edge gateways (such as smart NVRs equipped with multi-core CPUs or even NPUs) are only handling simple tasks like video forwarding and compression, leaving their built-in AI computing power completely idle, resulting in a huge waste of hardware resources. Furthermore, since all intelligent analysis resides in the cloud, upgrading algorithms requires updating cloud services, preventing the use of local computing power on edge devices for rapid iteration. Summary of the Invention

[0019] To overcome the aforementioned shortcomings of the prior art, this invention provides an edge vision autonomous system and method based on ultra-lightweight memory-safe agent runtime, which can achieve extremely low latency and network disconnection autonomy, and significantly reduce hardware costs and power consumption.

[0020] To achieve the above-mentioned objectives, the present invention provides an edge vision autonomous system based on the runtime of an ultra-lightweight memory-secure agent, the system comprising: On the edge side, there are camera modules, sensor arrays, and actuators. The camera modules are used to acquire raw images, and the actuators are used to execute the decision results of the Agent. On the edge side, it is deployed on resource-constrained edge computing nodes, including an ultra-lightweight memory-safe agent runtime, which is implemented using a memory-safe compiled language; The cloud layer is responsible for the continuous optimization of the model.

[0021] Furthermore, the ultra-lightweight memory security agent has a runtime binary size of less than 5MB and a runtime memory usage of less than 5MB, providing ample resources for visual reasoning. Its cold start time is less than 10ms, and it can quickly wake up from a dormant state to respond to events.

[0022] Furthermore, the ultra-lightweight memory security agent runtime includes: The event-driven scheduler adopts an event-driven model based on interrupts and timers, and continuously listens for events; The model manager is responsible for the dynamic loading, unloading, and version management of local quantization mini-models. Safety sandbox actuators provide an isolated environment for the execution of all tools; A vision toolchain is a series of tools that implement specific vision functions, all of which are registered to the runtime by implementing the Provider Trait; The acquisition tools can directly manipulate the camera hardware through V4L2, Video for Linux 2, or the libcamera framework; The detection tool encapsulates the ONNX Runtime inference engine, loads and runs a quantized lightweight object detection model; the quantization algorithm uses INT8 symmetric quantization, and the original FP32 model is converted to INT8 representation using the ONNX Runtime Quantization tool. The quantization calibration dataset is a subset of COCO (1000 images). After quantization, the model size is compressed from 12MB to 3.2MB, with an accuracy loss of less than 1.5%. Encapsulate the llama.cpp inference engine, load and run a quantized lightweight visual language mini-model; Hybrid memory modules enable agent intelligent learning and false alarm suppression; A structured event library, based on an embedded SQLite database, stores structured information for each event and supports fast keyword retrieval; The vector feature library integrates a lightweight local embedding model to convert the visual features and semantic descriptions of the detected target into vectors, store them, and perform cosine similarity retrieval.

[0023] The present invention provides an edge visual autonomous method based on the system described in any one of claims 1-3, comprising the following steps: Step S1: System Initialization Deploy an ultra-lightweight memory security agent runtime on edge computing nodes; Step S2: Enter event-driven sleep mode After the event-driven scheduler completes a round of tasks, it puts the system into a low-power sleep state, retaining only the necessary interrupt listeners. In this state, the power consumption drops to less than 1W, which is suitable for battery-powered devices. Step S3: Event Triggering When the camera motion detection algorithm detects a change in the scene, or the sensor array detects a change, or the timer reaches the preset time, a hardware or software interrupt is generated, waking up the scheduler. Step S4: Image Acquisition The scheduler calls the acquisition tool, which uses memory pool technology to directly obtain the latest image frame from the camera driver's DMA buffer, achieving zero copying. Step S5: Target Detection The scheduler passes image frames to the detection tool, and the output includes the detected target category, bounding box coordinates, and confidence score. Step S6: Key Target Identification The Agent judges the detection results according to preset rules. If no key target is detected, the process returns to step S2 to continue to sleep and wait. If a key target is detected, it proceeds to the next step. Step S7: Scene Semantic Description The scheduler invokes the description tool, taking the image region containing the target as input. This step provides the semantic understanding basis for the agent's decision-making. Step S8: Agent Decision Reasoning include: Receive the semantic description generated in step S7; The hybrid memory module is invoked for retrieval, and at the same time, the SQLite structured event database is queried to retrieve similar events within the past 5 minutes; Make a comprehensive judgment based on the preset strategy; Output the decision results, including whether to execute the action, the action type, and the action parameters; Step S9: Action Execution Judgment Determine whether a physical action needs to be performed based on the decision result of step S8; Step S10: Perform physical actions If an action needs to be performed, the scheduler calls the execution tool, which, based on the decision parameters, triggers the relay to close and activates the audible and visual alarm; at the same time, it sends an event summary to the monitoring center via an HTTP POST request. Step S11: Update hybrid memory Regardless of whether an action is performed, the complete information of this event is stored in the hybrid memory module, the structured data is stored in the SQLite event library, and the semantic description and visual features are converted into vectors and stored in the vector feature library for future retrieval. Step S12: Cloud-based collaborative judgment If cloud collaboration is configured, the Agent checks the network connection status. If the network is available, proceed to step S13; otherwise, skip. Step S13: Anonymous Statistical Upload and Model Update Edge nodes upload anonymous statistical data for the current period to the cloud strategy optimization engine through an encrypted channel. At the same time, they check if there are new versions of the model in the cloud model repository. If so, they perform OTA silent updates through the model manager. Step S14: Running status judgment The system checks whether a shutdown command has been received. If not, the process returns to step S2 and continues to hibernate, waiting for the next event. Otherwise, the system executes the safe shutdown procedure and terminates operation.

[0024] Compared with the prior art, the beneficial effects of the present invention are: Achieving extremely low latency and network disconnection autonomy Since all perception, cognition, and decision-making are completed at local edge nodes, the end-to-end processing latency is reduced from more than 500ms in the prior art to less than 80ms (in one embodiment, the measured value is 78ms), meeting the stringent requirements of industrial control and real-time security. More importantly, the system is completely free from dependence on the network - even if the network is interrupted, the edge nodes can still autonomously complete the entire closed-loop process of detection, description, decision-making, and execution, improving reliability from "dependence on the network" to 99.9% local availability. Significantly reduce hardware costs and power consumption Through extreme optimization of the Rust native runtime and quantized small models, the system can run smoothly on low-cost single-board computers (such as Raspberry Pi Zero 2W) (memory usage <40MB, including inference models), without the need for expensive GPU servers. Power consumption is reduced from >10W in traditional solutions to <2W (event mode), making long-term unattended deployment with solar power and battery power possible. To fundamentally protect data privacy and security All raw image data containing sensitive information such as faces and license plates are processed locally and never leave the edge device. Only anonymized statistical summaries (which do not contain any raw image information) can be uploaded to the cloud. This design fundamentally eliminates the risk of privacy leakage during data transmission and cloud storage, and perfectly complies with the requirements of regulations such as GDPR and the Personal Information Protection Act. Significantly reduce bandwidth and cloud costs There is no need to continuously upload high-definition video streams; only KB-level event summaries and screenshots (optional) are uploaded when an event occurs. Compared with existing technologies, network bandwidth consumption is reduced by more than 95%, and cloud storage and computing costs are reduced by more than 90%, which greatly improves the economics of large-scale deployment. Achieve intelligent false alarm suppression Through similarity retrieval using the hybrid memory module 40, the system can automatically learn and suppress repeated false alarms caused by environmental changes (such as light and shadow, swaying leaves). Actual test data shows that in complex outdoor environments, the false alarm rate is reduced from more than 30% in traditional solutions to less than 2%, greatly improving user experience and system reliability.

[0025] 6. Corresponding to the shortcomings of existing systems described in the background art, the beneficial effects of the present invention specifically solve the following problems: Shortcoming 1 (strong network dependence) → Achieve local full closed-loop autonomy; Shortcoming 2 (high bandwidth cost) → Reduce bandwidth consumption by more than 95%; Shortcoming 3 (privacy risk) → Raw data never leaves the edge device; Shortcoming 4 (waste of edge resources) → Fully release edge AI computing power. Attached Figure Description

[0026] Figure 1 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the invention, but not all embodiments. The embodiments of the present invention are described below with reference to the accompanying drawings.

[0028] An edge vision autonomous system based on an ultra-lightweight memory-safe agent runtime pushes core AI perception, cognition, and decision-making capabilities down to edge devices. By deploying an extremely lightweight AI agent runtime on resource-constrained edge nodes, it achieves a novel architecture of "edge-side data collection - edge autonomy - cloud-based collaboration," specifically including: On the edge side, there are camera modules, sensor arrays and actuators. The camera modules are connected via CSI or USB interfaces and are used to acquire raw images. The actuators include physical devices such as relays, motors, and alarms, which are used to execute the decision results of the Agent. On the edge side, it is deployed on resource-constrained edge computing nodes (such as single-board computers like Raspberry Pi and Jetson Nano, with optional Hailo-8L or Coral TPU accelerator 11), including an ultra-lightweight memory-safe agent runtime, implemented using a memory-safe compiled language; The cloud layer is responsible for the continuous optimization of the model.

[0029] In some embodiments, the ultra-lightweight memory security agent has a runtime binary size of less than 5MB, can be deployed in seconds via a simple cargo install command, has a runtime memory footprint of less than 5MB, provides ample resources for visual inference, has a cold start time of less than 10ms, and can quickly wake up from a dormant state to respond to events.

[0030] In some embodiments, the ultra-lightweight memory security agent runtime includes: The event-driven scheduler adopts an event-driven model based on interrupts and timers, continuously listening for events. The scheduler uses the Earliest Deadline First (EDF) algorithm for task scheduling, and its schedulability condition is determined by the formula Σ(Ci / Ti) ≤ 1, where Ci represents the actual execution time of a single visual inference task, and Ti represents the period at which the task is triggered (i.e., the maximum interval between the arrival of external events). In this system, the measured Ci for the object detection task is less than 15ms, and Ti is set to 100ms, which satisfies the schedulability condition. The model manager is responsible for the dynamic loading, unloading, and version management of local quantized small models. It maintains a model cache pool and loads models on demand according to task requirements, avoiding wasted memory due to persistent resident models. Simultaneously, it receives OTA updates from the cloud model repository. The security sandbox executor provides an isolated environment for the execution of all tools. It restricts file system access through path whitelisting, limits shell command execution through command filtering, and controls network access through network policies, ensuring that even if the agent is attacked by a malicious prompt injection, it cannot harm the underlying system. Specifically, the security sandbox executor uses the Linux seccomp-bpf mechanism to filter system calls, prohibiting the following high-risk system calls: execve, fork (unless through a whitelist), clone, socket (restricted to only established connections), and open (restricted to only whitelisted paths). Example filtering rule pseudocode is as follows: / / Allow necessary calls such as read, write, close, mmap, etc. / / Deny execve, socket, etc. BPF_JUMP(BPF_JMP + BPF_JEQ + BPF_K, __NR_execve, 0, 1); BPF_STMT(BPF_RET + BPF_K, SECCOMP_RET_ERRNO); A vision toolchain is a series of tools that implement specific vision functions, all of which are registered to the runtime by implementing the Provider Trait; The acquisition tool directly operates the camera hardware through V4L2, Video for Linux 2, or libcamera framework. Its core optimization lies in zero-copy technology - using a memory pool to pre-allocate a DMA buffer, image frames are directly written from the camera hardware to the pre-allocated memory, with zero CPU intervention, greatly reducing latency and CPU usage. The detection tool encapsulates the ONNX Runtime inference engine, loads and runs a quantized lightweight object detection model, with YOLOv8n-int8 as the preferred embodiment. The model file is less than 5MB. The tool receives image frame input and outputs the detected target bounding box, category and confidence. The inference process can achieve real-time performance of 15-30 FPS through hardware accelerators such as Hailo-8L. The llama.cpp inference engine is encapsulated, and a lightweight visual language model is loaded and run. LLaVA-Phi-3-4bit is the preferred embodiment; after 4-bit quantization, the model size is less than 2GB. After the detection tool identifies a key target, it generates a natural language semantic description of the target region or the entire image (e.g., "A person in red overalls is opening the power distribution cabinet door"), providing the agent with a language-understanding-based judgment basis for decision-making. This is crucial for achieving autonomous decision-making. Hybrid memory modules enable agent intelligent learning and false alarm suppression; A structured event library, based on an embedded SQLite database, stores structured information for each event and supports fast keyword retrieval; The vector feature library integrates a lightweight local embedding model to convert the visual features and semantic descriptions of detected targets into vectors, store them, and perform cosine similarity retrieval. When a new event occurs, the agent calculates the cosine similarity between the semantic description vector of the current event and the vectors of historical events. If the similarity is greater than 0.9 and the time interval is less than 5 minutes, it is determined to be a duplicate event. If it is determined to be a duplicate event more than 3 times consecutively, subsequent alarms are suppressed. Real-world testing shows that in complex outdoor lighting conditions, enabling this mechanism reduced the false alarm rate from 31.2% to 1.8%.

[0031] refer to Figure 1 An edge visual autonomy method based on ultra-lightweight memory-safe agent runtime includes the following steps: Step S1: System Initialization Deploying an ultra-lightweight memory-safe agent runtime on edge computing nodes is specifically implemented as follows: the runtime is installed via cargoinstall or binary copy, and after startup, the runtime occupies less than 5MB of memory. The model manager preloads the YOLOv8n-int8 model required by the detection tool into the memory cache.

[0032] Step S2: Enter event-driven sleep mode After the event-driven scheduler completes a round of tasks, it puts the system into a low-power sleep state, retaining only necessary interrupt listening (such as GPIO interrupts and network wake-up packets). In this state, the power consumption drops to less than 1W, which is suitable for battery-powered devices. Step S3: Event Triggering When the camera motion detection algorithm detects a change in the scene, or the sensor array detects a change, or the timer reaches the preset time, a hardware or software interrupt is generated, waking up the scheduler. Step S4: Image Acquisition The scheduler invokes the acquisition tool, which uses memory pool technology to directly obtain the latest image frame from the camera driver's DMA buffer, achieving zero-copy. Specifically, it requests the DMA buffer via V4L2's VIDIOC_REQBUFS, maps the buffer to user space using mmap, and then calls VIDIOC_QBUF and VIDIOC_DQBUF to repeatedly obtain the buffer index of the filled frame, directly passing this memory address to the subsequent detection tool, avoiding any memory copy operations. Specifically, it uses V4L2's VIDIOC_DQBUF operation to obtain the buffer index of the filled frame and directly maps it to user space.

[0033] Step S5: Target Detection The scheduler passes image frames to the detection tool. The detection tool loads a pre-built YOLOv8n-int8 model (3.2MB) via the ONNX Runtime C API and calls OrtSessionRun to perform inference. If a hardware accelerator is available, the inference is offloaded to the accelerator via the ONNXRuntime TensorRT execution provider, achieving an inference latency of less than 15ms; the output includes the detected target category, bounding box coordinates, and confidence score. Step S6: Key Target Identification The agent determines the detection result based on preset rules. Example rule: "If the category 'person' is detected and the confidence level is >0.6, it is determined to be a key target." If no key target is detected, the process returns to step S2 to continue waiting; if a key target is detected, it proceeds to the next step. Step S7: Scene Semantic Description The scheduler invokes the description tool, taking the image region (or the entire image) containing the target as input. The description tool loads a 4-bit quantized LLaVA-Phi-3 model (approximately 1.8GB) via llama.cpp and calls llama_eval to generate a natural language description. A typical output is: "A person wearing blue overalls is standing in front of equipment number 3 in workshop, checking the instruments." This step provides the semantic understanding foundation for the agent's decision-making. Step S8: Agent Decision Reasoning include: Receive the semantic description generated in step S7; The hybrid memory module is invoked for retrieval. Specifically, the semantic description is converted into a 384-dimensional vector using a local sentence-transformers model, and its cosine similarity is calculated with historical event vectors stored in the vector feature library 42. Simultaneously, the SQLite structured event library is queried to retrieve similar events from the past 5 minutes. A comprehensive judgment is made based on a preset strategy; example of the strategy: "If more than 3 similar events with a similarity of >0.9 occur in the past 3 minutes, it is judged as a duplicate false alarm and the alarm is suppressed; otherwise, it is judged as a real event and an alarm needs to be issued."

[0034] Output decision results, including whether to execute the action, the type of action (audio-visual alarm / notification), and action parameters; Step S9: Action Execution Judgment Determine whether a physical action needs to be performed based on the decision result of step S8; Step S10: Perform physical actions If an action needs to be performed, the scheduler calls the execution tool. Based on the decision parameters, the execution tool operates the GPIO pin through / sys / class / gpio to trigger the relay to close and start the audible and visual alarm. At the same time, it sends an event summary (including semantic description and screenshot) to the monitoring center via an HTTP POST request. Step S11: Update hybrid memory Regardless of whether an action is performed, the complete information of this event (detection results, semantic description, decision results, timestamp) is stored in the hybrid memory module, the structured data is stored in the SQLite event library, and the semantic description and visual features are converted into vectors and stored in the vector feature library for future retrieval. Step S12: Cloud-based collaborative judgment If cloud collaboration is configured, the Agent checks the network connection status. If the network is available, proceed to step S13; otherwise, skip. Step S13: Anonymous Statistical Upload and Model Update The edge nodes upload the statistical data for the current period to the cloud-based strategy optimization engine after anonymizing it. The anonymization uses an attribute removal method: removing all identifiable information such as timestamps, device serial numbers, and IP addresses, and retaining only aggregated event type counts, false alarm rate percentages, and average model inference latency (accurate to 10ms). At the same time, the edge nodes check if there are new versions of the model in the cloud model repository. If so, they perform OTA silent updates through the model manager. If the update fails, they automatically roll back to the previous stable version. Step S14: Running status judgment The system checks whether a shutdown command has been received. If not, the process returns to step S2 and continues to hibernate, waiting for the next event. Otherwise, the system executes the safe shutdown procedure and terminates operation.

[0035] The technical solution of the present invention has been described above in conjunction with specific embodiments. However, it should be noted that the above descriptions are only for explaining the solution of the present invention and should not be construed as a specific limitation on the scope of protection of the invention in any way. Based on this explanation, those skilled in the art can conceive of other specific embodiments or equivalent substitutions of the present invention without creative effort, and all such embodiments or substitutions will fall within the scope of protection of the present invention.

Claims

1. An edge vision autonomous system based on the runtime of an ultra-lightweight memory-safe agent, characterized in that, The system includes: On the edge side, there are camera modules, sensor arrays, and actuators. The camera modules are used to acquire raw images, and the actuators are used to execute the decision results of the Agent. On the edge side, deployed on resource-constrained edge computing nodes, including an ultra-lightweight memory-safe agent runtime implemented using a memory-safe compilation language; the binary size of the ultra-lightweight memory-safe agent runtime is less than 5MB, the runtime memory usage is less than 5MB, and the cold start time is less than 10ms; The cloud layer is responsible for the continuous optimization of the model.

2. The edge visual autonomous system based on ultra-lightweight memory-secure agent runtime as described in claim 1, characterized in that, The ultra-lightweight memory security agent has a runtime binary size of less than 5MB and a runtime memory usage of less than 5MB, leaving ample resources for visual reasoning. Its cold start time is less than 10ms, and it can quickly wake up from a dormant state to respond to events.

3. The edge visual autonomous system based on ultra-lightweight memory-secure agent runtime as described in claim 1, characterized in that, The ultra-lightweight memory security agent runtime includes: The event-driven scheduler adopts an event-driven model based on interrupts and timers, and continuously listens for events; The model manager is responsible for the dynamic loading, unloading, and version management of local quantization mini-models. Safety sandbox actuators provide an isolated environment for the execution of all tools; A vision toolchain is a series of tools that implement specific vision functions, all of which are registered to the runtime by implementing the Provider Trait; The acquisition tools can directly manipulate the camera hardware through V4L2, Video for Linux 2, or the libcamera framework; The detection tool encapsulates the ONNX Runtime inference engine, loads and runs a quantized lightweight object detection model; the quantization algorithm uses INT8 symmetric quantization, and the original FP32 model is converted to INT8 representation using the ONNX Runtime Quantization tool. The quantization calibration dataset is a subset of COCO (1000 images). After quantization, the model size is compressed from 12MB to 3.2MB, with an accuracy loss of less than 1.5%. Encapsulate the llama.cpp inference engine, load and run a quantized lightweight visual language mini-model; Hybrid memory modules enable agent intelligent learning and false alarm suppression; A structured event library, based on an embedded SQLite database, stores structured information for each event and supports fast keyword retrieval; The vector feature library integrates a lightweight local embedding model to convert the visual features and semantic descriptions of the detected target into vectors, store them, and perform cosine similarity retrieval.

4. An edge visual autonomous method based on the system of any one of claims 1, 2, or 3, comprising the following steps: Step S1: System Initialization Deploy an ultra-lightweight memory security agent runtime on edge computing nodes; Step S2: Enter event-driven sleep mode After the event-driven scheduler completes a round of tasks, it puts the system into a low-power sleep state, retaining only the necessary interrupt listeners. In this state, the power consumption drops to less than 1W, which is suitable for battery-powered devices. Step S3: Event Triggering When the camera motion detection algorithm detects a change in the scene, or the sensor array detects a change, or the timer reaches the preset time, a hardware or software interrupt is generated, waking up the scheduler. Step S4: Image Acquisition The scheduler calls the acquisition tool, which uses memory pool technology to directly obtain the latest image frame from the camera driver's DMA buffer, achieving zero copying. Step S5: Target Detection The scheduler passes image frames to the detection tool, and the output includes the detected target category, bounding box coordinates, and confidence score. Step S6: Key Target Identification The Agent judges the detection results according to preset rules. If no key target is detected, the process returns to step S2 to continue to sleep and wait. If a key target is detected, it proceeds to the next step. Step S7: Scene Semantic Description The scheduler invokes the description tool, taking the image region containing the target as input. This step provides the semantic understanding basis for the agent's decision-making. Step S8: Agent Decision Reasoning include: Receive the semantic description generated in step S7; The hybrid memory module is invoked for retrieval, and at the same time, the SQLite structured event database is queried to retrieve similar events within the past 5 minutes; Make a comprehensive judgment based on the preset strategy; Output the decision results, including whether to execute the action, the action type, and the action parameters; Step S9: Action Execution Judgment Determine whether a physical action needs to be performed based on the decision result of step S8; Step S10: Perform physical actions If an action needs to be performed, the scheduler calls the execution tool, which, based on the decision parameters, triggers the relay to close and activates the audible and visual alarm; at the same time, it sends an event summary to the monitoring center via an HTTP POST request. Step S11: Update hybrid memory Regardless of whether an action is performed, the complete information of this event is stored in the hybrid memory module, the structured data is stored in the SQLite event library, and the semantic description and visual features are converted into vectors and stored in the vector feature library for future retrieval. Step S12: Cloud-based collaborative judgment If cloud collaboration is configured, the Agent checks the network connection status. If the network is available, proceed to step S13; otherwise, skip. Step S13: Anonymous Statistical Upload and Model Update Edge nodes upload anonymous statistical data for the current period to the cloud strategy optimization engine through an encrypted channel. At the same time, they check if there are new versions of the model in the cloud model repository. If so, they perform OTA silent updates through the model manager. Step S14: Running status judgment The system checks whether a shutdown command has been received. If not, the process returns to step S2 and continues to hibernate, waiting for the next event. Otherwise, the system executes the safe shutdown procedure and terminates operation.

Images