Edge-based query processing systems and methods

Abstract

Capturing and processing edge data, like video feeds, enables a hierarchy of analysis, optimizing resource usage and enhancing responsiveness. Edge devices process data using computer vision (CV) models, identifying objects and events in real-time. In embodiments described herein, edge devices and intermediate devices can further refine the data, leveraging CV model output and large language models (LLMs) for contextual understanding. Queries can be posed to the system which may span content and context, with users exploring specific objects, events, or patterns detected in the video data. The processing described herein can provide deeper insights by analyzing textual descriptions, facilitating queries about temporal, spatial, or semantic context. This approach empowers diverse applications, from surveillance to smart city initiatives, by enabling users to extract actionable information tailored to their needs. Ultimately, processing edge data can increase responsiveness and reduce the amount of data needed to transmit to centralized devices for additional processing.

Description

Attorney Docket No. 67023.72WO01EDGE-BASED QUERY PROCESSING SYSTEMS AND METHODS CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 750,742 filed Januaiy 28, 2025 and entitled “EDGE-BASED QUERY PROCESSING,’' which is incorporated herein by reference in its entirety7.TECHNICAL FIELD

[0002] The present disclosure relates to image processing. More particularly, the present disclosure relates to effectively processing image data across one or more edge devices.BACKGROUND

[0003] Computer vision processes play a vital role in extracting meaningful information from digital images or videos, encompassing various tasks such as object detection, image classification, and segmentation. In the context of loT networks, deploying computer vision involves devices both at the edge and within a centralized data network. At the edge of the network, where loT devices equipped with cameras capture image data, data is sent to a centralized server for processing. This processing might include tasks like noise reduction, resizing, or compression to optimize the data before transmission.

[0004] In the centralized network, additional leveraging of heavier-duty algorithms such as deep learning models for complex object detection, recognition, or scene understanding can occur. The centralized network also serves as the hub for aggregating data from multiple edge devices, allowing for a comprehensive analysis of the overall environment. This aggregated data enables higher-level decision-making processes based on the insights derived from computer vision processing.

[0005] However, deploying computer vision in loT networks comes with its own set of challenges and considerations. Firstly, latency can be a critical issue, especially for real-time applications. Bandwidth constraints also pose challenges, as transmitting large volumes of image data from edge devices to the centralized network can strain network resources. Moreover, security and privacy concerns are paramount in loT environments, particularly when dealing with sensitive image data. Additionally, scalability and resource constraints must be addressed to accommodate the growing number of edge devices and increasing computational demands.Attorney Docket No. 67023.72WO01SUMMARY

[0006] Systems and methods for effectively processing image data across one or more edge devices in accordance with embodiments of the disclosure are described herein. In some embodiments, a device includes a processor, a network interface controller configured to provide access to a network, and a memory communicatively coupled to the processor, wherein the memory includes an edge processing logic. The logic is configured to: receive edge data from a plurality of network devices, store the edge data, format the edge data for a query, receive a query', and generate a result based on the received query'.

[0007] In some embodiments, the edge processing logic is further configured to process the edge data prior to being stored.

[0008] In some embodiments, the processing includes processing the edge data through at least a computer vision (CV) model to generate CV model output data.

[0009] In some embodiments, the processing further includes passing the CV model output data to a large language model (LLM) to generate LLM model output data.

[0010] In some embodiments, the edge processing logic is further configured to concatenate at least the edge data, CV model output data, and LLM model output data into a compressed data stream.

[0011] In some embodiments, the edge processing logic stores the compressed data stream.

[0012] In some embodiments, the edge processing logic is further configured to format the compressed data stream into one or more embeddings.

[0013] In some embodiments, generating the query result is based on at least the one or more embeddings.

[0014] In some embodiments, the query is received from a requesting device.

[0015] In some embodiments, the edge processing logic is further configured to transmit the result to the requesting device.

[0016] In some embodiments, the device exists within a networked system of devices including centralized devices and edge devices.Attorney Docket No. 67023.72WO01

[0017] In some embodiments, the device is an edge device.

[0018] In some embodiments, the edge processing logic is further configured to compress the processed edge data prior to storage.

[0019] In some embodiments, storing the processed edge data includes transmitting the processed edge data to a centralized device.

[0020] In some embodiments, compression of the processed edge data is based on at least one detected event.

[0021] In some embodiments, a device includes a processor, a network interface controller configured to provide access to a network, and a memory' communicatively coupled to the processor, wherein the memory includes an edge processing logic. The logic is configured to receive edge data from a plurality of network devices, parse the edge data, extract one or more contextual features from the edge data, detect at least one behavior based on the one or more contextual features, and compress the processed edge data based on the at least one behavior, wherein the device: exists within a networked system of devices including centralized devices and edge devices, and is an edge device.

[0022] In some embodiments, the compressed edge data is stored on a centralized device.

[0023] In some embodiments, the edge processing logic is further configured to store the compressed edge data.

[0024] In some embodiments, the edge processing logic is further configured to format the stored edge data into embedded data.

[0025] In some embodiments, a method includes receiving edge data from a plurality of network devices, storing the edge data, formatting the edge data for a query, receiving a query, and generating a result based on the received query.

[0026] Other objects, advantages, novel features, and further scope of applicability of the present disclosure will be set forth in part in the detailed description to follow, and in part will become apparent to those skilled in the art upon examination of the following or may’ be learned by practice of the disclosure. Although the description above contains many specificities, these should not be construed as limiting the scope of the disclosure but as merely providingAttorney Docket No. 67023.72WO01illustrations of some of the presently preferred embodiments of the disclosure. As such, various other embodiments are possible within its scope. Accordingly, the scope of the disclosure should be determined not by the embodiments illustrated, but by the appended claims and their equivalents.BRIEF DESCRIPTION OF DRAWINGS

[0027] The above, and other, aspects, features, and advantages of several embodiments of the present disclosure will be more apparent from the following description as presented in conjunction with the following several figures of the drawings.

[0028] FIG. 1 is a conceptual illustration of a video data processing system in accordance with various embodiments of the disclosure;

[0029] FIG. 2 is a conceptual system architecture for edge video processing in accordance with various embodiments of the disclosure;

[0030] FIG. 3 is a conceptual system block chart for edge video processing in accordance with various embodiments of the disclosure;

[0031] FIG. 4A is a conceptual illustration of a sample image within one or more sets of image data in accordance with an embodiment of the disclosure;

[0032] FIG. 4B is a conceptual illustration of a sample image within one or more sets of image data processed to recognize a plurality' of subjects in accordance with an embodiment of the disclosure;

[0033] FIG. 5 is a flowchart depicting a process 500 for generating a result based on a query' in accordance with various embodiments of the disclosure;

[0034] FIG. 6 is a flowchart depicting a process for generating a query in accordance with various embodiments of the disclosure;

[0035] FIG. 7 is a flowchart depicting a process for generating embedded data in accordance with various embodiments of the disclosure;

[0036] FIG. 8 is a flowchart depicting a process for formatting data for query' processing in accordance with various embodiments of the disclosure; andAttorney Docket No. 67023.72WO01

[0037] FIG. 9 is a device suitable for utilizing an edge processing logic in accordance with various embodiments of the disclosure.

[0038] Corresponding reference characters indicate corresponding components throughout the several figures of the drawings. Elements in the several figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures might be emphasized relative to other elements for facilitating understanding of the various presently disclosed embodiments. In addition, common, but well-understood, elements that are useful or necessary7in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments of the present disclosure.DETAILED DESCRIPTION

[0039] In response to the issues described above and herein, devices, methods, and system are discussed herein that allow for the capturing of edge data, such as video data and processing the edge data for use in responding to queries associated with the edge data. In various embodiments, a network deployment may have various devices situated across a large area. Network devices on the edge can traditionally need to send excess data to centralized devices for processing, such as responding to queries. In a number of embodiments, the edge devices can process the edge data into numerous structures that allow for the processing of queries on the edge devices. In more embodiments, the edge data can be processed such that less data is required to be transferred to centralized devices. This can improve the function of the network devices by providing responses to queries in less time and reduce the amount of data transferred across the network, increasing overall network bandwidth.

[0040] Capturing edge data and processing it on edge devices before sending it to centralized devices on the network is a strategy7adopted in various embodiments, particularly those requiring real-time or near-real-time analysis. This approach offers several advantages, including reduced latency, decreased network bandwidth usage, and enhanced privacy by minimizing data transmission. In this context, edge devices such as loT sensors or cameras capture raw data, such as video streams, directly at the source. These devices are equipped with computational capabilities to perform initial processing tasks locally. For example, a camera equipped with a computer vision model can analyze the video feed in real-time to detect objects, recognize faces, or identify events of interest. This initial processing on the edge deviceAttorney Docket No. 67023.72WO01filters and prioritizes the data, transmitting only relevant or actionable information to centralized devices on the network.

[0041] Once the edge data, such as preprocessed video frames, is sent to other edge or intermediate devices, it can undergo further analysis and refinement. Here, the preprocessed data may be routed through more sophisticated models like large language models (LLMs) for deeper contextual understanding or inference. The LLM model can generate textual descriptions, extract insights, or provide additional context based on the processed video data. This multi-stage processing pipeline allows for a hierarchical approach to data analysis, where edge devices handle immediate processing needs while centralized devices tackle more complex tasks requiring broader context or historical data.

[0042] By integrating both computer vision models and LLMs into the edge processing pipeline, edge devices can perform increasingly sophisticated analyses locally, reducing reliance on centralized resources and mitigating latency issues associated with transmitting large volumes of raw data over the network. Moreover, this distributed processing architecture enhances scalability and resilience, as edge devices can operate autonomously even in scenarios where network connectivity is limited or intermittent. Overall, capturing and processing edge data on edge devices before sending it to centralized devices represents a powerful paradigm for efficient, responsive, and privacy-aware data processing in distributed environments.

[0043] The processing of edge data, particularly video data, opens up a wide range of query possibilities that span both content and context. On the content side, queries can focus on specific objects, events, or patterns detected within the video frames by the edge devices' computer vision models. For example, users might inquire about the presence of certain objects (e.g., cars, people, or animals) in the video feed, inquire about the frequency or duration of particular events (e.g., vehicle movements, crowd gatherings, or unusual activities), or request summaries or highlights of significant occurrences captured in the video data.

[0044] Additionally, queries can delve into the context surrounding the video content, leveraging the insights provided by large language models (LLMs) processing the preprocessed edge data. Users can pose questions that require understanding the temporal, spatial, or semantic context of the video frames, such as inquiries about the location, time, or weather conditions depicted in the footage. Furthermore, LLMs can provide deeper contextualAttorney Docket No. 67023.72WO01understanding by analyzing textual descriptions, generating captions, or inferring relationships between objects and events captured in the video data.

[0045] By combining content-based queries with contextual inquiries, users can gain comprehensive insights into the edge data, extracting actionable information relevant to their specific interests or objectives. These queries enable diverse applications across various domains, including surveillance, security, industrial monitoring, and smart city initiatives, empowering users to make informed decisions and respond effectively to dynamic situations captured in the edge data. Overall, the processing of edge data facilitates interactive and exploratory analysis, unlocking valuable insights and enhancing situational awareness in distributed environments.

[0046] Aspects of the present disclosure may be embodied as an apparatus, system, method, or computer program product. Accordingly , aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, or the like) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “function,’' “module,” “apparatus,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more non-transitory computer-readable storage media storing computer-readable and / or executable program code. Many of the functional units described in this specification have been labeled as functions, in order to emphasize their implementation independence more particularly. For example, a function may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A function may also be implemented in programmable hardware devices such as via field programmable gate arrays, programmable array logic, programmable logic devices, or the like.

[0047] Functions may also be implemented at least partially in software for execution by various types of processors. An identified function of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions that may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified function need not be physically located together but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the function and achieve the stated purpose for the function.Attorney Docket No. 67023.72WO01

[0048] Indeed, a function of executable code may include a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, across several storage devices, or the like. Where a function or portions of a function are implemented in software, the software portions may be stored on one or more computer-readable and / or executable storage media. Any combination of one or more computer-readable storage media may be utilized. A computer-readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing, but would not include propagating signals. In the context of this document, a computer readable and / or executable storage medium may be any tangible and / or non-transitory medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, processor, or device.

[0049] Computer program code for carry ing out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object-oriented programming language such as Python, Java, Smalltalk, C++, C#, Objective C, or the like, conventional procedural programming languages, such as the “C” programming language, scripting programming languages, and / or other similar programming languages. The program code may execute partly or entirely on one or more of a user's computer and / or on a remote computer or server over a data network or the like.

[0050] A component, as used herein, comprises a tangible, physical, non-transitory device. For example, a component may be implemented as a hardware logic circuit comprising custom VLSI circuits, gate arrays, or other integrated circuits; off-the-shelf semiconductors such as logic chips, transistors, or other discrete devices; and / or other mechanical or electrical devices. A component may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, or the like. A component may comprise one or more silicon integrated circuit devices (e.g., chips, die. die planes, packages) or other discrete electrical devices, in electrical communication with one or more other components through electrical lines of a printed circuit board (PCB) or the like. Each of the functions and / or modules described herein, in certain embodiments, may alternatively be embodied by or implemented as a component.

[0051] A circuit, as used herein, comprises a set of one or more electrical and / or electronic components providing one or more pathways for electrical current. In certain embodiments, aAttorney Docket No. 67023.72WO01circuit may include a return pathway for electrical current, so that the circuit is a closed loop. In another embodiment, however, a set of components that does not include a return pathway for electrical current may be referred to as a circuit (e.g., an open loop). For example, an integrated circuit may be referred to as a circuit regardless of whether the integrated circuit is coupled to ground (as a return pathway for electrical current) or not. In various embodiments, a circuit may include a portion of an integrated circuit, an integrated circuit, a set of integrated circuits, a set of non-integrated electrical and / or electrical components with or without integrated circuit devices, or the like. In one embodiment, a circuit may include custom VLSI circuits, gate arrays, logic circuits, or other integrated circuits; off-the-shelf semiconductors such as logic chips, transistors, or other discrete devices; and / or other mechanical or electrical devices. A circuit may also be implemented as a synthesized circuit in a programmable hardware device such as field programmable gate array, programmable array logic, programmable logic device, or the like (e.g., as firmware, a netlist, or the like). A circuit may comprise one or more silicon integrated circuit devices (e.g., chips, die, die planes, packages) or other discrete electrical devices, in electrical communication with one or more other components through electrical lines of a printed circuit board (PCB) or the like. Each of the functions and / or modules described herein, in certain embodiments, may be embodied by or implemented as a circuit.

[0052] Reference throughout this specification to “one embodiment,” “an embodiment,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment, but mean “one or more but not all embodiments” unless expressly specified otherwise. The terms “including,” “comprising,” “having,” and variations thereof mean “including but not limited to”, unless expressly specified otherwise. An enumerated listing of items does not imply that any or all of the items are mutually exclusive and / or mutually inclusive, unless expressly specified otherwise. The terms “a,” “an,” and “the” also refer to “one or more” unless expressly specified otherwise.

[0053] Further, as used herein, reference to reading, writing, storing, buffering, and / or transferring data can include the entirety of the data, a portion of the data, a set of the data, and / or a subset of the data. Likewise, reference to reading, writing, storing, buffering, and / orAttorney Docket No. 67023.72WO01transferring non-host data can include the entirety of the non-host data, a portion of the nonhost data, a set of the non-host data, and / or a subset of the non-host data.

[0054] Lastly, the terms “or” and “and / or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and / or C” mean “any of the following: A; B; C A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or acts are in some way inherently mutually exclusive.

[0055] Aspects of the present disclosure are described below with reference to schematic flowchart diagrams and / or schematic block diagrams of methods, apparatuses, systems, and computer program products according to embodiments of the disclosure. It will be understood that each block of the schematic flowchart diagrams and / or schematic block diagrams, and combinations of blocks in the schematic flowchart diagrams and / or schematic block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor or other programmable data processing apparatus, create means for implementing the functions and / or acts specified in the schematic flowchart diagrams and / or schematic block diagrams block or blocks.

[0056] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. Other steps and methods may be conceived that are equivalent in function, logic, or effect to one or more blocks, or portions thereof, of the illustrated figures. Although various arrow types and line types may be employed in the flowchart and / or block diagrams, they are understood not to limit the scope of the corresponding embodiments. For instance, an arrow may indicate a waiting or monitoring period of unspecified duration between enumerated steps of the depicted embodiment.

[0057] In the following detailed description, reference is made to the accompanying drawings, which form a part thereof. The foregoing summary7is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, andAttorney Docket No. 67023.72WO01features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. The description of elements in each figure may refer to elements of proceeding figures. Like numbers may refer to like elements in the figures, including alternate embodiments of like elements.

[0081] Referring to FIG. 1, a conceptual illustration of a video data processing system 90 in accordance with various embodiments of the disclosure is shown. In many embodiments, it may be desired to monitor one or more visual areas by installing cameras 150 within those areas. The cameras 150 can generate a plurality of video data (i.e., “video content”) that can be processed. In some embodiments, the video data can be video data captured on the edge of a network and processed as edge data. In a number of embodiments, the processing of the video data will be configured to determine if one or more specific objects are within the monitored visual areas. In a more embodiments, this detection may be presented as an inference map image from an object detection neural network or computer vision model which can generate data such as a segmentation or panoptic map. These maps can be utilized as a classification as to whether a specific object is present within the image data or not. These maps can be generated as an output from a neural network such as, but not limited to, a convolutional neural network (CNN). By way of example, and not limitation, video data processing can be established to detect the presence of one or more pedestrians within the monitored visual areas or tracking various shoppers within a retail environment, etc. It will be understood by those skilled in the art that the video data processing may be performed by processors internal to cameras 150 or elsewhere in the system 90, within devices on the edge of the system 90 or in some combination thereof. The video data processing may be implemented in software operating in conventional processors (e.g., CPU, MPU, GPU, RISC. etc.), and / or software operating in specifically purposed processors optimized to implement neural networks - or some combination thereof.

[0082] Often, traditional methods, the video data processing system 90 will process the video data within a centralized video processing server 110. However, embodiments described herein can offload various processing tasks to other devices such as, but not limited to servers 130 which may be specialized or edge type servers, or internal to the cameras 150 (themselves edge network devices). A video processing server 110 can be connected to a network 120 such as the Internet as depicted in FIG. 1. A plurality of cameras 150 can also be attached to the network 120 such that they are communicatively coupled to the video processing server 110Attorney Docket No. 67023.72WO01which comprises one or more processors for processing video data like, for example, CPUs, MPUs, GPUs, etc. Although the embodiment of FIG. 1 depicts cameras 150. it will be understood by those skilled in the art that any video data capture device may be utilized as required by the desired application.

[0083] The cameras 150 can be wired directly to the network 120 or may be wirelessly connected via one or more wireless access points 160. In this way, a variety of potential deployment arrangements may be realized to properly cover the desired areas for surveillance. In theory', there is no limit to the number of deployed cameras 150 or other video data capture devices that may be communicatively coupled with the video processing server 110. The limitations experienced may relate to the available bandwidth of the network 120 and computational resources of the video processing server 110 or other supplemental server 130. As discussed below, many embodiments of the video data processing system 90 can handle the bulk of the processing locally by cameras 150 or other edge network devices 140 to minimize network traffic and reduce the need for centralized computing resources like video processing server 110 and servers 130.

[0058] Although a specific embodiment for a video data processing system suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 1, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the cameras 150 may be deployed over many different environments. The elements depicted in FIG. 1 may also be interchangeable with other elements of FIGS. 2 - 9 as required to realize a particularly desired embodiment.

[0059] Referring to FIG. 2, a conceptual system architecture 200 for edge video processing in accordance with various embodiments of the disclosure is shown. In an edge artificial intelligence (Al) network architecture, a hardware abstraction layer (HAL) 210 serves as an intermediary between the Al algorithms and the underlying hardware components. HAL 210 abstracts the hardware complexities and provides a unified interface for interacting with various hardware devices, enabling seamless integration and portability of Al applications across different hardware platforms. At the edge, where computational resources are often limited and heterogeneous, HAL 210 plays a crucial role in optimizing resource utilization and ensuring efficient execution of Al w orkloads.Attorney Docket No. 67023.72WO01

[0060] The HAL 210 abstracts the specific hardware details, such as processor architectures, memory configurations, and communication interfaces, into a standardized set of interfaces and APIs (Application Programming Interfaces). This allows Al developers to write platform-independent code that can run on different hardware platforms without modification. For example, instead of directly interfacing with the hardware accelerators or peripheral devices, Al algorithms interact with the HAL 210 through standardized APIs, which handle the low-level hardware interactions transparently.

[0061] Furthermore, HAL 210 provides services for resource management, scheduling, and optimization to maximize the performance and efficiency of Al orkloads on edge devices. It dynamically allocates computational resources based on the workload requirements and device capabilities, ensuring that Al tasks are executed optimally while minimizing resource contention and latency. Additionally, HAL 210 abstracts the communication between Al modules and external sensors or actuators, enabling seamless integration of Al capabilities with loT devices and other edge peripherals.

[0062] Within the hardware abstraction layer (HAL 210) of an edge artificial intelligence (Al) network architecture, a Real-Time Operating System (RTOS) 211 plays a crucial role in orchestrating the execution of tasks and managing resources on the underlying hardware devices. The RTOS 211 serves as the core software layer that ensures the timely and deterministic execution of Al algorithms, sensor data processing, and control tasks in real-time or near-real-time scenarios. Task management is a primary' function of the RTOS 211, as it oversees the scheduling and execution of various tasks associated with Al inference, data acquisition, preprocessing, and communication. By prioritizing tasks based on their urgency and timing constraints, the RTOS 211 guarantees that critical operations, such as real-time control or event handling, are completed within specified deadlines, maintaining system responsiveness and stability.

[0063] Resource allocation is another key aspect managed by the RTOS 211 within the HAL 210. It dynamically assigns computational resources, including CPU time, memory', and I / O bandwidth, to different tasks based on their priority levels and resource requirements. This ensures optimal resource utilization and performance, even in resource-constrained edge computing environments. Moreover, the RTOS 211 handles interrupts generated by peripheral devices, such as sensors or actuators, promptly and deterministically. By prioritizing and servicing interrupts based on their criticality, the RTOS 211 ensures that time-sensitive events,Attorney Docket No. 67023.72WO01such as sensor data acquisition or control signals, are processed without delay, enabling timely responses to external stimuli.

[0064] In addition to task management and resource allocation, the RTOS 211 facilitates communication and synchronization between different tasks and hardware components within the edge device. It provides mechanisms for inter-task communication, such as message queues, semaphores, and shared memory, enabling seamless data exchange and coordination between concurrent tasks. This allows Al algorithms to interact with sensor data, control actuators, and communicate with other edge devices in a synchronized manner, supporting complex edge Al applications. Furthermore, the RTOS 211 incorporates features for fault tolerance and reliability, ensuring that the system can detect and recover from errors or faults gracefully. This capability is essential for mission-critical edge Al applications where reliability and uptime are paramount, safeguarding against system failures and ensuring uninterrupted operation in cHAL 2101enging environments. Overall, the integration of an RTOS 211 into the HAL 210 of an edge Al network architecture provides a robust and efficient platform for deploying real-time Al applications in diverse edge computing scenarios.

[0065] The utilization of a Long-Term Support (LTS) Linux Kernel 212 within the hardware abstraction layer (HAL 210) of an edge artificial intelligence (Al) network architecture offers several advantages for managing hardware resources and facilitating seamless integration of Al algorithms. The LTS Linux Kernel 212 serves as the underlying operating system layer that interacts directly with the hardware components of the edge devices, providing essential services such as device drivers 214 (shown as “wifi, cellular drivers”, and “camera drivers”), memory management, and process scheduling. Within the HAL 210, the LTS Linux Kernel 212 abstracts the hardware complexities and provides a standardized interface for higher-level software components, including Al algorithms and application framework 220s.

[0066] One of the key benefits of utilizing an LTS Linux Kernel 212 in the HAL 210 is its broad support for a wide range of hardware platforms and devices commonly found in edge computing environments. The LTS Linux Kernel 212 includes a comprehensive set of device drivers 214 and support for various hardware interfaces, enabling edge devices to seamlessly interface with sensors, cameras, accelerometers, and other peripheral devices. This ensures compatibility and interoperability across different hardware configurations, simplifying the development and deployment of edge Al applications.Attorney Docket No. 67023.72WO01

[0067] Moreover, the LTS Linux Kernel 212 offers robust memory management and process scheduling capabilities, which are essential for efficient resource utilization and multitasking in edge computing scenarios. The kernel manages memory allocation, virtual memory mapping, and process execution, ensuring that Al algorithms and other tasks run smoothly without resource contention or system instability. Additionally, the kernel provides mechanisms for power management and system optimization, enabling edge devices to conserve energy and extend batters’ life, which is critical for battery-powered or resource-constrained edge devices.

[0068] Furthermore, the LTS Linux Kernel 212 supports real-time capabilities through features such as the PREEMPT RT patchset, which enhances the kernel's responsiveness and determinism for real-time applications. This is particularly beneficial for edge Al applications that require low-latency and predictable performance, such as control systems, robotics, and industrial automation. By leveraging the real-time capabilities of the LTS Linux Kernel 212, the HAL 210 can ensure that critical Al tasks, sensor data processing, and control algorithms are executed with precise timing and rel i abi 1 i t , meeting the stringent requirements of real-time edge computing applications.

[0069] Open-source libraries 213 play a crucial role in enhancing the functionality and flexibility of a hardware abstraction layer (HAL 210) within an edge artificial intelligence (Al) network architecture. These libraries provide pre-built software components, modules, and algorithms that can be leveraged to streamline the development and implementation of the HAL 210, reducing time-to-market and development costs. In the context of the HAL 210, open-source libraries 213 can be utilized in various ways to facilitate hardware interaction. Al algorithm execution, and overall system management.

[0070] One common use of open-source libraries 213 in the HAL 210 is for device drivers 214 and hardware abstraction. These libraries provide standardized interfaces and APIs for interacting with different hardware components, including sensors, cameras, communication interfaces, and peripherals. By abstracting the hardware details and providing a unified interface, these libraries simplify the integration of diverse hardware devices into the edge computing platform. Developers can use these libraries to access and control hardware resources without needing to understand the intricacies of low-level hardware programming, enabling rapid prototyping and deployment of edge Al applications.Attorney Docket No. 67023.72WO01

[0071] Furthermore, open-source libraries 213 offer a wealth of Al algorithms and machine learning models that can be integrated into the HAL 210 to enhance its capabilities. These libraries include popular frameworks such as TensorFlow, PyTorch, and OpenCV, which provide a wide range of algorithms for tasks such as object detection, image classification, natural language processing, and anomaly detection. By leveraging these pre-built algorithms, developers can quickly implement Al-driven functionalities in the HAL 210, enabling edge devices to perform advanced analytics and decision-making tasks autonomously.

[0072] Moreover, open-source libraries 213 can be used for system management and optimization within the HAL 210. These libraries provide tools and utilities for tasks such as performance monitoring, resource allocation, power management, and security enforcement. For example, libraries like Apache Kafka or MQTT can be used for efficient data streaming and communication between edge devices and cloud services. Similarly, libraries like Docker or Kubemetes can be used for containerization and orchestration of edge applications, ensuring scalability, reliability, and portability’ of the HAL 210 across different deployment environments.

[0073] A BSP (Board Support Package) for Linux 216 serves as a foundational component within the hardware abstraction layer (HAL 210) of an edge artificial intelligence (Al) network architecture, providing essential software components and drivers to enable Linux to run on specific hardware platforms. BSP-Linux 216 packages typically include device drivers 214, kernel configurations, boot loaders, and other software components tailored to a particular hardware platform. In the context of the HAL 210, BSP-Linux facilitates seamless interaction between the underlying hardware devices and the higher-level software components, including Al algorithms and system management utilities.

[0074] One of the primary functions of BSP-Linux within the HAL 210 is to provide device drivers 214 and hardware abstraction for the edge computing platform. These device drivers 214 enable Linux to interface with various hardware components, such as sensors, cameras, communication interfaces, and peripheral devices, by abstracting the hardware details and providing a standardized interface. By incorporating BSP-Linux into the HAL 210, developers can ensure compatibility and interoperability’ across different hardware configurations, simplifying the integration of diverse hardware devices into the edge computing platform.Attorney Docket No. 67023.72WO01

[0075] Moreover, BSP-Linux contributes to system initialization and configuration within the HAL 210, ensuring that the hardware devices are initialized and configured correctly during the boot process. This involves tasks such as configuring kernel parameters, initializing memory, setting up interrupt handlers, and loading necessary firmware or device tree overlays. By handling these initialization tasks, BSP-Linux streamlines the boot process and ensures that the hardware devices are ready for operation when the system is booted up, reducing startup time and improving system reliability.

[0076] Additionally, BSP-Linux plays a crucial role in managing power consumption and optimizing system performance within the HAL 210. BSP-Linux includes power management features, such as CPU frequency scaling, device power states management, and system sleep modes, to minimize energy consumption and extend battery life in resource-constrained edge devices. Moreover, BSP-Linux incorporates performance optimization techniques, such as kernel configuration tuning, memory' optimization, and I / O scheduling, to maximize the efficiency and responsiveness of the edge computing platform.

[0077] Furthermore, BSP-Linux provides support for real-time capabilities through features such as the PREEMPT_RT patchset, which enhances the Linux Kernel 212's responsiveness and determinism for real-time applications. This is particularly beneficial for edge Al applications that require low-latency and predictable performance, such as control systems, robotics, and industrial automation. By leveraging BSP-Linux, the HAL 210 can ensure that critical Al tasks, sensor data processing, and control algorithms are executed with precise timing and reliability, meeting the stringent requirements of real-time edge computing applications.

[0078] Drivers 215, including those for Wi-Fi, cellular, camera, and other peripherals, are essential components utilized within the hardware abstraction layer (HAL 210) of an edge artificial intelligence (Al) network architecture. These drivers 215 serve as intermediaries between the higher-level software components, such as Al algorithms and system management utilities, and the underlying hardware devices. In the context of the HAL 210, drivers 215 play a crucial role in abstracting the hardware details and providing a standardized interface for interacting with various hardware components, enabling seamless integration and interoperability across different hardware platforms.Attorney Docket No. 67023.72WO01

[0079] For example, Wi-Fi and cellular drivers 214 enable edge devices to establish wireless connectivity and communicate with remote servers, cloud platforms, or other edge devices. These drivers 215 handle tasks such as network configuration, authentication, encryption, and data transmission, allowing edge devices to access external data sources, upload telemetry' data, or receive software updates over the network. By abstracting the complexities of wireless communication protocols and hardware interfaces. Wi-Fi and cellular drivers 214 simplify the integration of wireless connectivity into the edge computing platform, enabling flexible and scalable deployment of edge Al applications.

[0080] Similarly, camera drivers 214 facilitate the acquisition and processing of visual data from cameras or imaging sensors deployed on edge devices. These drivers handle tasks such as camera initialization, configuration, frame capture, and image processing, enabling edge devices to capture images or video streams for analysis by Al algorithms. Camera drivers abstract the hardware-specific details of camera interfaces, such as USB, MIPI CSI, or Ethernet, providing a unified interface for accessing camera data regardless of the underlying hardware platform. This enables developers to seamlessly integrate camera functionality into edge Al applications without needing to understand the intricacies of camera hardware or communication protocols.

[0081] Additionally, other drivers 215, such as those for sensors, actuators, storage devices, or communication interfaces, are utilized within the HAL 210 to enable interaction with various hardware peripherals and components. These drivers 215 provide standardized interfaces and APIs for accessing and controlling hardware resources, allowing edge devices to interface with sensors for environmental monitoring, control actuators for device automation, or communicate with external devices using protocols like SPI, I2C, orUART. By abstracting the hardware-specific details of peripheral devices, these drivers 215 ensure compatibility and interoperability across different hardware configurations, facilitating rapid prototyping and deployment of edge Al applications.

[0082] In the context of an edge Al network architecture, an application framework 220 plays a pivotal role in facilitating the development, deployment, and management of edge Al applications. An application framework 220 provides a comprehensive set of tools, libraries, and APIs that streamline the development process and abstract the complexities of building and running Al-driven applications at the edge. These frameworks offer standardized interfaces for tasks such as data preprocessing, model training, inference execution, and resultAttorney Docket No. 67023.72WO01visualization, enabling developers to focus on application logic and functionality without needing to delve into low-level implementation details.

[0083] One primary use of an application framework 220 within the edge Al network architecture is for Al model management and deployment. These frameworks typically include tools for model training, optimization, and conversion, allowing developers to train machine learning models using diverse datasets and algorithms. Once trained, the models can be optimized for deployment on edge devices with resource constraints, such as limited memory, processing power, or energy7consumption. The framework handles tasks such as model quantization, pruning, and compression to reduce model size and complexity, ensuring efficient execution on edge devices without sacrificing performance or accuracy.

[0084] Furthermore, application framework 220s offer APIs and libraries for executing Al inference tasks on edge devices. These inference engines provide runtime environments for loading and executing pre-trained models, handling data input / output, and performing inference calculations efficiently. By leveraging these inference engines, developers can deploy Al models on edge devices and execute real-time inference tasks, such as object detection, image classification, anomaly detection, or predictive maintenance, directly on the device without relying on cloud services or external servers. This enables edge devices to make autonomous decisions and take actions based on local data, reducing latency, bandwidth consumption, and reliance on network connectivity.

[0085] Moreover, application framework 220s support edge Al application development by providing tools for data preprocessing, feature extraction, and result visualization. These tools simplify data management tasks, such as data cleaning, transformation, and augmentation, ensuring that input data is appropriately processed and formatted before being fed into Al models. Additionally, frameworks offer utilities for visualizing model outputs, analyzing inference results, and monitoring system performance, enabling developers to validate model accuracy, debug errors, and optimize application behavior iteratively. This facilitates the development of robust and reliable edge Al applications that meet the specific requirements and constraints of the edge computing environment.

[0086] Runtime libraries 221 play a crucial role within the application framework 220 discussed in the context of an edge Al network architecture, providing essential functionalities and services for executing Al-driven applications on edge devices. These libraries offer a setAttorney Docket No. 67023.72WO01of pre-built software components, APIs, and runtime environments that streamline the deployment, execution, and management of Al models and applications at the edge. Within the application framework 220, runtime libraries 221 are utilized in various ways to facilitate model deployment, inference execution, and system management tasks.

[0087] One primary use of runtime libraries 221 within the application framework 220 is for model deployment and inference execution. These libraries provide runtime environments and APIs for loading, initializing, and executing pre-trained machine learning models on edge devices. By leveraging these runtime environments, developers can deploy Al models directly onto edge devices and perform real-time inference tasks, such as object detection, image classification, or anomaly detection, without relying on external servers or cloud services. The runtime libraries 221 handle tasks such as model loading, input data processing, inference calculation, and result output, ensuring efficient execution of Al models on resource-constrained edge devices.

[0088] Furthermore, runtime libraries 221 offer optimization and acceleration techniques to improve the performance and efficiency of Al inference tasks on edge devices. These libraries include optimizations for hardware acceleration, such as GPU, FPGA, or neural processing unit (NPU), to leverage specialized hardware accelerators for accelerating model inference calculations. Additionally, runtime libraries 221 may incorporate techniques such as model quantization, pruning, and compression to reduce model size and complexity, enabling efficient execution on edge devices with limited memory and processing power. By optimizing inference performance, runtime libraries 221 ensure that Al-driven applications can meet realtime performance requirements and operate effectively in edge computing environments.

[0089] Moreover, runtime libraries 221 support system management and monitoring tasks within the application framework 220. These libraries provide utilities for monitoring system performance, analyzing inference results, and logging runtime metrics, enabling developers to assess application behavior, diagnose errors, and optimize system performance iteratively. Additionally, runtime libraries 221 may include tools for managing system resources, such as memory' allocation, task scheduling, and power management, to ensure efficient utilization of hardware resources and optimize energy consumption on edge devices. By providing these system management capabilities, runtime libraries 221 enable developers to build robust and reliable edge Al applications that meet the specific requirements and constraints of the edge computing environment.Attorney Docket No. 67023.72WO01

[0090] Middleware 222, including routing, DHCP (Dynamic Host Configuration Protocol), switches, device servers, and other components, play a crucial role within the application framework 220 discussed in the context of an edge Al network architecture. These middleware 222 components provide essential infrastructure services and networking capabilities that enable communication, data exchange, and system management within the edge computing environment. Within the application framework 220. middleware 222 is utilized in various ways to facilitate seamless integration, connectivity, and interoperability of edge devices and applications.

[0091] Routing middleware 222, for example, is utilized to establish netw ork routes and paths for data transmission between edge devices, cloud services, and external networks. Routing algorithms determine the most efficient path for data packets to travel based on factors such as network topology, bandwidth availability, and latency constraints, ensuring optimal data routing and delivery'. By leveraging routing middleware 222, edge Al applications can communicate with remote servers, cloud platforms, or other edge devices over wired or wireless networks, enabling seamless data exchange and collaboration across distributed environments.

[0092] Similarly, DHCP middleware 222 is used to dynamically assign IP addresses and network configuration parameters to edge devices within the netw ork. DHCP servers automate the process of IP address allocation, ensuring that each device on the network has a unique identifier and can communicate with other devices and services. By automating IP address management, DHCP middleware 222 simplifies network configuration and administration, reducing the burden on system administrators and facilitating rapid deployment and scaling of edge computing solutions.

[0093] Additionally, switches and device servers are utilized within the application framework 220 to manage network connectivity and facilitate data exchange between edge devices and external systems. Switches provide network segmentation and traffic management capabilities, enabling efficient data transmission and reducing network congestion. Device servers act as intermediaries between edge devices and external systems, providing protocol translation, data aggregation, and integration services. By utilizing switches and device servers within the application framework 220, edge Al applications can establish reliable and secure communication channels with external services, loT devices, or cloud platforms, enabling seamless integration and interoperability across heterogeneous environments.Attorney Docket No. 67023.72WO01

[0094] Moreover, other middleware 222 components, such as message brokers, data brokers, and protocol gateways, may be utilized within the application framework 220 to facilitate data exchange, event processing, and integration between different systems and protocols. Message brokers, for example, enable publish-subscribe messaging patterns, allowing edge devices and applications to exchange data asynchronously and decoupling producers from consumers. Data brokers facilitate data aggregation, filtering, and transformation, enabling edge devices to process and transmit data efficiently. Protocol gateways provide translation and interoperability between different communication protocols, enabling seamless integration of legacy systems, proprietary' protocols, and modem loT standards within the edge computing environment.

[0095] Al middleware 223, including DL (Deep Learning) containers, loaders, and neural network (NN) frameworks such as PyTorch, TensorFlow, ONNX, and MXNet, play a crucial role within the application framework 220 discussed in the context of an edge Al network architecture. These middleware 222 components provide essential tools, libraries, and runtime environments for developing, deploying, and executing Al-driven applications on edge devices. Within the application framework 220, Al middleware 223 is utilized in various ways to facilitate model deployment, inference execution, and integration with edge computing systems.

[0096] DL containers sen e as portable, self-contained environments for packaging and deploying Al models and dependencies. These containers encapsulate all the necessary software components, libraries, and dependencies required to run Al applications, ensuring consistency and reproducibility across different computing environments. Within the application framework 220, DL containers are utilized to package and distribute pre-trained Al models, enabling developers to deploy models on edge devices with ease. By leveraging DL containers, developers can ensure that Al models are packaged with their dependencies and can be deployed seamlessly on diverse edge computing platforms without compatibility issues.

[0097] Loaders within the Al middleware 223 are responsible for loading and initializing pre-trained Al models into memory for inference execution. These loaders handle tasks such as model loading, input data preprocessing, and result output, ensuring that Al models are ready for inference tasks on edge devices. Within the application framework 220, loaders abstract the complexities of model loading and initialization, providing a unified interface for accessing and executing Al models across different frameworks and platforms. By leveraging loaders,Attorney Docket No. 67023.72WO01developers can integrate pre-trained models into their edge Al applications efficiently, enabling real-time inference tasks without needing to implement low-level model loading logic.

[0098] Moreover, NN frameworks such as PyTorch, TensorFlow, ONNX, and MXNet provide comprehensive sets of tools, APIs, and runtime environments for developing and executing Al models on edge devices. These frameworks offer support for various neural network architectures, optimization techniques, and hardware accelerators, enabling developers to build and deploy Al models tailored to specific edge computing requirements. Within the application framework 220, NN frameworks are utilized to train, optimize, and deploy Al models, as well as to perform real-time inference tasks on edge devices. By leveraging NN frameworks, developers can leverage state-of-the-art machine learning algorithms and techniques to build robust, efficient, and scalable edge Al applications.

[0099] Additionally, Al middleware 223 facilitates model interoperability and portability through standardized formats such as ONNX (Open Neural Network Exchange), enabling seamless integration and interoperability between different Al frameworks and platforms. ONNX provides a common representation for trained models, allowing models trained in one framework to be exported, imported, and executed in another framework without conversion or compatibility issues. Within the application framework 220. ONNX enables developers to leverage the strengths of different Al frameworks and tools, ensuring flexibility and interoperability in model development and deployment.

[0100] Services 224, including SIM management, positioning, software and firmware updates, consoleflow, AWS loT, MQTT, and LWM2M, play essential roles within the application framework 220 discussed in the context of an edge Al network architecture. These sendees 224 provide critical functionalities and capabilities that enable edge devices to connect, communicate, and interact with external systems, cloud platforms, and loT networks. Within the application framework 220, services 224 are utilized in various ways to facilitate device management, data exchange, and system integration tasks.

[0101] SIM management services 224 enable the provisioning, activation, and management of SIM cards and cellular connectivity for edge devices. These services 224 handle tasks such as SIM card registration, subscription management, data usage monitoring, and roaming support, ensuring seamless connectivity and reliable communication over cellular networks. Within the application framework 220, SIM management services 224 enable edgeAttorney Docket No. 67023.72WO01devices to establish and maintain cellular connections, enabling data transmission, remote monitoring, and control of edge devices in diverse deployment environments.

[0102] Positioning services 224 provide accurate location information and geospatial data for edge devices, enabling applications such as asset tracking, fleet management, and geofencing. These services 224 utilize GPS. Wi-Fi, cellular triangulation, or other positioning technologies to determine the geographical coordinates and movements of edge devices. Within the application framework 220, positioning sendees 224 enable developers to track the location of edge devices in real-time, monitor their movements, and trigger location-based actions or alerts based on predefined rules and geofences.

[0103] Software and firmware update services 224 facilitate the remote deployment and management of software updates, patches, and firmware upgrades for edge devices. These senices 224 handle tasks such as version management, dependency resolution, deployment scheduling, and rollback procedures, ensuring seamless and secure updates of software and firmware on edge devices. Within the application framework 220, update services 224 enable developers to deploy bug fixes, security patches, and new features to edge devices over-the-air, without requiring manual intervention or physical access to the devices.

[0104] Consoleflow, AWS loT, MQTT, and LWM2M are communication and messaging protocols / services 224 used within the application framework 220 to enable seamless data exchange, device management, and integration with external systems and cloud platforms. Consoleflow provides a centralized dashboard and management console for monitoring, configuring, and managing edge devices and applications. AWS loT offers a suite of services 224 for connecting, managing, and securing loT devices, enabling seamless integration with AWS cloud services 224 and infrastructure. MQTT (Message Queuing Telemetry' Transport) is a lightweight messaging protocol used for publish-subscribe messaging patterns, facilitating asynchronous communication between edge devices, cloud services 224, and other systems. LWM2M (Lightweight M2M) is a protocol for managing and provisioning loT devices, enabling remote configuration, firmware updates, and device management functionalities.

[0105] Within the application framework 220, these services 224 enable edge devices to communicate with cloud platforms, exchange data with external systems, and perform remote management and monitoring tasks efficiently. By leveraging services 224 such as SIM management, positioning, software updates, and communication protocols like Consoleflow,Attorney Docket No. 67023.72WO01AWS loT, MQTT. and LWM2M, developers can build robust, scalable, and interconnected edge Al applications that leverage the capabilities of edge computing and loT technologies to enable real-time insights, decision-making, and automation in diverse deployment scenarios.

[0106] Protocols 225 such as cellular, Bluetooth Low Energy (BLE), mesh networking, and Wi-Fi play integral roles within the application framework 220 discussed in the context of an edge Al network architecture. These protocols 225 provide the communication standards and technologies that enable edge devices to connect, exchange data, and interact with each other, as well as with external systems and cloud platforms. Within the application framework 220, protocols 225 are utilized in various ways to establish communication channels, facilitate data exchange, and enable seamless integration with diverse networking environments.

[0107] Cellular protocols 225, such as LTE (Long-Term Evolution) and 5G, are widely used for providing wide-area wireless connectivity to edge devices, enabling communication over cellular networks. Cellular protocols 225 offer high-speed data transmission, wide coverage, and reliable connectivity, making them suitable for applications that require mobility or remote access. Within the application framework 220, cellular protocols 225 enable edge devices to establish internet connectivity, transmit data to cloud platforms, and receive commands or updates remotely, facilitating real-time monitoring, control, and management of edge devices in diverse deployment scenarios.

[0108] Bluetooth Low Energy (BLE) is a short-range wireless communication protocol commonly used for connecting edge devices to smartphones, wearables, and other nearby devices. BLE enables low-power, energy-efficient communication between devices, making it suitable for applications such as proximity sensing, indoor navigation, and personal area networking. Within the application framework 220, BLE enables edge devices to interact with user devices, exchange data, and trigger actions based on proximity or sensor data, enhancing user experiences and enabling context-aware applications.

[0109] Mesh networking protocols 225, such as Zigbee, Z-Wave, and Thread, enable edge devices to form self-organizing, self-healing networks for communication and data exchange. Mesh networks consist of interconnected nodes that relay data between each other, enabling communication over extended distances and through obstacles. Within the application framework 220, mesh networking protocols 225 enable edge devices to establish resilient,Attorney Docket No. 67023.72WO01decentralized communication networks, facilitating peer-to-peer data exchange, group coordination, and collaborative decision-making in distributed edge computing environments.

[0110] Wi-Fi protocols 225, such as IEEE 802.11 standards, are widely used for providing local-area wireless connectivity7to edge devices, enabling communication over Wi-Fi networks. Wi-Fi protocols 225 offer high-speed data transmission, low latency, and interoperability with existing network infrastructure, making them suitable for applications that require high-bandwidth communication or integration with enterprise networks. Within the application framework 220, Wi-Fi protocols 225 enable edge devices to connect to local networks, access cloud services, and exchange data with other devices, enabling seamless integration with existing IT infrastructure and enabling scalable deployment of edge Al applications.

[0111] Customer applications 230 play a central role within the Edge Al Architecture, serving as the primary interface through which end-users interact with and derive value from the system. These applications are tailored to specific use cases, business objectives, and user requirements, leveraging the capabilities of edge computing and Al technologies to deliver real-time insights, decision-making, and automation at the edge. Within the Edge Al Architecture, customer applications 230 are deployed on edge devices or cloud servers, depending on the deployment model and application requirements, and they interact with various components of the architecture to fulfill their intended functionalities.

[0112] One key aspect of customer applications 230 within the Edge Al Architecture is their ability to leverage Al models and algorithms deployed on edge devices for real-time inference tasks. These applications utilize pre-trained machine learning models to analyze sensor data, image streams, audio recordings, or other inputs collected by edge devices, enabling tasks such as object detection, anomaly detection, predictive maintenance, or environmental monitoring. By performing Al inference locally at the edge, customer applications 230 can minimize latency, reduce bandwidth consumption, and ensure data privacy and security, making them wel 1 -suited for latency-sensitive or bandwddth-constrained environments.

[0113] Moreover, customer applications 230 within the Edge Al Architecture often incorporate functionalities for data visualization, analytics, and decision support, enabling endusers to derive actionable insights and make informed decisions based on the processed data.Attorney Docket No. 67023.72WO01These applications provide interactive dashboards, reports, alerts, and visualization tools that enable users to monitor system performance, track key metrics, detect trends, and identify anomalies in real-time. By presenting actionable insights in a user-friendly and intuitive manner, customer applications 230 empower end-users to optimize operations, improve efficiency, and drive business value through data-driven decision-making.

[0114] Furthermore, customer applications 230 may integrate with external systems, cloud platforms, or third-party services through APIs, enabling seamless data exchange, interoperability, and integration with existing IT infrastructure. These integrations enable customer applications 230 to access additional data sources, leverage external services for specific functionalities (e.g., geolocation services, weather APIs, payment gateways), and synchronize data with enterprise systems or cloud-based applications. By integrating with external systems, customer applications 230 can extend their capabilities, enhance user experiences, and leverage existing investments in IT infrastructure and software solutions.

[0115] Native apps 231 can be utilized within the customer applications 230 discussed in the Edge Al Architecture to provide tailored experiences and enhanced functionalities for endusers across various platforms, including mobile devices, tablets, and desktop computers. Native apps 231 are developed specifically for a particular operating system (e.g., iOS, Android, Windows) and leverage the native features and capabilities of the platform to deliver optimized performance, seamless user experiences, and access to device-specific functionalities. Within the customer applications 230, native apps 231 serve several key purposes to enhance user engagement, productivity, and accessibility.

[0116] Firstly, native apps 231 enable users to access the functionality of the customer applications 230 directly from their mobile devices, providing on-the-go access to real-time insights, decision-making tools, and automation features. By leveraging the capabilities of mobile devices, such as GPS, camera, accelerometer, and push notifications, native apps 231 can deliver context-aware experiences and enable functionalities such as location-based sendees, image capture, barcode scanning, or push notifications. This enhances user engagement, enables faster decision-making, and facilitates seamless interaction with the edge Al system regardless of the user's location or device.

[0117] Moreover, native apps 231 within the customer applications 230 offer offline capabilities, enabling users to access critical functionalities and data even when they are notAttorney Docket No. 67023.72WO01connected to the internet or have limited network connectivity. By caching data locally on the device and implementing offline modes, native apps 231 ensure that users can continue to perform tasks, view information, and make decisions even in remote or offline environments. This is particularly beneficial for field workers, remote locations, or scenarios where network connectivity7is unreliable or intermittent, ensuring uninterrupted access to essential functionalities and data.

[0118] Additionally, native apps 231 provide a consistent and familiar user experience across different platforms, leveraging platform-specific design guidelines, navigation patterns, and user interface components. By adhering to platform conventions and design principles, native apps 231 ensure that users can navigate, interact with, and understand the application interface intuitively, regardless of the device or operating system they are using. This enhances usability, reduces learning curves, and increases user satisfaction, resulting in a more engaging and enjoyable experience for end-users.

[0119] Furthermore, native apps 231 enable seamless integration with device-specific features and functionalities, such as biometric authentication, voice commands, and hardware sensors. By leveraging these device capabilities, native apps 231 can enhance security, accessibility, and usability, enabling features such as fingerprint authentication, voice-controlled commands, or augmented reality7experiences. This not only enhances the overall user experience but also adds value to the customer applications 230 by leveraging the full potential of mobile devices and their capabilities.

[0120] lot containers 232, encompassing telemetry data and data orchestration, play a pivotal role within the customer applications 230 discussed in the Edge Al Architecture, facilitating seamless integration, data exchange, and orchestration of loT devices and sensor data. These containers serve as encapsulated environments for deploying, managing, and orchestrating loT devices and data streams, enabling end-to-end connectivity, data ingestion, processing, and analysis within the edge computing environment. Within the customer applications 230, lot containers 232 fulfill several key functions to enhance data-driven insights, decision-making, and automation capabilities.

[0121] Firstly, lot containers 232 enable the ingestion and processing of telemetry data generated by loT devices, sensors, and edge computing nodes deployed in the field. Telemetry data includes sensor readings, environmental measurements, device status updates, and otherAttorney Docket No. 67023.72WO01real-time data streams generated by loT devices. lot containers 232 provide lightweight, scalable runtime environments for collecting, aggregating, and processing telemetry data in real-time, enabling applications to analyze sensor data, detect patterns, and derive actionable insights for decision-making. By ingesting and processing telemetry data at the edge, lot containers 232 minimize latency, reduce bandwidth consumption, and enable real-time monitoring, control, and automation of loT devices and systems.

[0122] Moreover, lot containers 232 facilitate data orchestration and workflow automation within the edge computing environment, enabling end-to-end data processing pipelines and workflows to be implemented seamlessly. These containers provide tools, libraries, and runtime environments for orchestrating data workflows, managing data pipelines, and coordinating tasks such as data transformation, enrichment, and routing. Within the customer applications 230, lot containers 232 enable developers to design, deploy, and manage complex data processing pipelines that ingest data from multiple sources, apply Al algorithms for analysis, and generate insights or actions based on the processed data. By orchestrating data workflows, lot containers 232 streamline data processing, improve efficiency, and enable automation of routine tasks, enhancing the scalability and agility of edge Al applications.

[0123] Furthermore, lot containers 232 enable seamless integration and interoperability with external systems, cloud platforms, and third-party services through standardized interfaces and protocols 225. These containers provide connectors, adapters, and APIs for integrating with cloud-based loT platforms, enterprise systems, and other loT ecosystems, enabling seamless data exchange, synchronization, and integration. Within the customer applications 230, lot containers 232 enable data to be securely transmitted and synchronized with cloudbased storage, databases, analytics platforms, or visualization tools, enabling centralized data management, analysis, and reporting. By integrating with external systems, lot containers 232 enable customers to leverage existing investments in IT infrastructure, extend the capabilities of their edge Al applications, and unlock additional value from their loT deployments.

[0124] Al containers 240 serve as essential components within the customer applications 230 discussed in the Edge Al Architecture, providing encapsulated environments for deploying, managing, and executing Al models and algorithms at the edge. These containers enable end-users to leverage the pow er of artificial intelligence for real-time insights, decisionmaking, and automation directly within their applications, without the need for extensive AlAttorney Docket No. 67023.72WO01expertise or infrastructure. Within the customer applications 230, Al containers 240 fulfill several key functions to enhance data analysis, predictive capabilities, and automation.

[0125] Firstly, Al containers 240 enable the deployment and execution of pre-trained machine learning models and algorithms directly on edge devices, enabling real-time inference and analysis of data streams generated by loT devices and sensors. These containers provide runtime environments for loading, initializing, and executing Al models, handling tasks such as data preprocessing, model inference, and result output. By leveraging Al containers 240, customer applications 230 can perform tasks such as object detection, image classification, anomaly detection, predictive maintenance, or natural language processing directly at the edge, minimizing latency, reducing bandwidth consumption, and enabling real-time decision-making and automation.

[0126] Moreover, Al containers 240 facilitate the deployment and management of Al models and algorithms in heterogeneous edge computing environments, spanning a diverse range of hardware platforms, operating systems, and deployment scenarios. These containers provide portability, scalability', and consistency across different edge devices, enabling Al models to be deployed seamlessly on edge servers, gateways, or edge devices with resource constraints. Within the customer applications 230, Al containers 240 enable developers to build and deploy Al-driven functionalities that are tailored to specific edge computing requirements, ensuring compatibility, interoperability, and performance across diverse deployment environments.

[0127] Furthermore, Al containers 240 enable seamless integration with data orchestration, data processing, and edge computing workflows within the edge Al architecture. These containers provide interfaces, APIs, and connectors for integrating with lot containers 232, data processing pipelines, and orchestration frameworks, enabling end-to-end data analysis and automation workflows. Within the customer applications 230, Al containers 240 enable developers to incorporate Al-driven insights and actions into their edge computing workflows, enabling tasks such as automated decision-making, predictive analytics, or intelligent control based on real-time data streams. By integrating Al containers 240 with data orchestration and edge computing frameworks, customer applications 230 can leverage the full potential of Al technologies to enhance efficiency, productivity, and intelligence in diverse edge computing scenarios.Attorney Docket No. 67023.72WO01

[0128] Computer vision models 242 can be effectively utilized within the Al containers 240 discussed in the Edge Al Architecture to enable a wide range of visual perception tasks directly at the edge. These models are specifically designed to analyze and interpret visual data, such as images and videos, enabling applications to extract valuable insights, make informed decisions, and automate processes based on visual information. Within the Al containers 240, computer vision models 242 fulfill several key functions to enhance edge computing capabilities, including object detection, image classification, semantic segmentation, and more.

[0129] Firstly, computer vision models 242 enable object detection, allowing edge devices to identify and locate objects of interest within images or video streams in real-time. These models utilize deep learning techniques, such as convolutional neural networks (CNNs), to detect and localize objects within an image, providing bounding boxes or segmentation masks that indicate the presence and location of objects. By leveraging computer vision models 242 within Al containers 240, customer applications 230 can perform tasks such as object tracking, inventory management, or security monitoring directly at the edge, without relying on cloudbased processing or external servers.

[0130] Moreover, computer vision models 242 enable image classification, enabling edge devices to categorize images into predefined classes or categories based on their visual content. These models learn to recognize patterns and features within images, enabling applications to classify objects, scenes, or events with high accuracy. Within the Al containers 240, computer vision models 242 can classify images in real-time, enabling applications such as visual inspection, quality control, or content-based filtering directly at the edge. By leveraging image classification capabilities, customer applications 230 can automate decision-making processes, identify anomalies or defects, and optimize operations based on visual data.

[0131] Furthermore, computer vision models 242 enable semantic segmentation, allowing edge devices to segment images into meaningful regions and assign semantic labels to individual pixels or regions within an image. These models utilize advanced deep learning architectures, such as fully convolutional networks (FCNs) or U-Net, to perform pixel-wise classification and delineate objects or structures within an image. Within the Al containers 240, semantic segmentation models can segment images in real-time, enabling applications such as medical image analysis, environmental monitoring, or autonomous navigation directly at the edge. By leveraging semantic segmentation capabilities, customer applications 230 can extractAttorney Docket No. 67023.72WO01detailed information from visual data, enable fine-grained analysis, and support complex decision-making processes.

[0132] Additionally, computer vision models 242 enable facial recognition and biometric authentication, enabling edge devices to identify and verify' individuals based on their facial features or other biometric characteristics. These models utilize deep learning techniques to extract and analyze facial features, enabling applications to recognize faces, verify identities, and enforce access control policies. Within the Al containers 240, facial recognition models can perform real-time identification and authentication tasks, enabling applications such as access control systems, attendance tracking, or personalized user experiences directly at the edge. By leveraging facial recognition capabilities, customer applications 230 can enhance security, privacy, and user experiences in diverse deployment scenarios.

[0133] Large language models 243, such as GPT (Generative Pre-trained Transformer) models, can be effectively utilized within the Al containers 240 discussed in the Edge Al Architecture to enable natural language understanding, generation, and interaction directly at the edge. These models are specifically designed to process and generate human-like text based on input prompts, enabling applications to understand and generate natural language responses in real-time. Within the Al containers 240, large language models 243 fulfill several key functions to enhance edge computing capabilities, including conversational Al, text generation, language translation, and more.

[0134] Firstly, large language models 243 enable conversational Al. allowing edge devices to engage in natural language conversations with users and provide intelligent responses based on their input. These models utilize advanced natural language processing (NLP) techniques, such as transformer architectures, to understand the semantics and context of user queries and generate appropriate responses. Within the Al containers 240, large language models 243 can perform real-time conversational Al tasks, enabling applications such as virtual assistants, chatbots, or customer support agents to interact with users directly at the edge. By leveraging conversational Al capabilities, customer applications 230 can enhance user engagement, provide personalized experiences, and automate customer interactions in diverse deploy ment scenarios.

[0135] Moreover, large language models 243 enable text generation, enabling edge devices to generate human-like text based on input prompts or context. These models can generateAttorney Docket No. 67023.72WO01coherent and contextually relevant text across various domains, enabling applications to generate product descriptions, news articles, email responses, or creative content in real-time. Within the Al containers 240, large language models 243 can generate text dynamically, enabling applications such as content generation, summarization, or dialogue systems to create engaging and informative content directly at the edge. By leveraging text generation capabilities, customer applications 230 can automate content creation, streamline communication processes, and deliver personalized experiences to users in real-time.

[0136] Furthermore, large language models 243 enable language translation, allowing edge devices to translate text between different languages and facilitate multilingual communication in real-time. These models utilize machine learning algorithms to understand the semantics and syntax of text in one language and generate equivalent text in another language, enabling applications to break down language barriers and enable seamless communication across diverse linguistic communities. Within the Al containers 240, large language models 243 can perform language translation tasks, enabling applications such as translation services, multilingual chatbots, or internationalization tools to facilitate cross-cultural communication directly at the edge. By leveraging language translation capabilities, customer applications 230 can enhance accessibility, inclusivity, and global reach, enabling users to communicate and interact effectively in their preferred languages.

[0137] Additionally, large language models 243 enable sentiment analysis and text classification, allowing edge devices to analyze the sentiment and emotions expressed in textual data and classify text into predefined categories or topics. These models utilize machine learning techniques to identify patterns and features within text data, enabling applications to categorize text based on sentiment, sentiment, emotions, or topics. Within the Al containers 240, large language models 243 can perform sentiment analysis and text classification tasks, enabling applications such as social media monitoring, brand reputation management, or content moderation to analyze and classify textual data in real-time. By leveraging sentiment analysis and text classification capabilities, customer applications 230 can gain insights into user preferences, sentiments, and trends, enabling them to make data-driven decisions and optimize user experiences effectively.

[0138] A network stack 241 can be utilized within the Al containers 240 discussed in the Edge Al Architecture to enable communication, data exchange, and integration with external systems, cloud platforms, and other edge devices within the network. The network stack 241Attorney Docket No. 67023.72WO01encompasses a set of protocols, libraries, and services that facilitate network communication, including networking protocols such as TCP / IP. UDP, HTTP, and MQTT, as well as networking libraries, drivers 215, and middleware 222 components. Within the Al containers 240, the network stack 241 fulfills several key functions to enhance edge computing capabilities, including connectivity, data transmission, security, and interoperability.

[0139] Firstly, the network stack 241 enables connectivity and communication between Al containers 240 deployed on edge devices and external systems or cloud platforms over wired or wireless networks. The network stack 241 provides support for netw orking protocols such as TCP / IP and UDP, enabling edge devices to establish connections, exchange data packets, and communicate with remote servers or services. By leveraging the network stack 241. Al containers 240 can transmit data, receive commands, and synchronize state information with external systems, enabling seamless integration and interoperability within the edge computing ecosystem.

[0140] Moreover, the network stack 241 facilitates data transmission and streaming between Al containers 240 deployed on edge devices and other edge devices or sensors within the network. The network stack 241 provides support for communication protocols such as MQTT, CoAP. or AMQP, enabling edge devices to publish and subscribe to data streams, send notifications, or exchange messages in a lightweight and efficient manner. By leveraging the network stack 241, Al containers 240 can stream sensor data, telemetry data, or control messages between edge devices, enabling real-time data exchange and collaboration within distributed edge computing environments.

[0141] Furthermore, the network stack 241 enables security and encryption of data transmitted between Al containers 240 and external systems or edge devices within the network. The network stack 241 provides support for security protocols such as TLS / SSL, IPSec, or VPN, enabling edge devices to encrypt data, authenticate connections, and ensure data integrity and confidentiality during transmission. By leveraging the network stack 241 , Al containers 240 can establish secure and encrypted communication channels, protecting sensitive data and ensuring compliance with security and privacy regulations within the edge computing ecosystem.

[0142] Additionally, the network stack 241 facilitates interoperability and integration with edge computing frameworks, orchestration platforms, and management systems deployedAttorney Docket No. 67023.72WO01within the network. The network stack 241 provides support for standardized interfaces, APIs, and protocols such as RESTful APIs, gRPC, or OpenAPI. enabling Al containers 240 to interact with edge computing services, orchestration frameworks, and management platforms seamlessly. By leveraging the network stack 241, Al containers 240 can register with edge computing platforms, discover available services, and exchange metadata or control messages, enabling dynamic and flexible deployment of edge Al applications within the network.

[0143] Developer support 260 within the edge Al architecture encompasses various resources, tools, and sendees aimed at assisting developers in building, deploying, and managing edge Al applications effectively. This support typically includes access to large language model (LLM) libraries, GitHub repositories, and comprehensive documentation to facilitate the development process. Firstly, LLM support involves providing developers with access to pre-trained language models, such as GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), or T5 (Text-To-Text Transfer Transformer), tailored for specific natural language processing (NLP) tasks. These pre-trained models serve as powerful building blocks for developing natural language understanding (NLU) and generation functionalities within edge Al applications. Developer support 260 may include access to model repositories, model serving frameworks, and deployment tools that streamline the integration of LLMs into edge Al applications, enabling developers to leverage state-of-the-art NLP capabilities without the need for extensive training or expertise.

[0144] Moreover, GitHub use cases play a crucial role in developer support 260 by providing access to open-source repositories, sample code, and community contributions that accelerate the development and collaboration process. Edge Al architecture may include GitHub repositories containing example applications, code snippets, and best practices for implementing various edge computing and Al functionalities. Developers can leverage these resources to learn, experiment, and bootstrap their edge Al projects, benefiting from the collective knowledge and expertise of the developer community. Additionally, GitHub repositories may host contributions from platform developers, partners, and third-party developers, fostering collaboration, innovation, and knowledge-sharing within the edge Al ecosystem.

[0145] Furthermore, comprehensive documentation is essential for providing developers with guidance, tutorials, API references, and technical documentation that facilitate theAttorney Docket No. 67023.72WO01development, deployment, and maintenance of edge Al applications. Developer support 260 includes well-documented APIs, SDKs (Software Development Kits), and programming guides that outline the capabilities, usage patterns, and best practices for integrating edge Al functionalities into applications. Documentation may cover topics such as model deployment, inference optimization, data preprocessing, error handling, and troubleshooting, enabling developers to navigate the complexities of edge Al development effectively. Additionally, developer support 260 may include tutorials, code samples, and step-by-step guides that demonstrate common use cases, implementation patterns, and deployment strategies, empowering developers to build robust, scalable, and efficient edge Al applications.

[0146] Developer environment 260s within the edge Al architecture are designed to provide developers with the necessary tools, resources, and infrastructure to build, test, and deploy edge Al applications efficiently. These environments typically encompass a range of command-line toolchains, diagnostic tools, build and package tools, as well as support for multiple programming languages, enabling developers to work seamlessly across diverse development scenarios.

[0147] Command-line toolchains are essential components of developer environment 260s, providing developers with command-line interfaces (CLIs) and utilities for performing various development tasks, such as code compilation, package management, version control, and deployment. These toolchains often include popular command-line interfaces, such as Git for version control, Docker for container management, and package managers like npm or pip for dependency management. By leveraging command-line toolchains, developers can automate repetitive tasks, streamline development workflows, and maintain consistency across development environments, enhancing productivity and collaboration within development teams.

[0148] Diagnostic tools play a crucial role in developer environment 260s by providing developers with insights into application performance, resource utilization, and runtime behavior, enabling them to identify and troubleshoot issues effectively. These tools may include profiling tools, debuggers, monitoring agents, and logging frameworks that enable developers to analyze application behavior, diagnose performance bottlenecks, and optimize code for efficiency. By leveraging diagnostic tools, developers can ensure the reliability, performance, and scalability of edge Al applications, enabling them to deliver high-quality solutions that meet user expectations and business requirements.Attorney Docket No. 67023.72WO01

[0149] Build and package tools are essential components of developer environment 260s, enabling developers to compile, bundle, and package edge Al applications for deployment on edge devices or cloud platforms. These tools may include build automation frameworks, packaging utilities, and deployment scripts that streamline the build process and generate deployable artifacts, such as container images, executable binaries, or firmware packages. By¬ leveraging build and package tools, developers can automate the build process, ensure reproducibility, and simplify the deployment of edge Al applications across diverse deployment environments, enabling faster time-to-market and more efficient release cycles.

[0150] Additionally, developer environment 260s provide support for multiple programming languages, enabling developers to choose the language and framework that best suits their development preferences and requirements. These environments may include support for popular programming languages such as Python, JavaScript, Java, or C / C++, as well as frameworks and libraries commonly used in edge Al development, such as TensorFlow, PyTorch. OpenCV. or scikit-leam. By providing multi-language support, developer environment 260s accommodate diverse development scenarios and enable developers to leverage their existing skills and expertise in building edge Al applications, fostering innovation and flexibility within development teams.

[0151] Although a specific embodiment for edge video processing suitable for earn ing out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 2, any of a variety- of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the specific layout of components may vary depending on the application and / or deployment. The elements depicted in FIG. 2 may also be interchangeable with other elements of FIGS. 1 and 3 - 9 as required to realize a particularly desired embodiment.

[0152] Referring to FIG. 3, a conceptual system block chart 300 for edge video processing in accordance with various embodiments of the disclosure is shown. Computer vision, an interdisciplinary field merging computer science and artificial intelligence, revolutionizes our ability- to interpret and understand the visual world. One of its pivotal applications is object detection, wherein computers identify and locate objects within images or video frames. It all begins with capturing images or video frames using cameras or similar devices. Preprocessing techniques are then employed to enhance image quality, mitigate noise, and normalize lighting conditions. These steps optimize subsequent object detection algorithms.Attorney Docket No. 67023.72WO01

[0153] Feature extraction often occurs. Algorithms analyze images to identify patterns like edges, comers, textures, colors, and shapes — distinctive attributes aiding in object recognition. Object localization follows, determining objects' spatial placement via techniques like bounding boxes. Next comes object classification. Algorithms are trained on labeled datasets using machine learning methods such as convolutional neural networks (CNNs). By analyzing features within bounding boxes, they predict objects' identities, categorizing them into predefined classes like "person" or "car."

[0154] Post-processing steps refine results. False positives may be filtered out, overlapping detections merged, or geometric constraints applied to ensure consistency. Finally, output visualization presents results. Detected objects and their classifications overlay the original image or appear in a separate format, often featuring bounding boxes and class labels. In real-world scenarios, such as surveillance, autonomous vehicles, medical imaging, and robotics, computer vision's object detection capabilities find numerous applications. Advances in deep learning and hardware accelerate these systems, empowering them to detect and recognize objects with unprecedented accuracy and efficiency across various environments.

[0155] A network with Internet of Things (loT) devices typically consists of interconnected physical objects or "things" embedded with sensors, actuators, and communication capabilities. These devices collect and exchange data over the internet or other networks, enabling them to interact with each other and w ith cloud-based services or applications. The structure of such a network involves several key components. At the edge, loT devices gather data from their surroundings, such as temperature, humidity, motion, or other environmental variables. These devices may include sensors, actuators, microcontrollers, and communication modules. The collected data is then transmitted to a local gateway or hub, which serves as an intermediary betw een the loT devices and the broader network infrastructure. The gateway aggregates and processes data, performs protocol translations, and manages communication with the cloud or other remote services. From the gateway, data is sent to the cloud, where it is stored, analyzed, and acted upon by applications or services. Cloud platforms provide storage 330, processing, and analytics capabilities, enabling organizations to derive insights from loT data and automate decision-making processes. In addition to the cloud, some loT deployments may utilize edge computing resources, where data processing and analysis occur closer to the source of data generation, reducing latency and bandwidth requirements. Overall, the network architecture of an loT deployment involves distributed computing resources, spanning from edge devices toAttorney Docket No. 67023.72WO01cloud-based platforms, to enable the seamless collection, processing, and utilization of data from connected devices.

[0156] Computer vision processing on a network with loT devices involves integrating cameras or imaging sensors into the loT ecosystem to capture visual data, which is then processed and analyzed to extract meaningful insights. At the edge of the network, loT devices equipped with cameras or image sensors capture images or video streams. These devices may include surveillance cameras, drones, or smart cameras embedded in various environments. Once captured at the edge, the visual data is then transmitted to the cloud or central server for analysis and storage 330. In the cloud, powerful computational resources and machine learning algorithms can be applied to perform tasks such as object recognition, scene understanding, and anomaly detection. The insights derived from the visual data can then be used to trigger actions, provide real-time alerts, or generate reports to support decision-making processes. By combining computer vision processing with loT devices and network infrastructure, organizations can unlock new opportunities for applications such as smart surveillance, industrial automation, retail analytics, and more, enabling them to extract valuable insights from visual data collected in various environments.

[0157] Video data 313 collected at the edge of a network for computer vision processing encompasses a range of visual information captured by cameras or imaging sensors deployed in the field. This data typically includes sequences of frames, each containing pixel-level information representing the scene observed by the camera at a particular moment in time. Within these frames, various elements may be present, such as objects, people, animals, vehicles, and environmental features like buildings, roads, or vegetation. Additionally, the video data 313 may include temporal information, capturing the dynamics of the scene over time, such as motion patterns, changes in object positions, and interactions between objects. The quality and resolution of the video data 313 can vary depending on factors like camera specifications, lighting conditions, and environmental factors. Preprocessing techniques may be applied at the edge to enhance the quality of the video data 313, mitigate noise, or compress the data for efficient transmission over the network. Ultimately, the video data 313 collected at the edge serves as the foundation for computer vision algorithms to analyze, extract insights, and make sense of the visual world in real-time or near-real-time applications, enabling various use cases such as surveillance, monitoring, object detection, and activity recognition.Attorney Docket No. 67023.72WO01

[0158] Alongside the visual information contained in video data 313 collected at the edge of a network, various types of metadata 312 can provide valuable context and additional insights. This metadata 312 may include timestamps indicating the exact moment each frame was captured, enabling temporal analysis and synchronization with other data sources. Geospatial metadata 312, such as GPS coordinates or location tags, can provide information about where the video was recorded, facilitating spatial analysis and mapping applications. Other types of metadata 312 may include camera settings, such as exposure, focal length, and aperture, which can affect the quality and characteristics of the video. Additionally, metadata 312 tags or annotations may be added manually or automatically to identify specific objects, events, or attributes within the video, aiding in search 354, indexing, and retrieval tasks. Metadata 312 plays a crucial role in enhancing the interpretability, usability, and utility of video data 313 for computer vision applications, enabling more informed analysis and decisionmaking processes.

[0159] In addition to basic metadata 312 like timestamps and geospatial information, video data 313 often includes context data 311 that provides deeper insights into the content and relationships within the scenes captured. Video features, such as motion vectors, optical flow, and keyframe selection, offer valuable context about the dynamics and visual characteristics of the video, aiding in tasks like action recognition and scene understanding. Object relations and interactions can be inferred from metadata 312 indicating the spatial and temporal relationships between objects, such as bounding boxes, trajectories, or semantic segmentation masks. These relationships help in analyzing object behavior, tracking movements, and identifying complex events involving multiple objects. Semantic annotations or tags associated with objects or regions within the video provide context about their identities, attributes, or actions, enabling more meaningful interpretation and analysis. Context data 311 may also include environmental factors like weather conditions, lighting conditions, or ambient noise, which influence the appearance and behavior of objects in the video. By incorporating context data 311 related to video features, object relations, and environmental factors, computer vision systems can achieve a deeper understanding of the visual content, enabling more accurate and context-aware analysis for a wide range of applications, from surveillance and security to autonomous systems and augmented reality7.

[0160] In addition to context data 311, metadata 312, and video data 313, various other ty pes of data may be captured at the edge of the network for processing in computer visionAttorney Docket No. 67023.72WO01systems. This includes sensor data from additional environmental sensors deployed alongside cameras, such as temperature sensors, humidity sensors, accelerometers, or LiDAR sensors. Integrating sensor data wi th video data 313 provides a richer understanding of the environment, enabling correlations between visual cues and physical parameters. Furthermore, contextual information such as contextual awareness can be derived from data sources like contextual knowledge data 310bases, historical data, or contextual signals from connected devices or systems. This contextual awareness enhances the interpretability and relevance of the analysis results, enabling more context-aware and adaptive computer vision applications. Additionally, metadata 312 about network conditions, device status, or system health metrics may be collected to monitor and optimize the performance of the edge computing infrastructure. By leveraging a diverse range of data sources at the edge, computer vision systems can achieve more comprehensive and robust analysis, enabling intelligent decision-making and automation in various domains.

[0161] To process all the edge data 310 — comprising context data 311. metadata 312, and video data 313 — with concatenation 320 before storage 330, you would first need to gather and organize the different types of data from their respective sources at the edge of the network. This involves collecting video streams, sensor readings, metadata 312, and any other relevant contextual information. Once collected, the data needs to be formatted and structured in a way that facilitates concatenation 320. This may involve standardizing data formats, aligning timestamps, and organizing metadata 312 to correspond with the associated video frames or sensor readings.

[0162] Next, you would concatenate the different types of data into a single unified dataset. This could be achieved by merging the data streams based on common identifiers or timestamps, ensuring that each data point retains its relevant contextual information. For example, sensor readings could be appended to corresponding video frames, and metadata 312 could be attached as additional attributes. After concatenation 320, the processed data can be stored in a suitable storage 330 system, such as a database or distributed file system. By consolidating the various types of edge data 310 into a single dataset, you simplify storage 330 management and streamline subsequent analysis tasks. Additionally, storing concatenated data allows for easier retrieval and querying, as all relevant information is stored together.

[0163] This concatenated edge data 310 can then be accessed and utilized by computer vision systems for tasks such as object detection, activity recognition, or anomaly detection.Attorney Docket No. 67023.72WO01The comprehensive dataset provides a holistic view of the environment, enabling more accurate and context-aware analysis. Moreover, by processing and concatenating edge data 310 before storage 330, you lay the foundation for more efficient and effective utilization of the data in downstream applications and workflows. Using the concatenated edge data 310 for query search 354 related to stored data involves several key steps. Initially, the data undergoes feature extraction, where relevant features are extracted from the video frames, sensor readings, metadata 312, and contextual information. This process involves leveraging computer vision algorithms to extract visual features from the video data 313 and considering metadata 312 and context data 311 as additional input features. Simultaneously, features are extracted from sensor data using techniques tailored to the specific sensor modalities. Once features are extracted, they are transformed into a lower-dimensional embedding space through embedding generation. This step aims to map the high-dimensional feature representations into a more compact and semantically meaningful space. Techniques like principal component analysis (PC A) or autoencoders are commonly used for dimensionality reduction and feature compression, allowing for efficient storage 330 and retrieval of data.

[0164] After embedding generation, the embeddings 340 are organized into an index structure to facilitate efficient similarity7search 354. Indexing techniques such as locality sensitive hashing (LSH) or tree-based structures like KD-trees or ball trees are employed to organize the embeddings 340 for fast retrieval based on similarity metrics. When a query 352 is initiated, features are extracted from the query data and transformed into the same embedding space as the stored data. A similarity7search 354 is then performed within the index to identify the most similar embeddings 340 to the query. This process effectively retrieves stored data entries that are most relevant or similar to the query criteria.

[0165] Once the similar embeddings 340 are retrieved, they are ranked based on their similarity scores or distance metrics. Finally, the ranked results are presented to the user, providing relevant data entries that match the query criteria. This approach enables efficient content-based search 354 and retrieval, facilitating tasks such as object recognition, scene understanding, and anomaly detection in various applications such as video surveillance, smart environments, and industrial monitoring. By leveraging embeddings 340 derived from concatenated edge data 310 and employing similarity search 354 techniques, organizations can effectively retrieve relevant information from stored data in response to user queries, enhancing decision-making processes and enabling actionable insights.Attorney Docket No. 67023.72WO01

[0166] Generating a semantic search 354 on the concatenated edge data 310 based on a query 352, available memory 353, and any collected sources 351 can involve understanding the underlying semantics of both the query 352 and the stored data. Unlike traditional keywordbased searches, semantic searches aim to comprehend the meaning and context of the search query to retrieve more relevant results. To achieve this, natural language processing (NLP) techniques can be employed to analyze and interpret the query text, extracting semantic entities, relationships, and concepts.

[0167] Once the query' is processed, semantic embeddings 340 are generated to represent the semantic content of the query. This involves encoding the semantic information extracted from the query text into a numerical vector representation using techniques such as word embeddings 340 or sentence embeddings 340. These semantic embeddings 340 capture the semantic similarity' and relationships between words or phrases in the query, enabling more nuanced search queries that go beyond simple keyword matching.

[0168] Similarly, the stored data, including video data 313, sensor readings, metadata 312, and contextual information, is preprocessed to extract semantic features and representations. This may involve analyzing textual metadata 312, contextual information, and associated annotations to derive semantic attributes and relationships. For video data 313, semantic features can be extracted using computer vision techniques to identify objects, scenes, actions, and other semantic elements within the video frames.

[0169] Once both the query and the stored data are represented in a semantic embedding space, a semantic similarity search 354 is performed to identify' the most relevant data entnes that match the semantic content of the query. This involves computing the semantic similarity between the query' embeddings 340 and the embeddings 340 of the stored data entries, considering the semantic context and relationships encoded in the embeddings 340. Techniques such as cosine similarity or Euclidean distance are commonly used to measure the similarity between embeddings 340.

[0170] The retrieved data entries are then ranked based on their semantic similarity to the query, with higher-ranked matches considered more relevant to the query semantics. These results are presented to the user as semantic search results, providing a more intuitive and contextually relevant representation of the stored data based on the semantic content of the query. By leveraging semantic search techniques, organizations can enhance theAttorney Docket No. 67023.72WO01discoverability and relevance of information within their data repositories, enabling more effective decision-making and knowledge discovery processes.

[0171] Although a specific embodiment for a block chart for edge video processing suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 3, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the specific configuration or data flows may be adjusted based on the specific application or deployment desired. The elements depicted in FIG. 3 may also be interchangeable with other elements of FIGS. 1 - 2 and 4A - 9 as required to realize a particularly desired embodiment.

[0172] Referring to FIG. 4A, a conceptual illustration of a sample image 410 within one or more sets of image data in accordance with an embodiment of the disclosure is shown. Utilizing the example of an edge camera, such a camera associated with an automobile capturing image data for use in one or more assisted driving features, the sample image 410 shows a scene which may be captured by an automobile while driving or parked at a stop light. In this sample image 410, there may be a number of items which will have unique levels of motion and or priority when processing. In the embodiment depicted in FIG. 4A, the sample image 410 comprises a number of pedestrians 412, other automobiles / vehicles 415, and landmark objects 418 such as street signs and traffic lights. Once captured, the sample image 410 may be processed by an edge device without having to send all of the data to a centralized server.

[0173] In various embodiments, similar video data may be captured from a variety of deployments. For example, images may be captured by one or more edge cameras from retail environments, or other environments that may be desired to be monitored. As those skilled in the art will recognize, the specific device and / or environment can vary' and cover multiple locations.

[0174] Although a specific embodiment for a sample image within one or more sets of image data suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 4A, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the number of elements, subjects, and / or objects that are tracked can vary based on the environmentAttorney Docket No. 67023.72WO01monitored. The elements depicted in FIG. 4A may also be interchangeable with other elements of FIGS. 1 - 3 and 4B - 9 as required to realize a particularly desired embodiment.

[0175] Referring to FIG. 4B, a conceptual illustration of a sample image 420 within one or more sets of image data processed to recognize a plurality of subjects in accordance with an embodiment of the disclosure is shown. In various embodiments, the automobile or other image processing device may analyze the sample image 420 and generate a plurality of bounding boxes 422, 425, 428 around detected subjects of interest such as, but not limited to, pedestrians 412, other automobiles / vehicles 415, and landmark objects 418 such as street signs and traffic lights. As those skilled in the art will recognize, the use of a bounding box 422, 425, 428 is not the only method of selecting an area of an image comprising a detected object. It is contemplated that an image mask, such as a pixel-level image mask may be suitable for use in segmentation a sample image 420.

[0176] In most embodiments, the bounding box 422, 425, 428 will be comprised of a rectangle that encases the detected object and has a size that includes the maximum height and width of the detected object. Based on the desired application and capabilities of the image capturing device, the generation of the bounding box 422, 425, 428 may be generated by a specialized logic or by the low-power, fast-response machine learning logic. Additionally, it is contemplated that each recognized subject within the sample image 420 may be segmented with a unique bounding box and / or image mask and may be grouped together based on recognized subject type.

[0177] In more embodiments, the bounding boxes or other segmentation can be formatted as metadata, which can be processed into one or more contextual cues, data, or other inputs that can be passed to a large language model. This can allow for the generation and extraction of contextual data, as well contextual features. These contextual features can be utilized to decide if a state should change, if a behavior is detected, and / or the potential compression levels to be used.

[0178] Although a specific embodiment for a sample image within one or more sets of image data processed to recognize a plurality of subjects suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG.4B, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the sample image 420 can be processed inAttorney Docket No. 67023.72WO01different ways beyond bounding boxes to extract detection of objects, subjects, etc. The elements depicted in FIG. 4B may also be interchangeable with other elements of FIGS. 1 -4A and 5 - 9 as required to realize a particularly desired embodiment.

[0179] Referring to FIG. 5, a flowchart depicting a process 500 for generating a result based on a query’ in accordance with various embodiments of the disclosure is shown. In a number of embodiments, the process 500 can receive edge data (block 510). The edge data can be comprised of various types of data. In certain embodiments, the edge data can comprise video data, metadata, and / or context data. The video data can be captured by one or more camera devices, such as loT devices.

[0180] In many embodiments, the process 500 can process edge data (block 520). The edge data can be processed in a variety of ways. In some embodiments, the edge data can be processed to extract, capture, group or otherwise process video data from one or more environments. The processing can be done to pre-format or change the edge data for subsequent processing. In still more embodiments, the processing can be associated with utilizing the edge data as an input into one or more computer vision models or the like.

[0181] In more embodiments, the process 500 can store edge data (block 530). The compressed data can be stored within a local storage device. However, in some embodiments, the compressed data can be transferred to a centralized server or storage device for further processing or storage.

[0182] In further embodiments, the process 500 can format edge data for queries (block 540). As discussed in more detail below, the formatting can include concatenation, compression, or other modification that can be utilized for subsequent query requests. As those skilled in the art will recognize, the specific formatting can vary on the type of data and queries that may be received.

[0183] In additional embodiments, the process 500 can receive a query' (block 550). The query can be received from a requesting device elsewhere on a network. For example, a network may comprise a variety' of devices wherein the deployment has a plurality of centralized devices and a plurality of edge devices. The query may be generated and / or transmitted from a centralized device to an edge device.Attorney Docket No. 67023.72WO01

[0184] In still more embodiments, the process 500 can generate a result based on the received query (block 560). The result can be generated by stored edge data. In various embodiments, the result can be formatted based on the type of query received and / or based on the type of stored edge data or format.

[0185] In certain embodiments, the process 500 can transmit the result (block 570). The generated result can be transmitted to the requesting device in some embodiments. In more embodiments, the results can be transmitted to a separate device or logic that is indicated within the query'.

[0186] Although a specific embodiment for a matnx of behavior detection suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 5, any of a variety' of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the edge data can be a plurality of video image data as well as meta data and or other output data from various machine learning processes. The elements depicted in FIG. 5 may also be interchangeable with other elements of FIGS. 1 - 4B and 6 - 9 as required to realize a particularly desired embodiment.

[0187] Referring to FIG. 6, a flowchart depicting a process 600 for generating a query in accordance with various embodiments of the disclosure is shown. In many embodiments, the process 600 can receive edge data from a plurality of network devices (block 610). The edge data can be comprised of various ty pes of data. In certain embodiments, the edge data can comprise video data, metadata, and / or context data. The video data can be captured by one or more camera devices, such as loT devices.

[0188] In a number of embodiments, the process 600 can process the edge data through a computer vision (CV) model (block 620). The edge data can be processed in a variety of ways. In some embodiments, the edge data can be processed to extract, capture, group or otherwise process video data from one or more environments. The processing can be done to pre-format or change the edge data for subsequent processing. In still more embodiments, the processing can be associated with utilizing the edge data as an input into one or more computer vision models or the like.

[0189] In some embodiments, inputting edge data, such as video data into a computer vision model involves a series of steps to convert visual information into a format that the model can understand and process. Initially, the video data can be captured or formatted in aAttorney Docket No. 67023.72WO01frame-by-frame method, with each frame containing visual information. These frames may then be preprocessed to enhance their quality and reduce noise, which may involve techniques such as resizing, normalization, and color space conversion. Next, the frames are fed into the computer vision model(s), either sequentially or in batches, depending on the architecture and requirements of the model. The model can then analyze each frame, extracting features and patterns to make predictions or perform specific tasks, such as object detection, tracking, or action recognition.

[0190] In more embodiments, the process 600 can process at least the CV model output data through a large language model (LLM) (block 630). The CV model output data may be configured as detected objects and other tracking data. In various embodiments, inputting object detection data and tracking data into a large language model can involve transforming the structured information derived from these tasks into a format compatible with natural language processing. For example, the object detection data, which typically consists of bounding boxes and class labels for detected objects in each frame, needs to be organized into a coherent representation. This may involve encoding the spatial coordinates of the bounding boxes, along with corresponding class labels and confidence scores, into a structured format such as JSON or XML. Similarly, tracking data, which tracks the movement of objects across multiple frames, needs to be processed to capture temporal dependencies and trajectories of objects. This could involve encoding object identities, positions, and velocities over time.

[0191] Once the data is structured, it can be fed into a large language model, either as input tokens or as structured inputs alongside textual descriptions or prompts. The model can then analyze the object detection and tracking data, generating textual descriptions, summaries, or responses based on the detected objects and their movements. Additionally, the language model may be fine-tuned on task-specific datasets to improve its ability to understand and generate text based on object detection and tracking data. In some embodiments, the output can be formatted as semantic or contextual data.

[0192] In additional embodiments, the process 600 can concatenate at least the edge data, CV model data, and LLM model output data into a compressed data stream (block 640). In some embodiments, the use of concatenation can allow for the combining or pairing of different types of data into a single unified dataset. This could be achieved by merging the data streams based on common identifiers or timestamps, ensuring that each data point retains its relevantAttorney Docket No. 67023.72WO01contextual information. In various embodiments, this concatenated data stream can be considered a “compressed’" data stream.

[0193] In further embodiments, the process 600 can store the compressed data stream (block 650). The compressed data can be stored within a local storage device. However, in some embodiments, the compressed data can be transferred to a centralized server or storage device for further processing or storage. In certain embodiments, the data stream may simply be a concatenated data stream that is merged or unified, but not compressed.

[0194] In still more embodiments, the process 600 can format the compressed data stream into one or more embeddings (block 660). In some embodiments, the data stream can be preprocessed to ensure its compatibility with the embedding model. This preprocessing might include tasks such as tokenization, where the data stream is divided into individual tokens such as words or characters, and normalization, where variations like capitalization or punctuation are standardized. Following preprocessing, the tokens can be converted into numerical representations suitable for the embedding model. This conversion could involve techniques like one-hot encoding or more sophisticated methods like word embeddings such as Word2Vec or GloVe. Once the tokens are transformed into numerical vectors, they can be fed into the embedding model to generate embeddings. Depending on the specific requirements of the application, the data stream may be formatted into a single embedding representing the entire stream, or multiple embeddings representing different aspects or segments of the data.

[0195] In various embodiments, the process 600 can receive a query (block 670). The query can be a text-based query received from a user or device operated by a user. However, the query can be preprocessed to be formatted into a proper structure for utilization by one or more logics, etc. The query can also be generated in an automated fashion or through one or more application programming interfaces (APIs).

[0196] In yet further embodiments, the process 600 can generate a query result based at least on the one or more embeddings (block 680). The query can be processed through one or more logics which can access at least embedded data. The result, which is often based on the query, can be formatted into a structure similar to the one of the query itself. In additional embodiments, the query may also include data indicating the type of response structure desired.

[0197] In certain embodiments, the process 600 can transmit the query (block 690). The transmission may be to another device or logic that can be configured to alert a user,Attorney Docket No. 67023.72WO01administrator, or other logic regarding the response. In some embodiments, the transmission can be broadcast so one or more devices configured to receive the response can do so. However, in additional embodiments, the transmission may be directed by the query or to a specific known device or logic that is a known destination for that type of response.

[0198] Although a specific embodiment for a process for processing edge data for behaviors suitable for earning out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 6, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the location of the stored data may be done on an edge device within a network, or an intermediate device to avoid transmission of excess data across the network. The elements depicted in FIG.6 may also be interchangeable with other elements of FIGS. 1 - 5 and 7 - 9 as required to realize a particularly desired embodiment.

[0199] Referring to FIG. 7, a flowchart depicting a process 700 for generating embedded data in accordance with various embodiments of the disclosure is shown. In many embodiments, the process 700 can receive edge data from a plurality of network devices (block 710). For example, a system may have a plurality of camera devices deployed at one or more locations that are configured to track a variety of environments, etc. However, edge data may be received from various storage devices as well.

[0200] In a number of embodiments, the process 700 can parse the edge data for video data, metadata, and context data (block 720). As described above, edge data may be comprised of various data. In certain embodiments, the edge data may be raw, unprocessed data, that, in response to processing, can be parsed into separate video data, metadata, and / or context data. The parsing can be done prior to storage and may be done by one or more devices on the edge of a network deployment.

[0201] In more embodiments, the process 700 can extract one or more contextual features from the edge data (block 730). The extraction of contextual features from edge data can be done by parsing the edge data for context data. In some embodiments, the extraction can be comprised of utilizing at least a portion of the edge data as an input into one or more computer vision models. The computer vision model(s) can be configured to output contextual features that can be extracted from the edge data.Attorney Docket No. 67023.72WO01

[0202] In additional embodiments, the process 700 can determine an event based on the contextual features (block 740). In leveraging data captured on edge devices like video cameras for event detection, a process can involve several intermediate steps. First, the data collected by the edge devices, such as video feeds, needs to be processed and may need to be preprocessed. Once the data is prepared for processing, it can undergo one or more preprocessing steps such as noise reduction, compression, and possibly feature extraction to streamline further analysis. Next, machine learning or computer vision algorithms can be applied to the preprocessed data to detect patterns or anomalies indicative of the target event. These algorithms might range from simple rule-based systems to complex deep learning models, depending on the specificity and complexity of the event being detected. These events can be selected that allow for facilitating swift decision-making and responses in various domains like security, surveillance, and industrial monitoring.

[0203] In further embodiments, the process 700 can classify the at least one behavior as suspicious or non-suspicious (block 750). As described above, some deployments may be configured within environments that are retail based or the like. In these embodiments, there may be events that are monitored that can be classified. The classifications of the events or behaviors may be tuned to focus on, for example, suspicious or non-suspicious behaviors. These behaviors can be related to various activities that can be occurring within the deployment environment, such as a retail store, working area, or other monitored location. However, those skilled in the art will recognize that any type of event may be classified based on the application desired.

[0204] In still more embodiments, the process 700 can select a compression level for storage of the edge data (block 760). Based on the classification, an event may be deemed as important or non-important. For example, a deployed system can be configured to exist in either an idle or alert state. Events associated with an idle state may be configured with a higher compression level compared to events associated with an alert state. In this way, events that may be considered more important can have less compression applied to their associated edge data, which can allow for greater review later. Likewise, by marking one or more events as unimportant and subj ect to a higher compression rate, the amount of data that needs to be stored and / or sent back to a centralized server is reduced, improving the efficiency of the overall system and process 700.Attorney Docket No. 67023.72WO01

[0205] In various embodiments, the process 700 can compress the edge data (block 770). As those skilled in the art will recognize, the type of compression utilized can vary greatly. For example, for video data, the image data can be compressed using one or more video compression algorithms. The metadata or contextual data may also be compressed using some text or basic data compression methods. The amount of compression utilized can vary7based on the type of data being compressed or in response to one or more user, administrator, or logic settings.

[0206] In yet further embodiments, the process 700 can store the compressed edge data (block 780). The compressed data can be stored within a local storage device. However, in some embodiments, the compressed data can be transferred to a centralized server or storage device for further processing or storage. In still more embodiments, the network may comprise one or more intermediate devices that can be utilized to store the compressed data.

[0207] In still additional embodiments, the process 700 can format the stored edge data into embedded data (block 790). In some embodiments, edge data can be converted to embedded data by transforming the raw visual information captured in the edge data into a condensed, numerical representation known as embeddings. Initially, the edge data is typically segmented into individual video frames, and each frame is processed using techniques such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to extract high-level features. These features capture spatial and temporal patterns within the video frames, encoding information about objects, scenes, and motion dynamics. Subsequently, these extracted features are transformed into dense, fixed-length vectors known as embeddings using methods like dimensionality reduction or learned representations. These embeddings effectively summarize the essential information contained within the video data in a compact numerical format, facilitating efficient storage, analysis, and comparison.

[0208] Although a specific embodiment for a process for determining alert states from edge data suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 7, any of a variety’ of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the generation of the embedded data can be done at an earlier step prior to storage. The elements depicted in FIG.7 may also be interchangeable with other elements of FIGS. 1 - 6 and 8 - 9 as required to realize a particularly^ desired embodiment.Attorney Docket No. 67023.72WO01

[0209] Referring to FIG. 8, a flowchart depicting a process 800 for formatting data for query processing in accordance with various embodiments of the disclosure is shown. In many embodiments, the process 800 can receive a series of video frames for processing (block 810). For example, a system may have a plurality of camera devices deployed at one or more locations that are configured to track a variety of environments, etc. However, video data may¬ be received from various storage devices as well.

[0210] In a number of embodiments, the process 800 can process the series of video frames through a computer vision (CV) model (block 820). As described above, edge data may be comprised of various data. In certain embodiments, the edge data may be raw, unprocessed data, that, in response to processing, can be parsed into separate video data, metadata, and / or context data. The parsing can be done prior to storage and may be done by one or more devices on the edge of a network deployment.

[0211] In some embodiments, inputting edge data, such as video data into a computer vision model involves a series of steps to convert visual information into a format that the model can understand and process. Initially, the video data can be captured or formatted in a frame-by-frame method, with each frame containing visual information. These frames may-then be preprocessed to enhance their quality and reduce noise, which may involve techniques such as resizing, normalization, and color space conversion. Next, the frames are fed into the computer vision model(s), either sequentially or in batches, depending on the architecture and requirements of the model. The model can then analyze each frame, extracting features and patterns to make predictions or perform specific tasks, such as object detection, tracking, or action recognition.

[0212] In more embodiments, the process 800 can process at least the CV model output data through a large language model (LLM) (block 830). The CV model output data may be configured as detected objects and other tracking data. In various embodiments, inputting object detection data and tracking data into a large language model can involve transforming the structured information derived from these tasks into a format compatible with natural language processing. For example, the object detection data, which typically consists of bounding boxes and class labels for detected objects in each frame, needs to be organized into a coherent representation. This may involve encoding the spatial coordinates of the bounding boxes, along with corresponding class labels and confidence scores, into a structured format such as JSON or XML. Similarly, tracking data, which tracks the movement of objects acrossAttorney Docket No. 67023.72WO01multiple frames, needs to be processed to capture temporal dependencies and trajectories of objects. This could involve encoding object identities, positions, and velocities over time.

[0213] In additional embodiments, the process 800 can concatenate at least the edge data, CV model data, and LLM model output data into a compressed data stream (block 840). In some embodiments, the output data from the CV model and LLM model can be fused or integrated. This integration can take various forms, such as correlating the detected objects with the generated textual descriptions to enhance understanding and context. Additionally, the outputs may undergo further analysis, such as sentiment analysis or topic modeling, to extract deeper insights from the combined visual and textual information. Finally, the integrated output can undergo evaluation against predefined criteria or objectives. This evaluation may involve assessing the accuracy of object detection, the coherence of generated textual descriptions, or the overall relevance of the information extracted from the video frames.

[0214] In further embodiments, the process 800 can classify one or more events within a portion of the video frame (block 850). Classification can be in response to the specific application or deployment that is being used. For example, in retail environments, classifications can be related to suspicious or non-suspicious behaviors. In driving related environments, the events can be associated with objects on the roadway or pedestrians near the roadway, etc. However, those skilled in the art will recognize that any classification can be determined based on the application desired.

[0215] In still more embodiments, the process 800 can determine a compression method associated with the event (block 860). As described above, based on the classification, an event may be deemed as important or non-important. For example, a system can configured to exist in an either an idle state or an alerted state. Events associated with an idle state may be configured with a higher compression level compared to events associated with an alert state. In this way, events that may be considered more important can have less compression applied to their associated edge data, which can allow for greater review later. Likewise, by marking one or more events as unimportant and subject to a higher compression rate, the amount of data that needs to be stored and / or sent back to a centralized server is reduced, improving the efficiency of the overall system and process 800.Attorney Docket No. 67023.72WO01

[0216] In some embodiments, a determination can be made on if all events have been processed (block 865). If it is determined that all events have not been processed, then the process 800 can again classify one or more events within a portion of the video frame (block 850).

[0217] However, if it is determined that all events have been processed, then, in various embodiments, the process 800 can compress the series of video frames based on the determined compression method (block 870). As those skilled in the art will recognize, the type of compression utilized can vary' greatly. For example, for video data, the image data can be compressed using one or more video compression algorithms. The metadata or contextual data may also be compressed using some text or basic data compression methods. The amount of compression utilized can vary based on the type of data being compressed or in response to one or more user, administrator, or logic settings.

[0218] In yet further embodiments, the process 800 can store the compressed series of video frames (block 880). The compressed data can be stored within a local storage device. However, in some embodiments, the compressed data can be transferred to a centralized server or storage device for further processing or storage. In still more embodiments, the network may comprise one or more intermediate devices that can be utilized to store the compressed data.

[0219] In still yet more embodiments, the process 800 can format the compressed series of video frames for query’ processing (block 890). Some embodiments may involve structuring the information extracted from the video frames into a structured data format, such as JSON or XML, where each frame is associated with metadata including object labels, textual descriptions, timestamps, and any other relevant attributes. This structured representation can allow for efficient querying and retrieval of specific information based on user inquiries. Additionally, keyframes or representative frames can be selected from the video sequence and annotated with descriptive tags or keywords, enabling quick reference and retrieval during query' processing. Furthermore, the processed video frames can be indexed and linked to a knowledge graph or ontology, facilitating semantic query ing and inference to derive insights beyond the explicit content of the frames.

[0220] Although a specific embodiment for a process for adjusting a state based on an event classification suitable for carry ing out the various steps, processes, methods, and operationsAttorney Docket No. 67023.72WO01described herein is discussed with respect to FIG. 8, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the specific types of data that is output from a computer vision model or large language model can vary based on the specific application desired. The elements depicted in FIG. 8 may also be interchangeable with other elements of FIGS. 1 - 7 and 9 as required to realize a particularly desired embodiment.

[0221] Referring to FIG. 9, a device suitable for utilizing an edge processing logic 924 in accordance with various embodiments of the disclosure is shown. The embodiment of the conceptual block diagram depicted in FIG. 9 can illustrate a conventional network device, such as, but not limited to, a server, switch, wireless LAN controller, access point, computer, workstation, desktop computer, laptop, tablet, network appliance, e-reader, smartphone, camera, or other loT computing device, and can be utilized to execute any of the application and / or logic components presented herein. The device 900 may, in many non-limiting examples, correspond to physical devices or to virtual resources described herein.

[0222] In many embodiments, the device 900 may include an environment 902 such as a baseboard or “motherboard,” in physical embodiments that can be configured as a printed circuit board with a multitude of components or devices connected by way of a system bus or other electrical communication paths. Conceptually, in virtualized embodiments, the environment 902 may be a virtual environment that encompasses and executes the remaining components and resources of the device 900. In more embodiments, one or more processors 904, such as, but not limited to, central processing units (“CPUs”) can be configured to operate in conjunction with a chipset 906. The processor(s) 904 can be standard programmable CPUs that perform arithmetic and logical operations necessary for the operation of the device 900.

[0223] In a number of embodiments, the processor(s) 904 can perform one or more operations by transitioning from one discrete, physical state to the next through the manipulation of switching elements that differentiate between and change these states. Switching elements generally include electronic circuits that maintain one of two binary states, such as flip-flops, and electronic circuits that provide an output state based on the logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements can be combined to create more complex logic circuits, including registers, adders-subtractors, arithmetic logic units, floating-point units, and the like.Attorney Docket No. 67023.72WO01

[0224] In various embodiments, the chipset 906 may provide an interface between the processor(s) 904 and the remainder of the components and devices within the environment 902. The chipset 906 can provide an interface to a random-access memory (“RAM”) 908, which can be used as the main memory in the device 900 in some embodiments. The chipset 906 can further be configured to provide an interface to a computer-readable storage medium such as a read-only memory (“ROM”) 910 or non-volatile RAM (“NVRAM”) for storing basic routines that can help with various tasks such as, but not limited to, starting up the device 900 and / or transferring information between the various components and devices. The ROM 910 or NVRAM can also store other application components necessary for the operation of the device 900 in accordance with various embodiments described herein.

[0225] Additional embodiments of the device 900 can be configured to operate in a networked environment using logical connections to remote computing devices and computer systems through a network, such as the network 940. The chipset 906 can include functionality for providing network connectivity through a network interface card (“NIC”) 912, which may comprise a gigabit Ethernet adapter or similar component. The NIC 912 can be capable of connecting the device 900 to other devices over the network 940. It is contemplated that multiple NICs 912 may be present in the device 900, connecting the device to other ty pes of networks and remote systems.

[0226] In further embodiments, the device 900 can be connected to a storage 918 that provides non-volatile storage for data accessible by the device 900. The storage 918 can, for instance, store an operating system 920. applications 922. The storage 918 can be connected to the environment 902 through a storage controller 914 connected to the chipset 906. In certain embodiments, the storage 918 can consist of one or more physical storage units. The storage controller 914 can interface with the physical storage units through a serial attached SCSI (“SAS”) interface, a serial advanced technology attachment (“SATA”) interface, a fiber channel (“FC”) interface, or other type of interface for physically connecting and transferring data between computers and physical storage units.

[0227] The device 900 can store data within the storage 918 by transforming the physical state of the physical storage units to reflect the information being stored. The specific transformation of physical state can depend on various factors. Examples of such factors can include, but are not limited to, the technology used to implement the physical storage units, whether the storage 918 is characterized as primary or secondary storage, and the like.Attorney Docket No. 67023.72WO01

[0228] In many more embodiments, the device 900 can store information within the storage 918 by issuing instructions through the storage controller 914 to alter the magnetic characteristics of a particular location within a magnetic disk drive unit, the reflective or refractive characteristics of a particular location in an optical storage unit, or the electrical characteristics of a particular capacitor, transistor, or other discrete component in a solid-state storage unit, or the like. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this description. The device 900 can further read or access information from the storage 918 by detecting the physical states or characteristics of one or more particular locations within the physical storage units.

[0229] In addition to the storage 918 described above, the device 900 can have access to other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable storage media is any available media that provides for the non-transitory storage of data and that can be accessed by the device 900. In some examples, the operations performed by a cloud computing network, and or any components included therein, may be supported by one or more devices similar to device 900. Stated otherwise, some or all of the operations performed by the cloud computing network, and or any components included therein, may be performed by one or more devices 900 operating in a cloud-based arrangement.

[0230] By way of example, and not limitation, computer-readable storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology. Computer-readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically-erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technology7, compact disc ROM (“CD-ROM”), digital versatile disk (“DVD”), high definition DVD (“HD-DVD”), BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information in a non-transitory fashion.

[0231] As mentioned briefly above, the storage 918 can store an operating system 920 utilized to control the operation of the device 900. According to one embodiment, the operating system comprises the LINUX operating system. According to another embodiment, the operating system comprises the WINDOWS® SERVER operating system from MICROSOFTAttorney Docket No. 67023.72WO01Corporation of Redmond, Washington. According to further embodiments, the operating system can comprise the UNIX operating system or one of its variants. It should be appreciated that other operating systems can also be utilized. The storage 918 can store other system or application programs and data utilized by the device 900.

[0232] In many additional embodiments, the storage 918 or other computer-readable storage media is encoded with computer-executable instructions which, when loaded into the device 900, may transform it from a general-purpose computing system into a special-purpose computer capable of implementing the embodiments described herein. These computerexecutable instructions may be stored as application 922 and transform the device 900 by specifying how the processor(s) 904 can transition between states, as described above. In some embodiments, the device 900 has access to computer-readable storage media storing computerexecutable instructions which, when executed by the device 900, perform the various processes described above with regard to FIGS. 1 - 8. In certain embodiments, the device 900 can also include computer-readable storage media having instructions stored thereupon for performing any of the other computer-implemented operations described herein.

[0233] In many further embodiments, the device 900 may include an edge processing logic 924. The edge processing logic 924 can be configured to perform one or more of the various steps, processes, operations, and / or other methods that are described above. Often, the edge processing logic 924 can be a set of instructions stored within a non-volatile memory that, when executed by the controller(s) / processor(s) 904 can carry out these steps, etc. In some embodiments, the edge processing logic 924 may be a client application that resides on a network-connected device, such as. but not limited to, a server, switch, personal or mobile computing device in a single or distributed arrangement.

[0234] In some embodiments, edge data 928 can be captured by devices such as cameras in a network refers to the information gathered at the periphery or "edge" of the network, typically at the point where data is generated or collected. This data is often unprocessed or minimally processed and can encompass a wide range of information depending on the specific application and context. In the case of cameras, edge data may consist of raw pixel values representing images or video frames captured by the camera sensor. These images or frames contain visual information about the environment or scene being monitored, including objects, people, motion, and other relevant details.Attorney Docket No. 67023.72WO01

[0235] Beyond raw pixel data, edge data from cameras may also include metadata providing additional context or information about the captured images or video. This metadata can vary depending on the capabilities of the camera and the configuration of the system but commonly includes timestamps indicating when the data was captured, geolocation data specifying the location where the data was collected, camera settings such as exposure and focus parameters, and any other relevant sensor readings such as temperature or humidity.

[0236] Furthermore, edge data may encompass derived features or preprocessed information extracted from the raw data at the edge device itself. For example, edge devices equipped with onboard processing capabilities may perform tasks such as object detection, motion tracking, or image enhancement directly on the captured data, generating intermediate representations or higher-level features that are more compact and informative than the raw pixel data. These derived features can significantly reduce the amount of data that needs to be transmitted over the network and enable more efficient and scalable processing and analysis.

[0237] In various embodiments, event data 930 can comprise information relating to specific events that are captured by the edge data 928. Event data 930 can be formatted into various textual structures including, but not limited to, text, or states associated with one or more systems, etc. Event data 930 can be related to events that are currently being observed / processed or may be stored events related to the past. Other aspects of event data 930

[0238] In a number of embodiments, embedded data 932 may comprise the data stored by devices such as cameras in a network, embedded data 932 may be configured as a condensed, numerical representation of the raw sensor readings or processed information, ty pically derived through techniques like dimensionality reduction or learned representations. This embedded data 932 captures essential features and patterns within the original data, enabling efficient storage, transmission, and analysis while preserving the critical information necessary' for downstream tasks. For example, in the case of video data, embedded representations summarize spatial and temporal features extracted from individual frames, facilitating tasks such as classification, similarity' retrieval, and content-based recommendation systems. By transforming raw data into compact embeddings, edge devices can optimize resource usage and enable more scalable and interpretable analysis of the captured information, driving advancements in various applications such as surveillance, smart cities, and industrial monitoring.Attorney Docket No. 67023.72WO01

[0239] In still further embodiments, the device 900 can also include one or more input / output controllers 916 for receiving and processing input from anumber of input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus, or other type of input device. Similarly, an input / output controller 916 can be configured to provide output to a display, such as a computer monitor, a flat panel display, a digital projector, a printer, or other type of output device. Those skilled in the art will recognize that the device 900 might not include all of the components shown in FIG. 9 and can include other components that are not explicitly shown in FIG. 9 or might utilize an architecture completely different than that shown in FIG. 9.

[0240] As described above, the device 900 may support a virtualization layer, such as one or more virtual resources executing on the device 900. In some examples, the virtualization layer may be supported by a hypervisor that provides one or more virtual machines running on the device 900 to perform functions described herein. The virtualization layer may generally support a virtual resource that performs at least a portion of the techniques described herein.

[0241] Finally, in numerous additional embodiments, data may be processed into a format usable by a machine-learning model 926 (e.g., feature vectors), and or other preprocessing techniques. The machine-learning ("ML”) model 926 may be any type of ML model, such as supervised models, reinforcement models, and / or unsupervised models. The ML model 926 may include one or more of linear regression models, logistic regression models, decision trees, Naive Bayes models, neural netw orks, k-means cluster models, random forest models, and / or other types of ML models 926.

[0242] The ML model(s) 926 can be configured to generate inferences to make predictions or draw conclusions from data. An inference can be considered the output of a process of applying a model to new data. This can occur by learning from at least the edge data 928. the event data 930. and embedded data 932. These predictions are based on patterns and relationships discovered within the data. To generate an inference, the trained model can take input data and produce a prediction or a decision. The input data can be in various forms, such as images, audio, text, or numerical data, depending on the type of problem the model was trained to solve. The output of the model can also vary depending on the problem, and can be a single number, a probability distribution, a set of labels, a decision about an action to take, etc. Ground truth for the ML model(s) 926 may be generated by human / administrator verifications or may compare predicted outcomes with actual outcomes.Attorney Docket No. 67023.72WO01

[0243] ML model(s) 926 can, in many embodiments, be a computer vision model or a large language model. A computer vision model is often a computational framework designed to interpret and understand visual information, typically in the form of images or video data. These models employ algorithms and techniques to process visual input, extract meaningful features, and perform tasks such as object detection, image classification, and scene understanding. They are trained on large datasets to leam patterns and relationships within the visual data, enabling them to make accurate predictions and inferences about the content of images or videos. On the other hand, a large language model is a type of artificial intelligence model trained to understand and generate human-like text. These models utilize deep learning architectures, such as transformers, to process and generate natural language based on the context provided in input text. They can perform tasks such as language translation, text summarization, and conversational dialogue generation, and they are trained on extensive textual corpora to develop a broad understanding of language semantics and syntax. Both computer vision and large language models represent powerful tools for processing and understanding different modalities of data, enabling a wide range of applications across various domains.

[0244] Unsupervised learning techniques are valuable when labeled data is scarce or unavailable. Clustering algorithms, such as k-means clustering or hierarchical clustering, can group similar traffic flows together based on their feature representations. These clusters can then be analyzed to identify different types of traffic or anomalous behavior within the network. Unsupervised learning provides a data-driven approach to traffic classification, allowing network administrators to discover hidden patterns and structures within the traffic data.

[0245] Deep learning models, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have demonstrated impressive performance in traffic classification tasks. CNNs can automatically leam hierarchical representations of traffic features, while RNNs can capture temporal dependencies in sequential data, such as packet traces or session logs. These deep learning architectures excel at handling complex, highdimensional traffic data and can adaptively leam from large-scale datasets to improve classification accuracy.

[0246] In certain embodiments the ML models(s) 926 can be combined into multiple ML models 926 to enhance classification performance. By aggregating predictions from diverse classifiers or feature subsets, ensemble methods can mitigate the limitations ofAttorney Docket No. 67023.72WO01individual models and improve generalization across different traffic scenarios. These can provide a robust and scalable approach to traffic classification, particularly in dynamic network environments where event patterns evolve over time.

[0247] Although a specific embodiment for a device suitable for utilizing an edge processing logic suitable for carrying out the various steps, processes, methods, and operations described herein is discussed with respect to FIG. 9, any of a variety of systems and / or processes may be utilized in accordance with embodiments of the disclosure. For example, the device 900 may be in a virtual environment, or it may be distributed across a variety' of network devices. The elements depicted in FIG. 9 may also be interchangeable with other elements of FIGS. 1 - 9 as required to realize a particularly desired embodiment.

[0248] Although the present disclosure has been described in certain specific aspects, many additional modifications and variations would be apparent to those skilled in the art. In particular, any of the various processes described above can be performed in alternative sequences and / or in parallel (on the same or on different computing devices) in order to achieve similar results in a manner that is more appropriate to the requirements of a specific application. It is therefore to be understood that the present disclosure can be practiced other than specifically described without departing from the scope and spirit of the present disclosure. Thus, embodiments of the present disclosure should be considered in all respects as illustrative and not restrictive. It will be evident to the person skilled in the art to freely combine several or all of the embodiments discussed here as deemed suitable for a specific application of the disclosure. Throughout this disclosure, terms like “advantageous”, “exemplary” or “example” indicate elements or dimensions which are particularly suitable (but not essential) to the disclosure or an embodiment thereof and may' be modified wherever deemed suitable by the skilled person, except where expressly required. Accordingly, the scope of the disclosure should be determined not by the embodiments illustrated, but by the appended claims and their equivalents.

[0249] Any reference to an element being made in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more.” All structural and functional equivalents to the elements of the above-described preferred embodiment and additional embodiments as regarded by those of ordinary skill in the art are hereby expressly incorporated by reference and are intended to be encompassed by7the present claims.Attorney Docket No. 67023.72WO01

[0250] Moreover, no requirement exists for a system or method to address each and every problem sought to be resolved by the present disclosure, for solutions to such problems to be encompassed by the present claims. Furthermore, no element, component, or method step in the present disclosure is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. Various changes and modifications in form, material, workpiece, and fabrication material detail can be made, without departing from the spirit and scope of the present disclosure, as set forth in the appended claims, as might be apparent to those of ordinary skill in the art, are also encompassed by the present disclosure.

Claims

Attorney Docket No. 67023.72WO01CLAIMSWhat is claimed is:

1. A device, comprising:a processor;a network interface controller configured to provide access to a network; and a memory communicatively coupled to the processor, wherein the memory comprises an edge processing logic that is configured to:receive edge data from a pl urali ty of network devices;store the edge data;format the edge data for a query;receive a query; andgenerate a result based on the received query.

2. The device of claim 1, wherein the edge processing logic is further configured to process the edge data prior to being stored.

3. The device of claim 2, wherein the processing includes processing the edge data through at least a computer vision (CV) model to generate CV model output data.

4. The device of claim 3, wherein the processing further includes passing the CV model output data to a large language model (LLM) to generate LLM model output data.

5. The device of claim 4, wherein the edge processing logic is further configured to concatenate at least the edge data, CV model output data, and LLM model output data into a compressed data stream.

6. The device of claim 5, wherein the edge processing logic stores the compressed data stream.

7. The device of claim 6, wherein the edge processing logic is further configured to format the compressed data stream into one or more embeddings.Attorney Docket No. 67023.72WO018. The device of claim 7, wherein generating the query result is based on at least the one or more embeddings.

9. The device of claim 8, wherein the query is received from a requesting device.

10. The device of claim 9, wherein the edge processing logic is further configured to transmit the result to the requesting device.

11. The device of claim 1, wherein the device exists within a networked system of devices comprising centralized devices and edge devices.

12. The device of claim 11, wherein the device is an edge device.

13. The device of claim 12, wherein the edge processing logic is further configured to compress the processed edge data prior to storage.

14. The device of claim 13, wherein storing the processed edge data comprises transmitting the processed edge data to a centralized device.

15. The device of claim 14, wherein compression of the processed edge data is based on at least one detected event.

16. A device, comprising:a processor;a network interface controller configured to provide access to a network; and a memory communicatively coupled to the processor, wherein the memory comprises an edge processing logic that is configured to:receive edge data from a plural, ly of network devices;parse the edge data;extract one or more contextual features from the edge data;detect at least one behavior based on the one or more contextual features; and compress the processed edge data based on the at least one behavior; wherein the device:Attorney Docket No. 67023.72WO01exists within a networked system of devices comprising centralized devices and edge devices; andis an edge device.

17. The device of claim 16, wherein the compressed edge data is stored on a centralized device.

18. The device of claim 16, wherein the edge processing logic is further configured to store the compressed edge data.

19. The device of claim 18, wherein the edge processing logic is further configured to format the stored edge data into embedded data.

20. A method, comprising:receiving edge data from a plurality of network devices;storing the edge data;formatting the edge data for a query;receiving a query ; andgenerating a result based on the received query.

Images