Camera data asynchronous task execution system and method based on instrumentalization capability set

Abstract

The invention provides a camera data asynchronous task execution system based on a instrumentalization capability set. The camera data asynchronous task execution system comprises a data processing module, a tool packaging module, an intention recognition and authentication module, a session context retrieval module, a retrieval and fusion module, a business decision and strategy application module, a response generation module, a response return module and an evidence recording module. The invention further provides a camera data asynchronous task execution method based on the instrumentalized capability set, and the problems of repeated calculation, result tracing, equipment reporting, privacy compliance and the like under high concurrency are solved by combining the instrumentalized capability set, Agent-driven decision based on a large model, SLA / resource awareness scheduling, end-cloud collaborative reporting and tampering-free evidence chain recording.

Description

Technical Field

[0001] This invention belongs to the field of video surveillance and intelligent event processing, and in particular relates to a system and method for asynchronous task execution of camera data based on a set of tool-based capabilities. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and smart security, the deployment of surveillance cameras in urban governance, traffic monitoring, park security, commercial operations, and home security scenarios continues to expand. A large number of cameras continuously generate massive video streams and event logs, and this video data is characterized by "long-term storage but infrequent active use." Traditional monitoring systems generally adopt a "passive playback + manual retrieval" working mode: only when an alarm occurs, a work order is received, or a manual request is made, do supervisors or maintenance personnel retrieve and play back historical videos to verify facts or extract evidence. Meanwhile, with the application of AI capabilities in areas such as target detection, behavior analysis, and face / license plate recognition, the industry is gradually attempting to evolve secondary video applications (such as automatic editing, alarm enhancement, statistical analysis, and automatic evidence collection) towards real-time and automation. However, existing technologies and practices, when faced with massive, continuous video data streams, multi-tenancy, diverse task demands, and compliance audit requirements, expose a series of technical and operational bottlenecks, affecting system efficiency, timeliness, and reliability, necessitating new, systematic solutions for improvement. Specific problems with existing technologies: 1. Low data utilization and high labor costs (1) The vast majority of video content is in a passive "cold data" state for a long time, and is only manually retrieved and played back at critical moments (accidents, complaints, evidence collection), resulting in a large amount of data being idle and wasting storage, management and retrieval costs. (2) Manual playback and retrieval are time-consuming and labor-intensive, rely on human judgment and are highly subjective, inefficient and prone to human oversight, and cannot meet the real-time and consistency requirements of large-scale scenarios. 2. Repetitive calculation leads to resource waste (1) In high-concurrency alarm or multi-task parallel scenarios, the system often repeatedly decodes, extracts frames, calculates features and performs AI inference on the same time window or the same camera segment, resulting in a large amount of redundant consumption of CPU / GPU computing power, memory and uplink bandwidth. (2) Existing platforms generally lack a unified identification and sharing mechanism for intermediate products (such as decoded frames, feature vectors, detection results, etc.), which cannot effectively reuse existing calculation results, resulting in repetitive labor and increased costs. 3. The results of asynchronous distributed execution are not traceable and are difficult to audit. (1) In order to improve throughput and avoid blocking the main link, the platform widely adopts asynchronous distribution of tasks to edge or cloud nodes for execution. However, the execution process, tool / model version, source and change of intermediate products are not recorded in a standardized way, and the source and processing link of task results are difficult to trace, which affects the credibility of the results.

[0003] (2) In judicial evidence collection or compliance audit scenarios, the lack of tamper-proof call evidence (such as input / output hashes, tool / model versions, timestamps and signatures) will weaken the probative value of the results in legal or regulatory procedures. 4. Device reporting strategy is out of sync with central task priority (1) Cameras / edge devices report data on a timed or event-driven basis under the default strategy, but these reporting strategies are usually not linked to the real-time task priority, SLA or sudden evidence collection needs of the central platform. As a result, when critical tasks come, it is impossible to obtain high frame rate or high quality video clips in time, affecting the speed of evidence collection and response. (2) In scenarios with fluctuating network bandwidth, offline or bandwidth-limited scenarios, the central control over the device is insufficient, and it is unable to issue short-term reporting strategies or rollback strategies as needed, thereby weakening the system's ability to guarantee time-sensitive tasks. 5. Static functional combinations and weak adaptability to diverse needs (1) Combining multiple functions such as editing, detection, desensitization, aggregation and synchronization often relies on manual assembly or predefined static processes, lacking the ability to dynamically generate the optimal processing link based on task SLA, privacy policy, network and resource status. (2) It is difficult to achieve automatic combination of compliance toolchains with the least privilege in multi-tenant scenarios, which brings privacy leakage risks or compliance challenges.

[0004] 6. AI inference results lack reproducibility and model governance capabilities. (1) AI outputs often only retain the final result, lacking key metadata such as confidence level, model version, inference environment and input snapshot, making it difficult to reproduce the result and to retrospectively verify the model upgrade / change, thus hindering model governance and compliance audit. (2) In the event of disputes or legal needs, it is impossible to provide a complete chain of evidence to prove the generation process and credibility of the result. 7. Insufficient intermediate product management and edge-cloud collaboration strategy (1) Under the edge-cloud hybrid deployment, there is a lack of a cost-time-privacy comprehensive decision-making mechanism for "when to compute at the edge, when to upload to the cloud, and when to save intermediate products", making it difficult to balance between meeting SLA, controlling costs and protecting privacy. (2) The index retrieval, cache hit, expiration recycling and on-demand retrieval mechanism of intermediate products is imperfect, affecting the reuse efficiency of intermediate products, and thus affecting the overall system performance. Summary of the Invention

[0005] This invention provides a system and method for asynchronous task execution of camera data based on a set of tool-based capabilities. By combining a set of tool-based capabilities, agent-driven decision-making based on a large model, SLA / resource-aware scheduling, edge-cloud collaborative reporting, and tamper-proof evidence chain recording, it solves problems such as repetitive calculations, result traceability, device reporting, and privacy compliance under high concurrency.

[0006] The purpose of this invention is to provide an agent-driven system and method for the orchestration, scheduling, and auditable execution of asynchronous camera data tasks based on a tool-based capability set, in order to solve the aforementioned problems. By encapsulating system capabilities into discoverable, describable "tools" with permissions and SLA estimations, the agent dynamically combines toolchains from a policy library based on task intent. The Planner converts the toolchains into asynchronously executable plans. The Scheduler adaptively adjusts the batch processing window and merges shared intermediate products based on task SLAs, system resources, and network status to reduce redundant computation. The Evidence module records immutable metadata (hash / signature / timestamp, etc.) to build an evidence chain for auditability during each tool invocation. The DeviceManager supports the central system issuing temporary reporting commands to camera devices to ensure the source of high-priority task data. The system also supports features such as least privilege deduction, multi-tenant privacy grading, tool version management, and playback reproduction.

[0007] The technical solution of the present invention is as follows: An asynchronous task execution system for camera data based on a set of tool-based capabilities includes: The data processing module is used to perform structured processing on camera data to obtain underlying logs and store them in the underlying log database; The tool encapsulation module encapsulates system capabilities into several tools and stores them in the tool database. The intent recognition and authentication module is used to receive user requests and perform user request parsing and permission verification to obtain the user's task intent. The session context retrieval module is used to retrieve session history, user configuration, or cache based on the user request, and construct context-aided understanding and response. The retrieval and fusion module is used to perform multimodal retrieval and comparison in the tool database according to the user's intent, and aggregate necessary information to generate a toolchain; The business decision-making and strategy application module adaptively adjusts the batch processing window based on task SLA, system resources and network status, and determines the final response strategy based on the toolchain and business rules according to the intermediate product reuse and merging strategy. The response generation module is used to generate an asynchronous execution plan based on the final response strategy and to call a model or template from the underlying log database to generate the final content. The response module is returned, which is used to display the final content; The evidence recording module is used to record immutable metadata and store it as a chain of evidence when tools are invoked and intermediate products are generated / referenced.

[0008] Preferably, in the camera data asynchronous task execution system based on a set of tool-based capabilities, the tools include date and time, system information, user information, device information, weather, people database, conversations, events, statistics, editing, downloading, reminders, searching, and others.

[0009] Preferably, in the asynchronous task execution system for camera data based on a set of tool-based capabilities, the metadata includes input / output hashes, tool and model versions, execution nodes, timestamps, and signatures.

[0010] Preferably, in the asynchronous task execution system for camera data based on a set of tool-based capabilities, the metadata is stored in a chained hash or trusted evidence storage manner.

[0011] The asynchronous task execution method for camera data based on a set of tool capabilities includes the following steps: The camera data is structured to obtain the underlying logs and stored in the underlying log database; The system capabilities are encapsulated into several tools and stored in a tool database; Receive user requests and perform user request parsing and permission verification to obtain the user's task intent; Based on the user request, retrieve session history, user configuration, or cache to construct a context to aid understanding and response; Based on the user's intent, a multimodal search and comparison is performed in the tool database to aggregate necessary information and generate a toolchain; The batch processing window is adaptively adjusted based on task SLA, system resources and network status, and the final response strategy is determined based on the toolchain and business rules, using intermediate product reuse and merging strategies. An asynchronous execution plan is generated based on the final response strategy, and the final content is generated and displayed by calling the model or template from the underlying log database. When tools are invoked and intermediate products are generated / referenced, immutable metadata is recorded and stored as a chain of evidence.

[0012] The present invention has the following beneficial effects: Reduce redundant computation and bandwidth consumption: Avoid repeated decoding / feature extraction within the same time window by using intermediate product reuse and merging strategies; 2. Ensure the timeliness of critical tasks: By implementing reporting strategies through edge-cloud collaboration, critical tasks can obtain high-quality data segments; 3. Auditable and reproducible: The entire process of tool invocation is recorded with a chain of evidence (input hash, output hash, tool version and signature), facilitating verification and legal evidence collection; 4. Flexible and scalable: The tool suite enables seamless registration, canary rollout, and dynamic combination of new features by agents; 5. Privacy Compliance: Supports de-identification / least privilege control based on tenant policies to meet multi-tenant compliance requirements.

[0013] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0014] Figure 1 This is a flowchart of the asynchronous task execution method for camera data based on a set of tool-based capabilities provided by the present invention; Figure 2 This is a tool illustration in one embodiment of the asynchronous task execution method for camera data based on a set of tool-based capabilities provided by the present invention; Figure 3 This is a data call diagram in one embodiment of the asynchronous task execution method for camera data based on a set of tool-based capabilities provided by the present invention; Figure 4 This is a video editing flowchart in one embodiment of the asynchronous task execution method for camera data based on a set of tool-based capabilities provided by the present invention; Figure 5 This is a schematic diagram of a specific process in one embodiment of the asynchronous task execution method for camera data based on a set of tool-based capabilities provided by the present invention. Detailed Implementation

[0015] The present invention will now be described in further detail with reference to specific embodiments, so that those skilled in the art can implement it based on the description.

[0016] This invention aims to provide an agent-driven system and method for the orchestration, scheduling, and auditable execution of asynchronous tasks for camera data, based on a tool-based capability set. Its overall objective is to improve data utilization efficiency, ensure the timeliness of critical tasks, guarantee the traceability and compliance of the processing chain, and achieve refined resource and privacy management in multi-tenant and constrained network environments, in the context of large-scale camera deployment and massive video data. To achieve this overall goal, this invention specifically includes the following objectives: 1. Reducing redundant computation and network overhead: By encapsulating platform capabilities into "tools" with input / output descriptions and SLA estimations, the Planner / Scheduler (based on a large model) identifies shareable intermediate products and merges similar / overlapping tasks, avoiding multiple decoding, frame extraction, and feature calculations for the same time window or segment, thereby significantly reducing CPU / GPU consumption and uplink bandwidth usage. 2. Achieve task-driven dynamic orchestration and automated expansion: The agent, based on a large model, dynamically combines toolchains and generates execution plans according to task intent, supporting automatic tool discovery, registration, canary release, and version management. This allows new capabilities to be seamlessly registered and automatically orchestrated by the agent according to policies, thereby reducing manual assembly processes and improving the system's flexibility and scalability.

[0017] 3. Enhance the timeliness of critical tasks and ensure reliable data sources. 4. The Scheduler adaptively adjusts the batch processing window and execution position based on task SLA, system load, and network status. The Device Manager (based on a message middleware such as EMQX) issues temporary reporting or priority instructions to the device, ensuring that high-priority or urgent evidence collection tasks can obtain high-quality video clips in a timely manner. 5. An immutable chain of evidence is established to support auditing and reproduction. Each time a tool is called and intermediate product is generated / referenced, metadata such as input / output hashes, tool and model versions, execution nodes, timestamps, and signatures are recorded. This metadata is stored using chained hashing or trusted evidence storage, forming a searchable and immutable traceability chain to meet the needs of judicial evidence collection and compliance auditing, and supports experiment / playback reproduction based on records. 6. End-to-end privacy and least privilege protection is provided. Permission tags and tenant policies are introduced during tool registration and Agent generation stages, supporting least privilege derivation, tool replacement (such as automatic selection of de-identified tools), and multi-tenant isolated storage to ensure functionality while meeting privacy and regulatory constraints.

[0018] Improve system response performance and intermediate product reuse rate Implement multi-layered caching (short-term cache within the tool, edge / nearline mid-product cache, and long-term archive) at the tool and system levels, combined with indexing and eviction strategies, to improve cache hit rate and shorten task response latency; while ensuring cache consistency and evidence chain association. 7. Support governance, replay, and model version management. Implement version management, canary release, and rollback strategies for the tool and model. Combined with version information recorded in the evidence chain, it is possible to replay and reproduce the processing environment and results at the time of the dispute or audit requirement, improving model governance and auditability. This invention provides an asynchronous task execution system for camera data based on a set of tool-based capabilities, which includes: The data processing module is used to perform structured processing on camera data to obtain underlying logs and store them in the underlying log database; The tool encapsulation module encapsulates system capabilities into several tools and stores them in the tool database. The intent recognition and authentication module is used to receive user requests and perform user request parsing and permission verification to obtain the user's task intent. The session context retrieval module is used to retrieve session history, user configuration, or cache based on the user request, and construct context-aided understanding and response. The retrieval and fusion module is used to perform multimodal retrieval and comparison in the tool database according to the user's intent, and aggregate necessary information to generate a toolchain; The business decision-making and strategy application module adaptively adjusts the batch processing window based on task SLA, system resources and network status, and determines the final response strategy based on the toolchain and business rules according to the intermediate product reuse and merging strategy. The response generation module is used to generate an asynchronous execution plan based on the final response strategy and to call a model or template from the underlying log database to generate the final content. The response module is returned, which is used to display the final content; The evidence recording module is used to record immutable metadata and store it as a chain of evidence when tools are invoked and intermediate products are generated / referenced.

[0019] The tools include date and time, system information, user information, device information, weather, people database, conversations, events, statistics, clipping, downloading, reminders, searching, and others.

[0020] The metadata includes input / output hashes, tool and model versions, execution nodes, timestamps, and signatures.

[0021] The metadata is stored in a chained hash or trusted evidence storage manner.

[0022] This invention also provides a method for asynchronous task execution of camera data based on a set of tool-based capabilities, which includes the following steps: The camera data is structured to obtain the underlying logs and stored in the underlying log database; The system capabilities are encapsulated into several tools and stored in a tool database; Receive user requests and perform user request parsing and permission verification to obtain the user's task intent; Based on the user request, retrieve session history, user configuration, or cache to construct a context to aid understanding and response; Based on the user's intent, a multimodal search and comparison is performed in the tool database to aggregate necessary information and generate a toolchain; The batch processing window is adaptively adjusted based on task SLA, system resources and network status, and the final response strategy is determined based on the toolchain and business rules, using intermediate product reuse and merging strategies. An asynchronous execution plan is generated based on the final response strategy, and the final content is generated and displayed by calling the model or template from the underlying log database. When tools are invoked and intermediate products are generated / referenced, immutable metadata is recorded and stored as a chain of evidence.

[0023] The cameras structure and upload specific events (faces, human figures, motion detection, timed keyframes, etc.) to the Agent platform. The Agent platform then stores the data in its database (all subsequent Q&A will be processed based on the Alog generated by these cameras). A series of tools process the underlying Alog, such as event statistics tools, event query tools, and keyframe image search tools. Editing and downloading are just two of the Agent's tools. Editing typically involves editing footage of interest to the user and returning it as time-lapse video. Downloading is generally used for investigation and evidence collection. The tool extracts the device and time range the user wants to access, then calls a video download tool to push the final result to the user via an HTTPS link. Video editing is just one of many tools. To perform video editing, the following toolchain is invoked (user tools, device tools, time tools; ultimately, the parameters from these three tools are extracted and used by the video editing tool).

[0024] Through the above-mentioned technical means, the present invention can significantly reduce computing and bandwidth costs, improve the timeliness of critical tasks, and provide a complete auditable and reproducible processing link in scenarios that ensure multi-tenant privacy compliance and resource constraints, thereby meeting the comprehensive requirements of modern large-scale video surveillance systems for efficiency, timeliness and compliance.

[0025] The specific task flow of the asynchronous task execution method for camera data based on a set of tool-based capabilities provided by this invention is as follows: Figure 1 As shown, it includes: 1. Initiate Request: The App initiates a conversation request, which may include images, text, audio, and video. 2. Intent Recognition and Authentication: The App parses the user's identity and performs permission checks to determine the subsequent processing path. 3. Conversation Context Retrieval: The App retrieves conversation history, user configurations, or cache to build a more complete context to aid understanding and response. 4. Retrieval and Fusion: Multimodal retrieval and comparison are performed within CamAgent to aggregate necessary information. 5. Business Decision and Strategy Application: The App determines the final response strategy based on business rules. 6. Generate Response: The App calls a model or template to generate the final content. 7. Return Response: The App returns the processing result to relevant tools in the AppCamAgent layer, such as... Figure 2 As shown, it includes: 1. Date and Time: The large model is stateless, so such a basic tool is needed. 2. System Information: Basic information about CamAgent. 3. User Information: 4. Device Information: User's bound devices. 5. Weather: Weather in a specific city or at a specific latitude and longitude. 6. People Database: Face database for face search. 7. Conversation: Conversation information generated by the user. 8. Events: Event records generated by the device. 9. Statistics: Statistical tasks. 10. Editing: Personalized editing of data generated by the camera. 11. Download: Downloaded videos can be shared with others via HTTP address. 12. Reminders: Reminders for future events. 13. Search: Searching for event information generated by the camera. 14. Other: These tools are currently pre-packaged. The agent will combine and invoke them based on the prompt words. For example... Figure 3 As shown, the specific process is as follows: 1. The IPC camera pushes the generated data to Device Manager; 2. CamAgent subscribes to Device Manager data; 3. Data import.

[0026] The agent platform does not interact directly with the devices. The camera pushes the generated Alog data to the platform (implemented here using Emqx), and the camera is only responsible for pushing this data. The agent platform subscribes to this data from Emqx and performs operations such as data storage and cleaning.

[0027] like Figure 4 and Figure 5 As shown, the specific process is as follows: 1. Question from an App user: 2. CamAgent Intent Analysis: Extracts the target to be edited, as well as the time range, etc.

[0028] 3. Synchronous return: To prevent users from waiting, the task creation result needs to be communicated. 4. Asynchronous execution: Performing specific editing operations. 5. Asynchronous callback: Push the editing results to the App.

[0029] The above example uses a user wanting to edit a video. The process involves user inquiry -> large-scale model intent recognition (analysis) [the video editing tool needs to provide a time range, device, and the target to be edited]. Since editing is a time-consuming operation, the agent will directly inform the user that the editing task has been created. Once the editing is complete, the agent platform will proactively push the notification to the app.

[0030] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details.

Claims

1. A camera data asynchronous task execution system based on a set of tool-based capabilities, characterized in that, include: The data processing module is used to perform structured processing on camera data to obtain underlying logs and store them in the underlying log database; The tool encapsulation module encapsulates system capabilities into several tools and stores them in the tool database. The intent recognition and authentication module is used to receive user requests and perform user request parsing and permission verification to obtain the user's task intent. The session context retrieval module is used to retrieve session history, user configuration, or cache based on the user request to build context-assisted understanding and response; the retrieval and fusion module is used to perform multimodal retrieval and comparison in the tool database based on the user intent, and aggregate necessary information to generate a toolchain; the business decision and strategy application module adaptively adjusts the batch processing window based on task SLA, system resources, and network status, and determines the final response strategy based on the toolchain and business rules based on intermediate product reuse and merging strategies. The response generation module is used to generate an asynchronous execution plan based on the final response strategy and to call a model or template from the underlying log database to generate the final content. The response module is returned, which is used to display the final content; The evidence recording module is used to record immutable metadata and store it as a chain of evidence when tools are invoked and intermediate products are generated / referenced.

2. The asynchronous task execution system for camera data based on a set of tool-based capabilities as described in claim 1, characterized in that, The tools include date and time, system information, user information, device information, weather, people database, conversations, events, statistics, clipping, downloading, reminders, searching, and others.

3. The asynchronous task execution system for camera data based on a set of tool-based capabilities as described in claim 2, characterized in that, The metadata includes input / output hashes, tool and model versions, execution nodes, timestamps, and signatures.

4. The asynchronous task execution system for camera data based on a set of tool-based capabilities as described in claim 3, characterized in that, The metadata is stored in a chained hash or trusted evidence storage manner.

5. A method for asynchronous task execution of camera data based on a set of tool capabilities, characterized in that, Includes the following steps: The camera data is structured to obtain the underlying logs and stored in the underlying log database; The system capabilities are encapsulated into several tools and stored in a tool database; user requests are received and parsed to obtain the user's task intent through permission verification; based on the user request, session history, user configuration, or cache are retrieved to construct context-aided understanding and response; based on the user intent, multimodal retrieval and comparison are performed in the tool database to aggregate necessary information and generate a toolchain; the batch processing window is adaptively adjusted based on task SLA, system resources, and network status; based on intermediate product reuse and merging strategies, the final response strategy is determined according to the toolchain and business rules; an asynchronous execution plan is generated based on the final response strategy, and the final content is generated and displayed by calling models or templates from the underlying log database. When tools are invoked and intermediate products are generated / referenced, immutable metadata is recorded and stored as a chain of evidence.

Images