A lightweight processing method and system for cloud-based network surveillance video based on U-ACE

Abstract

This invention discloses a lightweight processing method and system for cloud-based network surveillance videos based on U-ACE. The core of this method lies in building and running the multi-task, multi-modal AI core engine U-ACE. The processing flow includes: real-time perception and priority determination of the input video using U-ACE's Perception and Priority Decision Module (PPM); analysis, prediction, and enhancement of high-priority content using U-ACE's Deep Analysis and Prediction Enhancement Module (APEM); adaptive compression of non-high-priority or non-critical areas using U-ACE's Intelligent Compression Module (ICM); enhanced display and layered rendering using U-ACE's User Preference and Intelligent Presentation Module (UPSM) in conjunction with user preferences; and dynamic adjustment of the engine's operating parameters and strategies through U-ACE's Engine Self-Optimization Module (ESOM) in real-time feedback. This invention improves processing efficiency, reduces resource consumption, and optimizes the presentation of critical content and user experience.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a lightweight processing method for cloud-based network surveillance videos based on U-ACE. Background Technology

[0002] In the field of modern video surveillance and processing, cloud-based networked monitoring systems generate massive amounts of video data, and processing this data in real-time, efficiently, and intelligently is a major challenge. Traditional video processing methods, as well as early AI-based methods, typically employ multiple separate AI models or modules for specific sub-tasks (such as scene perception, object detection, video encoding, and user interaction analysis). This separate architecture has several inherent limitations: First, each separate model is usually designed and trained independently, and their information interaction is often unidirectional and coarse-grained, lacking deep feature sharing and fine-grained real-time collaboration mechanisms. This leads to information bottlenecks and redundant computations in the overall processing flow, making it difficult to achieve smooth end-to-end optimization. Second, due to model separation, it is difficult to perform unified and dynamic optimization of computing and storage resources from a system-wide perspective. Each model may pursue optimal performance for its own task, excessively consuming resources, resulting in low overall resource utilization, especially in complex and ever-changing monitoring scenarios, where it cannot adapt flexibly. Third, the separate models have a weak overall response capability to scene changes. For example, the judgments made by the scene awareness module may not be promptly and effectively transmitted to the analysis and compression modules to adjust their internal strategies, leading to system performance degradation or loss of critical information during sudden events or drastic environmental changes. Finally, improving the entire system requires coordinating the upgrades of multiple independent models, resulting in high development and maintenance costs.

[0003] Therefore, there is an urgent need for a core AI technology that can overcome the limitations of the traditional separation model, achieve breakthroughs in intelligence, lightweighting, adaptability and overall performance of surveillance video processing through high integration and deep collaboration, and effectively address the technical challenges of building such a unified system. Summary of the Invention

[0004] The purpose of this invention is to provide a cloud-based network surveillance video intelligent lightweight processing method and system based on U-ACE. By constructing a highly integrated, multi-task, multi-modal AI core engine (Unified AI Core Engine, U-ACE), it achieves end-to-end intelligent processing of surveillance video, from perception, analysis, prediction, enhancement, compression to presentation and optimization. This invention not only proposes the U-ACE architecture but also offers specific solutions to the technical challenges of building such a unified engine (such as multi-task conflicts, difficulties in gradient optimization, complex information flow control, and high computational resource requirements). It also reveals the synergistic gains generated by these solutions, thereby overcoming the limitations of traditional separate models, improving processing efficiency, reducing resource consumption, enhancing the quality of key content, and optimizing user experience.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A lightweight processing method for cloud-connected surveillance videos based on U-ACE is disclosed. This method is executed through a multi-task, multi-modal AI core engine (U-ACE). U-ACE employs a deep Encoder-Decoder architecture and integrates a Perception and Priority Decision Module (PPM), a Deep Analysis and Prediction Enhancement Module (APEM), an Intelligent Compression and Lightweighting Module (ICM), a User Preference and Intelligent Presentation Module (UPSM), and an Engine Self-Optimization Module (ESOM). The method includes at least the following steps:

[0007] Real-time perception and priority decision-making step S1: The PPM of the U-ACE uses the early features extracted by the shared encoder of the U-ACE to perceive the scene type, complexity and changes of the input monitoring video in real time, and combines the dynamic resource management unit inside the U-ACE to dynamically determine the processing priority of the video content and make preliminary planning of computing resources.

[0008] High-priority content deep processing step S2: For video content determined to be high-priority in step S1, the APEM of the U-ACE utilizes the deep spatiotemporal features extracted by the shared encoder to perform refined semantic analysis, adaptive enhancement of key information, and forward-looking prediction of scene dynamic change patterns; wherein, the scene perception results output by the PPM in step S1 influence the APEM's utilization of the shared encoder features through an attention modulation mechanism; the prediction results of the APEM are internally fed back to the PPM to assist in the dynamic adjustment of its priority decision and guide the APEM's own enhancement parameters;

[0009] Non-critical content intelligent compression step S3: For content determined to be non-high priority in step S1 or non-critical areas in the video frame identified by APEM in step S2, the ICM of U-ACE uses the features extracted by the shared encoder or the output of PPM to perform deep compression using a content-adaptive neural compression algorithm. Its compression strategy is jointly modulated by the priority determined in step S1 and the analysis results of APEM in step S2. User preference-driven intelligent presentation step S4: U-ACE's UPSM learns user historical interaction data to build a user preference model, and combines the high-priority content processed in step S2 and the compressed data in step S3 to perform customized enhanced display and refined layered rendering of the content that users care about.

[0010] Engine closed-loop self-optimization step S5: The ESOM of the U-ACE, based on deep reinforcement learning, continuously monitors the overall operating status of the U-ACE, the performance indicators of each module of PPM, APEM, ICM, UPSM, and user feedback, and dynamically adjusts the decision threshold of PPM in step S1, the analysis depth and enhancement level of APEM in step S2, the compression intensity of ICM in step S3, the presentation strategy of UPSM in step S4, and the operating parameters of the shared encoder and dynamic resource management unit inside the U-ACE.

[0011] The U-ACE achieves efficient collaboration and information sharing among the modules in steps S1 to S5 by sharing encoder features, the attention modulation mechanism, and an end-to-end joint training strategy that includes a collaborative gradient regularization term, thereby overcoming the limitations of traditional separation models.

[0012] Preferably, the dynamic change pattern prediction part of APEM in step S2 adopts a multi-head prediction decoder. This decoder is based on the features of CST-Transformer encoding, predicts multiple types of future information in parallel, and ensures the coordination and rationality between different prediction heads through internal prediction consistency regularization loss to deal with the target conflict problem in multi-task learning.

[0013] Preferably, the neural compression algorithm of ICM in step S3 adopts a learned context-based autoregressive entropy model, and combines it with the semantic segmentation map or saliency map output by APEM in step S2. It allocates fewer bits to the latent variable representation of non-critical regions, thereby achieving semantically guided efficient compression. This allows the prediction module of APEM in step S2 to indirectly benefit from the image compressibility features learned by ICM, resulting in synergistic gains.

[0014] Preferably, the user preference model of UPSM in step S4 is an Explainable Graph Attention Network (EGAT).

[0015] Preferably, the action space of the deep reinforcement learning agent of ESOM in step S5 is designed as a hierarchical action space. The top-level action selects a macro strategy, while the bottom-level action fine-tunes the specific parameters of each module of U-ACE in steps S1 to S4 under the selected macro strategy, so as to improve learning efficiency and policy robustness and cope with the optimization challenges brought about by the huge parameter space of the unified engine.

[0016] Preferably, a simplified mathematical form of the collaborative gradient regularization term is L_sgr=λ_sgr*Σ_ {i≠j} (1-cos(grad(L_i),grad(L_j))), where L_i and L_j are the gradients of the loss functions of different task modules executing steps S1 to S4 within U-ACE on the shared parameters, cos(·,·) represents the cosine similarity, and λ_sgr is the regularization coefficient. This regularization term promotes the synergy of parameter updates by penalizing the significant inconsistency of gradient directions between different tasks, thereby alleviating gradient conflicts and catastrophic forgetting problems in multi-task learning.

[0017] Preferably, the dynamic resource management unit inside the U-ACE can achieve fine-grained resource adaptation within the engine based on the priority determination of PPM in step S1 and the optimization instructions of ESOM in step S5. Specifically, this includes: adjusting the effective computation depth by selecting shared encoder model variants with different preset computation depths or using dynamic network structures that support early exit; dynamically activating or deactivating different decoder branches in APEM for specific tasks; and selecting neural compression models with different preset complexity levels in ICM.

[0018] Preferably, the attention modulation mechanism in step S2 includes: the scene category embedding vector e_sc and complexity score c output by PPM in step S1 are multiplied or concatenated with the feature map F_ape_in obtained by APEM from the shared encoder channel by channel, and then processed by a small convolutional network to generate a modulated feature map F_ape_mod=AttentionModulator(F_ape_in,e_sc,c), wherein the AttentionModulator network learns to dynamically adjust the channel response or spatial attention region of F_ape_in according to the scene information, so that APEM can more effectively utilize the features most relevant to the current scene.

[0019] The present invention also discloses a lightweight cloud network surveillance video processing system based on U-ACE, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned lightweight cloud network surveillance video processing method based on U-ACE.

[0020] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described lightweight processing method for cloud-based network surveillance video based on U-ACE.

[0021] The technical solution of the present invention may include the following beneficial effects:

[0022] This invention discloses a lightweight processing method and system for cloud-based network surveillance video based on U-ACE. Through a unified U-ACE architecture, shared encoder, internal attention modulation mechanism, and joint training, it achieves unprecedented deep collaboration and information sharing among functional modules. Feature reuse avoids redundant computation; ICM achieves precise deep compression under the guidance of PPM and APEM; ESOM's global optimization ensures on-demand resource allocation, thereby achieving resource utilization efficiency and system lightweighting; U-ACE, as a whole, responds to scene changes, and the strategies of its internal modules can be quickly and consistently adjusted, effectively addressing technical challenges such as multi-task conflicts and information flow control through internal collaboration mechanisms; APEM tightly couples analysis, prediction, and enhancement, forming a positive loop and improving the analysis quality of key content; ESOM performs end-to-end adaptive optimization of the entire U-ACE based on hierarchical DRL, ensuring long-term efficient system operation; high integration may bring performance improvements such as compressed feature-assisted prediction, further enhancing the overall intelligence level of the system. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall functional architecture and main internal data flow of the unified AI core engine U-ACE in the method of this invention.

[0024] Figure 2 This is an exemplary multi-level structure block diagram of the cascaded spatiotemporal Transformer (CST-Transformer) encoder used in the U-ACE of this invention.

[0025] Figure 3 This is an exemplary network structure block diagram of the multi-head prediction decoder and conditional GAN ​​enhancement part of the APEM module in the U-ACE of the present invention.

[0026] Figure 4 This is a schematic diagram illustrating the self-optimization process achieved by the hierarchical reinforcement learning agent interacting with U-ACE in the ESOM module of the U-ACE of this invention.

[0027] Figure 5 This is a line graph comparing the performance of the method of this invention (U-ACE) with the benchmark algorithm (separation model) on different test subsets in terms of average accuracy (mAP) for key target detection.

[0028] Figure 6 This is a comparison chart showing the "synergistic effect" of the method of this invention (U-ACE) with the benchmark algorithm and the U-ACE ablation variant in terms of overall performance score. Detailed Implementation

[0029] To further understand the content of this invention, a detailed description of the invention is provided in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0030] like Figure 1 As shown in this embodiment, a lightweight processing method for cloud-connected surveillance video based on U-ACE is presented. This method is executed through a unified multi-task, multi-modal AI core engine (U-ACE). U-ACE adopts a deep encoder-decoder architecture and integrates a perception and priority decision module (PPM), a deep analysis and prediction enhancement module (APEM), an intelligent compression and lightweighting module (ICM), a user preference and intelligent presentation module (UPSM), and an engine self-optimization module (ESOM). Specifically, it may include the following steps:

[0031] Real-time perception and priority decision-making step S1: executed by the U-ACE PPM module. PPM utilizes U-ACE's shared encoder (e.g., cascaded spatiotemporal Transformer, CST-Transformer, see appendix). Figure 2The PPM extracts early features (such as the output features F_s1 and F_s2 from Stage 1 or Stage 2 of the CST-Transformer, which retain more details and are relatively fast to compute). The PPM then perceives in real time the scene type of the input surveillance video (e.g., outputting scene category labels via a lightweight MLP classification head), scene complexity (e.g., outputting a complexity score of 0-1 via an MLP regression head), and significant changes in the scene (e.g., comparing feature differences within a continuous time window or training a specialized change detection head). Based on these perception results, the PPM interacts with the Dynamic Resource Management Unit (RMU) within U-ACE to dynamically determine the processing priority of the current video content (e.g., categorizing it into high, medium, and low levels), and preliminarily plans the computing resources to be consumed by subsequent modules such as APEM and ICM (e.g., suggesting that the RMU allocate more GPU cores to high-priority content or activate more complex analysis branches in APEM).

[0032] High-priority content deep processing step S2: For video content determined as high priority by PPM in step S1, this is performed by the U-ACE APEM module. APEM utilizes the deep spatiotemporal features extracted by the U-ACE shared encoder (especially its deep output or fused features, such as F_s3 or F_shared of CST-Transformer) for refined processing:

[0033] Refined semantic analysis: Build object detection heads (such as the DETR decoder based on Transformer) and instance segmentation heads (such as the Mask2Former decoder) on top of F_shared features, and output high-precision object bounding boxes, class labels, instance masks and possible behavior recognition results.

[0034] Adaptive enhancement of key information: A Conditional Generative Adversarial Network (GAN) is employed. Its generator takes the F_shared features and scene conditions determined by the PPM in step S1 (such as low light, rain, fog, etc.) as input to generate enhanced video frames (e.g., denoising, super-resolution, motion blur removal). The discriminator then evaluates the realism and quality of the generated results.

[0035] Forward-looking prediction of dynamic scene change patterns: Multi-head prediction decoder, based on F_shared features, predicts in parallel the movement trajectory of key targets in the short term and the probability of occurrence of mid-term scene events (such as crowd gathering trends).

[0036] Collaborative Mechanism 1 - Attention Modulation: In step S1, the scene perception results output by PPM (such as the scene category embedding vector e_sc and complexity score c) influence APEM's utilization of the shared encoder feature F_ape_in through a specific attention modulation mechanism (AttentionModulator). For example, AttentionModulator can be a small neural network whose inputs are F_ape_in, e_sc, and c, and whose output is a modulated feature map F_ape_mod = AttentionModulator(F_ape_in, e_sc, c). This modulation process can learn to recalibrate the channel dimension of F_ape_in (amplify the feature channels related to the current scene) or focus on the spatial dimension region (make APEM pay more attention to specific spatial regions related to the current scene type or complexity).

[0037] Collaborative Mechanism 2 - Predictive Feedback: APEM's predictions (especially the probabilities of medium-term events and short-term trajectories) are internally fed back to the PPM module in step S1. The PPM decision network can use these predictive features as additional inputs to achieve more forward-looking priority adjustments. Simultaneously, APEM's predictions also dynamically guide its own augmentation parameters (e.g., allocating more augmentation resources to targets on the predicted trajectory in advance).

[0038] Step S3, the intelligent compression step for non-critical content: For content that is determined to be non-high priority by PPM in step S1, or non-critical areas in the video frame (such as static backgrounds or objects of non-interest) identified by APEM in step S2, the ICM module of U-ACE performs the compression. ICM utilizes features extracted by the U-ACE shared encoder (which can be shallow or mid-level features F_s1, F_s2 with low computational cost) or the direct output of PPM (such as saliency maps) and employs a content-adaptive neural compression algorithm (e.g., an autoregressive entropy model based on learnable context).

[0039] Collaborative Mechanism 3 - Compression Strategy Modulation: The compression strategy of ICM (such as target bit rate and quantization parameters) is jointly finely modulated by the priority (P) of the PPM determination in step S1 and the APEM analysis results in step S2 (such as semantic segmentation map, indicating which regions are background and which are objects of a specific category), to achieve differentiated deep compression with "one strategy per frame and one strategy per region".

[0040] User preference-driven intelligent presentation step S4: executed by U-ACE's UPSM module. UPSM learns from historical user interaction data to build a user preference model, and combines the high-priority content processed in step S2 with the compressed data in step S3 to perform customized enhanced display and refined layered rendering of user-focused content. The data integration and presentation process is as follows: In step S2, APEM analyzes and enhances high-priority video content (e.g., raw pixel data or pre-decoded data) (e.g., through Conditional GAN). The enhanced high-priority content (e.g., pixel data from several keyframes or short video clips) is temporarily stored in a high-quality buffer, or, as needed, undergoes lightweight, near-lossless intra-frame encoding (e.g., using JPEG-LS or low-parameter intra-frame HEVC / AV1 encoding) for easy management and fast decoding; In step S3, ICM performs deep neural compression on non-high-priority content or non-critical regions to generate a compressed bitstream. During rendering, this bitstream is decoded in real time to restore pixel data, but its quality is usually lower than that of high-priority content. UPSM dynamically obtains pixel data for the corresponding regions based on the user's current focus area (predicted through a user preference model or determined through real-time interaction) and the content priority of each region, from the high-quality buffer (for high-priority regions of user focus) and the output of the ICM decoder (for other regions). Subsequently, UPSM uses graphics rendering interfaces (such as OpenGL, Vulkan, DirectX) to synthesize these image regions from different sources and of different qualities into the final display frame in real time. For example, the target of user focus and its surrounding area use high-quality data enhanced by APEM, while the background in the distance uses data compressed and decoded by ICM. UPSM handles the smooth transition between regions of different quality (e.g., through alpha blending or edge feathering) to improve the visual effect. Through this mechanism, UPSM can significantly reduce the overall data volume while ensuring high-quality rendering of key content, achieving efficient intelligent rendering.

[0041] UPSM first analyzes long-term historical user interaction data (viewing history, zooming, panning, tagging, etc.) and possible explicit feedback through its internal user preference learning submodule (e.g., using the interpretability-enhanced graph attention network EGAT) to build a dynamically updated, interpretable user preference model. Then, UPSM combines this preference model with high-quality, high-priority content processed by APEM in step S2 and lightweight data compressed by ICM in step S3 for intelligent presentation.

[0042] Customized Enhanced Display: Clearly overlay the analysis results of S2 step APEM (such as detection boxes, trajectories, and event labels) onto the video, and highlight specific targets or event trajectories according to user preferences.

[0043] Refined layered rendering: The rendering quality is dynamically adjusted based on the user's real-time focus areas (predicted through mouse hover, eye tracking, or user preference models) and the priority of the content itself. Highly focused areas receive priority use of high-quality data enhanced by APEM in step S2; other areas use data compressed by ICM in step S3 or a lower-resolution version.

[0044] Engine closed-loop self-optimization step S5: executed by the ESOM module of U-ACE. ESOM employs deep reinforcement learning (DRL) techniques (e.g., hierarchical DRL) as a meta-controller to adaptively optimize the entire U-ACE. The ESOM's DRL agent first undergoes extensive offline training, utilizing historical monitoring data and simulated environments to explore system performance under various parameter configurations, learning a robust initial policy. After actual system deployment, ESOM primarily performs online fine-tuning. This online tuning is either periodic (e.g., evaluation and adjustment decisions every few minutes or hours) or event-driven (e.g., triggered when a sustained performance degradation or significant change in user feedback is detected), rather than adjusting all parameters in real-time frame-by-frame, to balance the timeliness of optimization with system stability. The ESOM's policy network (especially in the top-level macro-policy selection part of the hierarchical DRL) is designed as a lightweight network structure, ensuring it can output decisions quickly (e.g., within milliseconds) upon receiving the current system state. To avoid system performance fluctuations caused by overly frequent or drastic parameter adjustments, the output of the ESOM is processed by a smoothing module (e.g., using exponential moving averages to update target parameters, or setting maximum step sizes and frequency limits for parameter adjustments). Furthermore, the Dynamic Resource Management Unit (RMU) considers the system's current load and task queue when executing ESOM instructions to avoid instantaneous overload. The RMU within the U-ACE then performs fine-grained adaptive resource management based on the instructions output by the ESOM. For example, if the instruction requires reducing the computational load on the shared encoder, the RMU will switch to a pre-trained CST-Transformer model variant with a shallower number of layers, or, for CST-Transformer architectures supporting early exit mechanisms, activate an earlier exit point to obtain the required features, thereby reducing computational load. Similarly, the RMU can dynamically activate or deactivate decoder branches in APEM for specific semantic analysis tasks based on instructions (e.g., retaining only basic object detection and disabling complex behavior analysis branches in low-priority scenarios), and select a predefined neural compression model for ICM that matches the current resource budget and content priority (e.g., switching from a high-bitrate, high-quality model to a low-bitrate, high-compression model). The RMU is also responsible for submitting resource request prompts (such as GPU utilization targets and task priorities) to the underlying operating system or computing platform, and optimizing task scheduling and batch size within U-ACE to indirectly influence and efficiently utilize allocated computing resources.ESOM continuously monitors the overall operational status of U-ACE (such as end-to-end latency, throughput, GPU / CPU utilization), the perception accuracy and decision rationality of the PPM module in step S1, the analysis accuracy (mAP), prediction error, and enhancement quality (MOS sub-simulation) of the APEM module in step S2, the compression ratio and reconstruction quality (PSNR / SSIM) of the ICM module in step S3, the user satisfaction simulation score of the effect presented by the UPSM module in step S4, and real-time feedback from the user end (such as the number of video stutters and explicit scores).

[0045] Based on this high-dimensional state information, the ESOM DRL agent dynamically adjusts the following parameters and strategies:

[0046] The scene complexity determination threshold and priority division rules for PPM in step S1.

[0047] The depth and breadth of APEM analysis tasks in step S2 (e.g., which specific analysis subtasks are activated, different complexity versions of the model are selected), and the level of key information enhancement.

[0048] The compression intensity and target bit rate range of ICM in step S3.

[0049] Details of UPSM's rendering strategy in step S4 (e.g., the number of layers in layered rendering and the quality threshold).

[0050] The computational depth (e.g., dynamically skipping certain layers or modules) or the number of active attention heads of the U-ACE internal shared encoder (such as CST-Transformer).

[0051] The resource allocation strategy of the U-ACE internal Dynamic Resource Management Unit (RMU). The goal of ESOM is to learn an optimal strategy to achieve a dynamic, globally optimal balance between processing efficiency, analysis quality, lightweightness, resource consumption, and user experience.

[0052] CST-Transformer's dynamic sparse attention mechanism, for example, can predict a Top-K subset of keys for each query, or a binary mask to indicate which keys are relevant, before calculating the attention score between each query and all keys. This allows attention computation to be performed only on this sparse subset, reducing computational cost, which is especially crucial for processing high-resolution or long video sequences and is an effective means of addressing the computational challenges of unified large-scale models.

[0053] For details on the multi-head prediction decoder and conditional GAN ​​enhancement part of the APEM module, please refer to the appendix. Figure 3The prediction consistency regularization loss in the APEM multi-head prediction decoder can be designed, for example, to encourage the temporal alignment of the endpoint of short-term trajectory predictions with the probability of intermediate events (such as "an object enters a region"). If the short-term trajectory predicts that an object will enter region A at time t, then the probability of "an object entering region A near time t" in the intermediate event prediction should also be increased accordingly. This loss can be a soft constraint based on KL divergence, cross-entropy, or certain logical rules.

[0054] The ICM employs a learnable contextual autoregressive entropy model, whose context can include: (1) spatially encoded neighboring latent variables; (2) temporally encoded latent variables at the corresponding positions in the previous frame; and (3) side information extracted from CST-Transformer shared features that is relevant to the current encoded block, such as texture complexity, motion vectors (if available), and semantic category labels output by APEM. This rich contextual information enables the entropy model to more accurately predict the probability distribution of the current latent variables, thereby achieving higher compression efficiency. APEM's prediction module may indirectly learn certain statistical characteristics of the scene from these rich contexts (especially semantic category and motion information), and even if these characteristics were initially intended for compression, they may still have a positive impact on the prediction task, forming a synergistic gain.

[0055] The ESOM module uses a hierarchical reinforcement learning agent to interact with U-ACE for self-optimization; see the appendix for details. Figure 4The hierarchical DRL of ESOM outputs a discrete macro-level pattern label M (such as "efficiency priority", "quality priority", "balance") from its high-level policy network. The low-level policy network is a parameterized function π_low(a_low|s_sub,M), which, given the current fine-grained state s_sub and the pattern M selected by the high-level network, outputs specific low-level actions a_low (i.e., parameter adjustments for each module of U-ACE). This hierarchical structure decomposes the complex optimization problem, with the high-level network responsible for strategic decision-making and the low-level network responsible for tactical execution, thus improving the stability and efficiency of learning. The RMU, based on the instructions in a_low output by ESOM (e.g., if the instruction requires reducing the computational depth of the CST-Transformer), modifies the computation graph, skipping some intermediate layer computations. Specifically, in the implementation of the collaborative gradient regularization term, the gradient calculation grad_shared(L_k) refers to the gradient vector of the task loss L_k with respect to all network parameters (mainly the parameters of the CST-Transformer) that participate in the task computation and belong to the shared part. Calculating the cosine similarity of gradients between all pairs of tasks and summing them in weight yields a scalar that measures the overall degree of gradient conflict. Adding this scalar to the total loss and minimizing it encourages cooperation.

[0056] The present invention also discloses a lightweight cloud network surveillance video processing system based on U-ACE, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned lightweight cloud network surveillance video processing method based on U-ACE.

[0057] The present invention also discloses a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the above-described lightweight processing method for cloud-based network surveillance video based on U-ACE.

[0058] The simulation environment and test data are as follows:

[0059] Simulation hardware platform: CPU: Intel Xeon Gold 6248R @ 3.00GHz (24 Cores) or equivalent multi-core processor

[0060] GPU: High-performance computing cards such as NVIDIA Tesla V100 (32GB HBM2), NVIDIA A100 (40 / 80GB HBM2e), or NVIDIA GeForce RTX 4090 (24GB GDDR6X).

[0061] Memory: 256GB DDR4 / DDR5 RAM or higher

[0062] Storage: 10TB NVMe SSD or high-speed storage array

[0063] Simulation software environment:

[0064] OS: Ubuntu 20.04 LTS / 22.04 LTS

[0065] Deep learning frameworks: PyTorch 1.12+ (2.x recommended) or TensorFlow 2.10+

[0066] CUDA Version: 11.6+

[0067] Video codec library: FFmpeg 5.0+, with optional dedicated neural video codec library.

[0068] Programming language: Python 3.9+

[0069] Other libraries: OpenCV, NumPy, Pandas, Matplotlib / Seaborn (for result visualization), RLlib / Stable Baselines3 (for DRL implementation).

[0070] Test dataset:

[0071] Public datasets:

[0072] UA-DETRAC: Includes various complex traffic scenarios for vehicle detection, tracking, and scenario complexity analysis.

[0073] MOTChallenge (MOT17, MOT20, MOT23): Contains crowd scenes under different lighting, density, and occlusion conditions, used for multi-target tracking and scene dynamics analysis.

[0074] VisDrone2019 / UAVDT: Videos taken from the perspective of a drone, characterized by small targets, large changes in perspective, and complex backgrounds, are used to test the model's perception and robustness towards small targets.

[0075] ActivityNet / Kinetics-700: A large-scale action recognition dataset that can be used for pre-training of spatiotemporal feature extractors in PPM and APEM or for evaluation of certain tasks.

[0076] DAVIS / YouTube-VOS: Video object segmentation dataset that can be used to evaluate the performance of the PPM preliminary saliency or APEM segmentation branch.

[0077] Self-built / Industry-specific datasets: Video data collected and labeled for specific monitoring application scenarios (such as smart city intersections, industrial park perimeters, shopping mall interiors, specific events such as abandoned object detection, abnormal crowd behavior, etc.). The data should cover different weather conditions, lighting, and time periods, including various normal activities and simulated abnormal events. The total dataset size is recommended to be several hundred hours or more, with fine-grained annotations, including scene type, complexity level, key object bounding boxes and IDs, trajectories, event types and spatiotemporal ranges, and user-focused areas (obtained through simulated user behavior).

[0078] Input video specifications: The performance comparison experiments are mainly based on a scenario with an input video resolution of 1080p (1920×1080)@25 / 30fps. For core computing modules such as CST-Transformer, their computational load is significantly related to the input resolution. The U-ACE proposed in this invention aims to seek a balance between performance and efficiency under different input resolutions through dynamic sparse attention, dynamic adjustment of the encoder's effective computational depth, and priority decision-making based on PPM.

[0079] For performance comparison, a benchmark algorithm was selected to verify the effectiveness of the method of this invention. The benchmark algorithm is a typical enhanced AI monitoring system that uses fixed high-quality H.264 / AVC or H.265 / HEVC encoding (e.g., fixed bitrate or fixed quality mode) for all input video streams, and then uniformly uses advanced object detection models such as YOLOv5 / YOLOv8 for real-time analysis. This benchmark algorithm does not include scene-adaptive dynamic resource allocation, differentiated processing flow, deep neural compression, user preference learning, or system-level closed-loop feedback optimization mechanisms.

[0080] The performance indicators and comparison results are shown in the following figure, presented in line chart format:

[0081] Indicators / Conditions Baseline Algorithm Method of the present invention mAP@UA-DETRAC (Sunny Day) 0.72 0.76 mAP@UA-DETRAC (Night) 0.65 0.70 mAP@MOT17 (Indoor) 0.68 0.72 mAP@MOT17 (Complex Outdoor) 0.60 0.65 Average video loading time (ms) 1200ms 800ms (average) User satisfaction rating (1-5) 3.2 4.0 (average)

[0082] Results Analysis (in conjunction with appendix) Figure 5 , Figure 6 (To be elaborated in detail):

[0083] Analysis accuracy and intelligence level (refer to) Figure 5 ): Figure 5The line graphs illustrate the comparison of the mean average accuracy (mAP) for key object detection between the proposed method (for high-priority content) and benchmark algorithms on several challenging test subsets (e.g., the clear day and night subsets of UA-DETRAC, and the indoor and outdoor complex subsets of MOT17). Although the proposed system prioritizes lightweight and high efficiency, by prioritizing computational resources for high-value scenarios through PPM and performing in-depth, refined analysis (including feature enhancement and context awareness) by APEM, the accuracy of core AI tasks (such as object detection) in these critical scenarios is actually improved or at least no lower than that of benchmark algorithms. Specifically, for high-priority areas in the scene identified by PPM and deeply processed by APEM (e.g., areas containing key moving targets or specific events), the key object detection accuracy (mAP) is effectively guaranteed or even improved. For example, on the UA-DETRAC night subset, mAP improves from 0.65 to 0.70, and on the MOT17 outdoor complex subset, it improves from 0.60 to 0.65. This demonstrates the intelligent strategy of the method in this invention, and its ability to improve analysis quality through complex model design and scenario integration. This improvement is mainly attributed to the following synergistic effects: U-ACE concentrates computational resources on APEM for refined semantic analysis of high-priority regions and adaptive enhancement of key information (such as Conditional GAN, which performs denoising, super-resolution, or contrast enhancement on high-priority content in the APEM processing flow), effectively improving the visibility and recognizability of targets in these regions. PPM ensures that regions containing key information are prioritized for high-quality processing and enhancement by APEM, and this high-quality data is then preserved or encoded with high quality in the final presentation. ICM, on the other hand, is responsible for processing non-high-priority content determined by PPM or non-critical regions identified by APEM, achieving significant compression. Here, the mAP improvement refers to the evaluation result of high-priority content processed by the APEM module, not the evaluation of non-critical regions after extreme compression by ICM. For non-critical regions, their information fidelity will decrease due to deep compression, but the goal is to maximize compression efficiency.

[0084] User Experience: As shown in the performance comparison table, the method of this invention significantly reduces the average video loading time (e.g., from 1200ms to 800ms) due to the reduction in overall bitrate, layered rendering, and user preference-driven presentation. Simultaneously, the overall user satisfaction score obtained through user surveys or by collecting implicit user feedback (such as stuttering rate and task completion efficiency) has also improved (e.g., from 3.2 to 4.0 out of 5). This indicates that the invention has also achieved good results in optimizing the user interaction experience. User satisfaction is comprehensively evaluated through the following methods:

[0085] Objective performance metrics: Objective performance data of key interactions were recorded and compared, including average video loading time (the time from when the user initiates viewing to when the first frame appears; the average for the method of this invention is 800ms, while the baseline algorithm is 1200ms), alarm latency when high-priority events occur, and system response smoothness when simulating user target tracking and playback operations (for example, by statistically analyzing the number of stutters or frame drops per unit time, the method of this invention is significantly lower than the baseline algorithm).

[0086] Subjective Evaluation: A simulated user study was conducted, recruiting 100 participants with experience using surveillance videos. Participants were shown video clips processed using the method of this invention and video clips processed using a benchmark algorithm (the clip content, scene complexity, and event types were matched and balanced). They were then asked to complete a series of typical surveillance tasks (such as locating specific targets, identifying abnormal behavior, and evaluating video clarity). After completing the tasks, participants filled out a questionnaire based on a five-point Likert scale, evaluating the two methods from multiple dimensions, including video clarity, loading speed, interaction smoothness, and efficiency in obtaining key information. Based on the combined scores, the average user satisfaction score for the method of this invention was 4.0 (out of 5), while the benchmark algorithm scored 3.2.

[0087] Layered rendering latency considerations: UPSM's layered rendering mechanism is designed with potential rendering latency in mind. Through rapid prediction of user-focused areas (e.g., based on historical preferences or lightweight real-time interaction analysis) and asynchronous loading and efficient caching of data streams of different quality, it ensures that the loading and rendering latency of high-quality content is controlled within [e.g., less than 150ms] when user focus shifts, minimizing the impact on user experience. In the aforementioned subjective evaluation, the smoothness of rendering transitions is also one of the evaluation dimensions. This invention optimizes the user experience of interacting with surveillance video by combining UPSM's user preference-driven presentation and layered rendering with improved overall processing efficiency.

[0088] Furthermore, to verify the contribution of each core module and collaborative mechanism within the U-ACE of this invention to the overall performance, ablation experiments were conducted, and the results are as follows: Figure 6 As shown. Figure 6 The invention demonstrates a comparison of the method of the present invention (complete U-ACE) with the benchmark algorithm (separate model) and several U-ACE ablation variants (e.g., U-ACE without APEM feedback PPM, U-ACE without shared encoder, and U-ACE without ESOM optimization) on the overall performance rating.

[0089] from Figure 6It is clear that the complete U-ACE method achieved the highest overall performance score, significantly outperforming the benchmark algorithm and all ablation variants. This intuitively demonstrates the "synergistic effect" generated by the unified engine architecture and its internal collaborative mechanism proposed in this invention. The specific synergistic effect and the resulting performance improvements include at least the following:

[0090] 1. Comparing the performance of "complete U-ACE" and "U-ACE (without internal feedback)" reveals that when APEM's prediction results are fed back to PPM to assist in dynamic priority adjustment, PPM can make more forward-looking and accurate decisions, thus more effectively allocating computing resources to content that truly requires in-depth processing. Simultaneously, the scene-aware results output by PPM in step S1 influence APEM's utilization of shared encoder features through an attention modulation mechanism, making APEM's analysis more targeted. This bidirectional collaboration between PPM and APEM jointly improves the effectiveness of high-priority content processing and the capture rate of key information, significantly contributing to the overall performance score.

[0091] 2. A comparison of the performance of “full U-ACE” and “U-ACE (without shared encoder)” shows that the entire U-ACE architecture is based on shared encoder features and an end-to-end joint training strategy that includes a collaborative gradient regularization term. This fundamentally promotes information sharing among modules, reduces redundant computation, and alleviates conflicts in multi-task learning, providing a solid foundation for synergistic effects between specific modules.

[0092] 3. A performance comparison between "complete U-ACE" and "U-ACE (without ESOM optimization)" demonstrates that the ESOM module, through continuous monitoring based on deep reinforcement learning and dynamic adjustment of the operating parameters of each U-ACE module (PPM decision threshold, APEM analysis depth, ICM compression strength, UPSM presentation strategy) and core components (shared encoder, dynamic resource management unit), enables U-ACE to adapt as a whole to different input video characteristics, operating environments, and user needs, achieving a dynamic and optimal balance between processing efficiency, resource consumption, analysis quality, and user experience. This global closed-loop self-optimization capability is not possessed by separate models or other ensemble models lacking such mechanisms, and is crucial for improving the long-term comprehensive performance of the system in complex and ever-changing scenarios.

[0093] In summary, this invention, through the collaborative work of highly integrated and functionally complex AI models and in-depth optimization for monitoring application scenarios, achieves significant improvements in multiple dimensions of monitoring video processing, including efficiency, lightweight design, intelligent analysis accuracy, and user experience, and has important practical application value.

[0094] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A lightweight processing method for cloud-based network surveillance video based on U-ACE, characterized in that, This method is executed through the multi-task, multi-modal AI core engine U-ACE. U-ACE adopts a deep Encoder-Decoder architecture and integrates a perception and priority decision module PPM, a deep analysis and prediction enhancement module APEM, an intelligent compression and lightweighting module ICM, a user preference and intelligent presentation module UPSM, and an engine self-optimization module ESOM. The method includes at least the following steps: The PPM of the U-ACE uses the early features extracted by the shared encoder of the U-ACE to perceive the scene type, complexity and changes of the input surveillance video in real time, and combines the dynamic resource management unit inside the U-ACE to dynamically determine the processing priority of the video content and make preliminary plans for computing resources. For video content determined to be of high priority, the APEM of the U-ACE utilizes the deep spatiotemporal features extracted by the shared encoder to perform refined semantic analysis, adaptive enhancement of key information, and forward-looking prediction of dynamic scene change patterns. The scene perception results output by the PPM influence the APEM's utilization of the shared encoder features through an attention modulation mechanism. The prediction results of the APEM are internally fed back to the PPM to assist in the dynamic adjustment of its priority decision and guide the enhancement parameters of the APEM. For content deemed to be of low priority or non-critical areas in video frames identified by APEM, the U-ACE's ICM utilizes features extracted by the shared encoder or the output of the PPM to perform deep compression using a content-adaptive neural compression algorithm. The compression strategy is modulated by both the determined priority and the analysis results of APEM. The U-ACE learns user history interaction data through UPSM to build a user preference model, and combines the processed high-priority content and compressed data to perform customized enhanced display and refined layered rendering of the content that users care about. The ESOM of the U-ACE, based on deep reinforcement learning, continuously monitors the overall operating status of the U-ACE, the performance indicators of each module (PPM, APEM, ICM, UPSM), and user feedback, and dynamically adjusts the decision threshold of PPM, the analysis depth and enhancement level of APEM, the compression intensity of ICM, the presentation strategy of UPSM, and the operating parameters of the shared encoder and dynamic resource management unit within the U-ACE. The U-ACE achieves efficient collaboration and information sharing among its modules through shared encoder features, the attention modulation mechanism, and an end-to-end joint training strategy including a collaborative gradient regularization term.

2. The method according to claim 1, characterized in that, The dynamic change pattern prediction part of the APEM adopts a multi-head prediction decoder. This decoder is based on the features of CST-Transformer encoding, predicts multiple types of future information in parallel, and ensures the coordination and rationality between different prediction heads through internal prediction consistency regularization loss.

3. The method according to claim 1, characterized in that, The neural compression algorithm of ICM adopts an autoregressive entropy model based on learnable context and combines it with the semantic segmentation map or saliency map output by APEM. It allocates fewer bits to the latent variable representation of non-critical regions to achieve semantically guided efficient compression. This allows the prediction module of APEM to indirectly benefit from the image compressibility features learned by ICM, resulting in synergistic gains.

4. The method according to claim 1, characterized in that, The user preference model of UPSM is an interpretability-enhanced graph attention network.

5. The method according to claim 1, wherein The action space of the deep reinforcement learning agent in the ESOM is designed as a hierarchical structure. The top-level actions select macro-policies, while the bottom-level actions fine-tune the specific parameters of each module of U-ACE under the selected macro-policy.

6. The method according to claim 1, characterized in that, The collaborative gradient regularization term is: L_sgr=λ_sgr*Σ_ {i≠j} (1-cos(grad(L_i),grad(L_j))), Where L_i and L_j are the gradients of the loss functions of different task modules within U-ACE on the shared parameters, cos(·,·) represents the cosine similarity, and λ_sgr is the regularization coefficient. This regularization term promotes the synergy of parameter updates by penalizing the significant inconsistency of gradient directions of different tasks.

7. The method according to claim 6, characterized in that, The dynamic resource management unit inside the U-ACE can achieve fine-grained resource adaptation within the engine based on the priority determination of PPM and the optimization instructions of ESOM. Specifically, it includes: adjusting the effective computation depth by selecting shared encoder model variants with different preset computation depths or using dynamic network structures that support early exit; dynamically activating or deactivating different decoder branches in APEM for specific tasks; and selecting neural compression models with different preset complexity levels in ICM.

8. The method according to claim 1, characterized in that, The attention modulation mechanism includes: the scene category embedding vector e_sc and complexity score c output by PPM are multiplied or concatenated with the feature map F_ape_in obtained by APEM from the shared encoder channel by channel, and then processed by a small convolutional network to generate a modulated feature map F_ape_mod=AttentionModulator(F_ape_in,e_sc,c). The AttentionModulator network learns to dynamically adjust the channel response or spatial attention region of F_ape_in according to the scene information, so that APEM can more effectively utilize the features most relevant to the current scene.

9. A lightweight cloud-based network surveillance video processing system based on U-ACE, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the lightweight processing method for cloud-based network surveillance video based on any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the lightweight processing method for cloud-based network surveillance video based on any one of claims 1 to 8.

Images