An edge video perceptual analysis method and system based on codec metadata

Abstract

The application relates to the fields of edge computing and video analysis, and discloses an edge video perception analysis method and system based on codec metadata, specifically, a terminal-side device extracts physical features, target scale, texture entropy and motion intensity, from a video, and pre-prunes a search space of resolution, code rate and frame rate; an edge device constructs an explicit analysis mapping model, performs fine-grained dynamic precision prediction, and solves an online optimization problem to lock an optimal configuration; the terminal-side device executes the optimal configuration, a video stream is uploaded to the edge device for inference tracking, and inference results and update states are fed back to the terminal-side device, thereby forming a millisecond-level adaptive closed-loop control; the application perceives video content features in an approximate zero-cost manner, and establishes a deterministic physical constraint between the features and the configuration, so that resource scheduling can be quickly and accurately guided.

Description

Technical Field

[0001] This invention relates to the fields of edge computing and video analytics, specifically to an edge video perception and analysis method and system based on encoding and decoding metadata. Background Technology

[0002] With the rapid development of smart cities, autonomous driving, and the Industrial Internet, billions of cameras have been deployed globally, generating massive amounts of video data. These applications not only require extremely high transmission bandwidth but also rely on low-latency, high-precision analysis of real-time video streams. Mobile edge computing (MEC), by offloading computing tasks to the network edge and avoiding long-distance transmission, has become a key paradigm for solving bandwidth bottlenecks. However, edge nodes face strict limitations in computing power, storage, and thermal power consumption design, creating a fundamental contradiction with the ever-increasing computational demands of modern deep neural networks (DNNs). In a typical edge-assisted video analytics system, the configuration parameters of the video stream (such as resolution, frame rate, and bitrate) directly determine the consumption levels of system resources such as computing, storage, and network bandwidth, and also the detection accuracy of the final service. How to select the optimal configuration for highly dynamically changing video content on resource-constrained heterogeneous edge devices to maintain high-precision video inference is a pressing challenge.

[0003] Currently, there are several main technical solutions for resource scheduling and configuration optimization in edge video analytics, but all of them have significant limitations:

[0004] The first category is the "content-agnostic" network adaptive method. This type of method primarily focuses on network bandwidth fluctuations, treating the video stream as a "black box" of data packets, and dynamically adjusting parameters such as resolution and frame rate solely based on the currently available network bandwidth. For example, some early adaptive streaming media transmission technologies and some edge offloading strategies (such as Chameleon) employ this approach. The drawback of this technique is that this fixed modeling strategy ignores the spatiotemporal variations of the video content itself. Degrading the configuration solely based on network conditions leads to a sharp decrease in inference accuracy, failing to meet business requirements.

[0005] The second category is content adaptation methods based on CNN perception. To address the aforementioned content insensitivity issue, recent research has proposed using additional lightweight convolutional neural networks (CNNs) to extract semantic features from video frames in real time, and then adjusting configurations or selecting offloading strategies accordingly. The drawback of this technique is that introducing an additional CNN network for feature extraction itself introduces significant computational overhead and inference latency. On edge devices where computing power is already limited, this approach of "consuming more resources to save resources" is often counterproductive, leading to increased end-to-end processing latency and making it difficult to meet real-time requirements.

[0006] The third category comprises online scheduling methods based on "black-box optimization" or deep reinforcement learning (DRL). These methods typically model resource scheduling as a Markov decision process (MDP), utilizing reinforcement learning agents to learn optimal policies through interactions with the environment (e.g., scheduling algorithms based on Lyapunov or DRL proposed by Sheng et al. and Fan et al.). The drawbacks of this technique are: 1. Search space explosion and slow convergence: The combination space of video configuration parameters (resolution × bitrate × frame rate × model selection) is enormous, leading to low algorithm search efficiency, slow convergence speed, and difficulty in coping with rapid environmental changes. 2. Lack of interpretability: Most DRL methods are "black-box" models, failing to reveal the physical constraints between video features and configuration parameters. This results in poor generalization ability when facing unseen scenes and an inability to explain why a specific configuration is chosen, easily leading to invalid decisions that violate physical laws.

[0007] In summary, existing edge video analytics technologies generally suffer from problems such as a lack of physical interpretability of models, difficulty in perceiving complexities, and challenges in solving multi-dimensional resource joint optimization problems. Therefore, there is an urgent need for a new method that can perceive video content features in a near-zero-cost manner and establish deterministic physical constraints between features and configurations, thereby quickly and accurately guiding resource scheduling. Summary of the Invention

[0008] To address the technical problems of existing technologies, such as coarse accuracy-energy consumption models, additional computing power consumption introduced by the perception module, and the difficulty in solving multi-parameter combination optimization problems, this invention provides an edge video perception analysis method and system based on encoding and decoding metadata. The technical solution is as follows:

[0009] An edge video perception analysis method based on encoding and decoding metadata includes the following steps:

[0010] Step 1: In the end-side device, based on the codec metadata or the intermediate results of the tracking module, decouple and extract physical features from the video stream. The physical features include target scale, texture entropy, and motion intensity.

[0011] Step 2: Based on the physical characteristics, construct a deterministic physical constraint model, pre-prune the search space of video configuration parameters, remove invalid configurations, and generate a set of physically feasible configurations. The video configuration parameters include resolution, bitrate, and frame rate.

[0012] Step 3: Based on the physical features and the set of physically feasible configurations, construct an explicit analytical mapping model from video features to inference accuracy, and predict the comprehensive inference utility under different configuration combinations based on the current video features;

[0013] Step 4: Based on the set of physically feasible configurations and comprehensive reasoning utility, perform online optimization to solve the resource scheduling problem of the multi-user edge system and lock the optimal configuration for the current time slot;

[0014] Step 5: Execute the optimal configuration. The video stream is uploaded to the edge device for high-precision inference and tracking. The inference results and the updated network load status are then fed back to the edge device, forming millisecond-level adaptive closed-loop control.

[0015] Further, in step 1, the target scale is calculated as follows: based on the target bounding box set predicted by the tracking module for the next frame, the ratio of the sum of the pixel areas of all regions of interest to the total resolution of the current image is calculated; the texture entropy is calculated as follows: for the pixel regions inside the bounding box, the gray-level distribution probability is statistically analyzed, and the mean of the Shannon entropy is calculated; the motion intensity is calculated as follows: based on the inter-frame target center point coordinates provided by the tracker, the mean of the L2 norm of the target displacement vector is calculated.

[0016] Furthermore, in step 2, the deterministic physical constraint model includes the following deterministic pruning mechanism:

[0017] Resolution scale pruning: Based on the effective receptive field theory of convolutional neural networks, a lower bound for resolution is set according to the target scale;

[0018] Bitrate configuration pruning: Based on classical rate-distortion theory and hardware coding characteristics, a lower bound on the bitrate is set according to texture complexity;

[0019] Temporal dynamic pruning: Based on the overlap requirements of target tracking, a lower bound for the frame rate is set according to the motion intensity.

[0020] Furthermore, in step 3, the explicit analytical mapping model includes:

[0021] Spatial detection accuracy model: A product model including resolution-scale coupling coefficient and bit rate-entropy coupling coefficient is established, and the Sigmoid function and exponential function are used to describe the nonlinear effects of feature completeness and quantization loss on accuracy, respectively.

[0022] Temporal tracking accuracy model: Establish an exponential decay model for normalized displacement, where the normalized displacement is the ratio of motion intensity to frame rate;

[0023] Fine-grained accuracy model: combining spatial detection accuracy and temporal tracking accuracy with overall inference accuracy.

[0024] Furthermore, in step 4, the online optimization solution adopts online cooperative scheduling based on Lyapunov optimization. The specific process is as follows: a virtual queue mechanism is introduced to dynamically track the cumulative delay default of each user, transforming the long-term average delay constraint into a queue stability problem in a single time slot; a drift penalty strategy is used to balance system cost and queue backlog, and a Lagrange dual decomposition technique is used to decompose the global bandwidth coupling problem into parallel user-side sub-problems and edge-side main problems.

[0025] Furthermore, in step 4, the bandwidth shadow price is solved online based on local feature information and feedback from edge devices.

[0026] Furthermore, in step 4, the resource scheduling problem is formalized as a constrained long-term average utility maximization problem. The core objective is to minimize the time-averaged system weighted cost while satisfying the long-term average delay QoS constraint and the instantaneous bandwidth capacity constraint.

[0027] An edge video perception analysis system based on encoding / decoding metadata, used to implement any of the methods described above, includes:

[0028] End-side devices are used to perform physical feature extraction and pruning, receive and execute optimal configurations, including codecs;

[0029] Edge devices are used to perform online optimization solutions for resource scheduling, perform inference and tracking, and feed the results back to the end-side devices. They include detection modules, tracking modules, multi-dimensional evaluation modules, and decision-making modules.

[0030] Beneficial effects

[0031] This invention constructs a feature decoupling and physical constraint model based on three elements: target scale, texture entropy, and motion intensity. This effectively enhances the model's ability to interpret the content-configuration coupling relationship in video analysis tasks. Especially in extreme scenarios with sudden appearance of small targets or high dynamic motion, it can more accurately define the physical boundaries of effective configurations, preventing accuracy collapse caused by feature vanishing or tracking drift. In addition, a deterministic pruning mechanism is adopted to eliminate a large number of invalid configurations that violate physical laws, compressing the search space of the optimization algorithm by 80%-90%, and significantly enhancing the real-time response capability of the resource scheduling algorithm at the edge. Compared with traditional scheduling methods that rely on black-box fitting or high-overhead CNN perception, this invention has stronger physical interpretability and computational efficiency, and can more accurately guide the resource allocation strategy of edge devices under bandwidth and computing power constraints. At the same time, this method achieves millisecond-level decision latency, making it particularly suitable for edge video analysis tasks with extremely high real-time requirements. Attached Figure Description

[0032] Figure 1 This is a framework diagram of an edge video perception analysis system based on encoding and decoding metadata.

[0033] Figure 2 The flowchart shows the feature perception and pruning process of the edge video perception analysis method based on encoding and decoding metadata.

[0034] Figure 3 This is a flowchart of the edge-end collaborative scheduling process for an edge video perception analysis method based on encoding and decoding metadata. Detailed Implementation

[0035] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0036] This invention proposes an edge video perception analysis method and system based on encoding and decoding metadata, primarily applicable to mobile edge computing (MEC) scenarios with limited bandwidth and computational resources, such as smart city monitoring and autonomous driving perception. Compared to existing methods that rely on content-agnostic blind configuration or expensive perception methods dependent on additional CNN networks, this method achieves near-zero-cost feature decoupling by reusing metadata from video encoding / decoding and tracking modules. It utilizes physical first principles to construct a deterministic constraint model to prune the configuration space and achieves millisecond-level online decision-making based on the Lyapunov optimization framework. This method is particularly suitable for complex video stream analysis scenarios with rapidly changing target scales and non-stationary motion states, maximizing inference accuracy while ensuring low end-to-end latency.

[0037] like Figure 1As shown, in order to achieve efficient edge-edge collaborative video analysis, this invention organically connects and interacts with edge-side feature perception, physical space pruning, and edge-edge collaborative scheduling in real time through the following closed-loop process:

[0038] The first stage is the edge feature perception stage, such as Figure 2 As shown, the system workflow begins at the edge device, using encoding / decoding metadata and intermediate results from the tracking module to decouple and extract the three main physical features of the current video stream in real time: target scale, texture entropy, and motion intensity. This is followed by a physical space pruning stage, where the physical features are immediately pruned. The system establishes constraints based on the effective receptive field, rate-distortion theory, and the principle of motion continuity, eliminating invalid configurations that could lead to feature disappearance or tracking loss, thereby significantly compressing the search space of the optimization algorithm.

[0039] Then comes the edge-to-end collaborative scheduling phase, such as... Figure 3 As shown, based on the pruned feasible region, the decision module combines local feature information with the bandwidth shadow price fed back by the edge server to perform online optimization, quickly locking in the optimal configuration for the current time slot: including resolution, frame rate, and bitrate. Finally, the optimal configuration instruction is executed, and the compressed video stream is uploaded to the edge device for high-precision inference and tracking. The inference results and updated network load status are then fed back to the edge, forming millisecond-level adaptive closed-loop control, thereby minimizing system energy consumption and latency while ensuring detection accuracy.

[0040] The following are the specific steps of an embodiment of the present invention:

[0041] 1. Zero-cost feature awareness and decoupling based on encoding / decoding metadata

[0042] To avoid introducing additional computational overhead, this embodiment does not use CNNs to extract features, but instead defines feature tuples. In the time slot For users The video stream is extracted at near-zero cost as follows:

[0043] Average target scale ratio ( ): The set of target bounding boxes predicted for the next frame using the tracking module (Kalman filtering). Calculate the sum of the pixel areas of all regions of interest (RoIs) and the sum of the pixel areas of the current image's total resolution. The ratio of )

[0044] ,

[0045] in, and For the width and height of the region of interest, The number of all regions of interest is represented by this metric, which directly reflects the minuteness of the target in the image and is used to determine whether feature vanishing will occur.

[0046] Target texture entropy ( ): Calculates the probability of grayscale distribution only for the pixel region inside the bounding box. ( ), calculate the mean of Shannon entropy:

[0047] ,

[0048] This metric characterizes the average information density of the foreground target and is directly related to the lower bound of the bit resources required for encoding.

[0049] Average target motion intensity ( (): Based on the inter-frame target center point coordinates provided by the tracker Calculate the mean L2 norm of the target displacement vector:

[0050] ,

[0051] This metric reflects the dynamic rate of change of the scene and determines the minimum frame rate required to maintain tracking continuity.

[0052] 2. Construction of a deterministic physical constraint model based on physical characteristics

[0053] Before executing the resource scheduling algorithm, this embodiment uses the aforementioned physical characteristics to establish a physical constraint model, eliminates invalid configurations, and decouples the video stream to the target scale. Texture entropy and exercise intensity Three features are used to construct deterministic pruning mechanisms in the following three dimensions to remove invalid configurations before optimization.

[0054] 1. Resolution-Scale Pruning—Preventing Feature Vanishing: Object detection performance is inherently limited by the Effective Receptive Field (ERF) of the Convolutional Neural Network (CNN). According to ERF theory, feature vanishing only occurs when the projected pixel area of ​​the target on the input image is greater than the network's minimum resolvable threshold. Only when the target features are within a certain range can they be preserved in the deep network; otherwise, the target features will disappear, leading to detection failure. Therefore, this example constructs a physical lower bound constraint on the resolution. The tracking module is used to predict the target scale of the previous frame. (Ratio of target area to total image area), resolution The following scaling relationship must be satisfied:

[0055] ,

[0056] in, These are model-specific constants determined through offline experimental analysis. This constraint defines the feasible region of resolution. All configurations that made tiny targets invisible due to low resolution were removed.

[0057] 2. Bitrate Configuration Pruning – Preventing Quantization Collapse: According to classical rate-distortion theory, bitrate is positively correlated with source complexity (entropy). In zero-latency coding mode for edge video analysis, when the texture entropy of the video frame… The allocated code rate when it exceeds a certain critical threshold When the bitrate is insufficient, the efficiency of the inter-frame prediction mechanism in the encoder drops sharply, forcing it to degenerate into high-frequency intra-frame coding. This generates a large amount of macroblock header information and residual data, a phenomenon known as prediction collapse. This causes the bitrate to no longer increase linearly with entropy, but instead exhibits a superlinear explosive trend. To avoid falling into this inefficient coding region, this example, based on extensive data analysis, uses an exponential empirical model to define the physical lower bound of the bitrate:

[0058] ,

[0059] in, , For encoder-specific parameters determined through offline experiments, this constraint defines the feasible domain of the bit rate. This ensures that the allocated bandwidth resources can support the texture complexity of the current image.

[0060] 3. Temporal Dynamic Pruning—Preventing Tracking Drift: The accuracy of multi-target tracking depends on the overlap of bounding boxes (Intersection over Union, IoU) between adjacent frames. Assuming the target moves with increasing intensity... (pixels / second) performs linear motion, with inter-frame displacement of... To prevent identity switching, the IoU must be maintained at a security threshold. Above. Based on this, this example defines the frame rate. Lower bound constraint:

[0061] ,

[0062] in, This is the average target width. This constraint defines the feasible region for the frame rate. Ultimately, the user In the time slot Effective search space It is compressed into the intersection of the three physically feasible regions mentioned above:

[0063] ,

[0064] This pruning mechanism can reduce the search space by 80%-90%, significantly accelerating the convergence of subsequent optimization algorithms.

[0065] 3: Feature-driven fine-grained dynamic accuracy prediction

[0066] Unlike traditional black-box fitting, this embodiment constructs configuration parameters. With video content features Explicit analytic mapping between them will improve inference accuracy. The model is a joint probability distribution of spatial detection quality and temporal tracking continuity.

[0067] 1. Spatial detection accuracy model:

[0068] Spatial precision is affected by both resolution and bit rate:

[0069] Resolution-scale coupling coefficient : Represents the theoretical upper bound of accuracy under uncompressed conditions. Based on the effective receptive field principle, the detectability of a target depends on its projected area on the feature map. This example uses the Sigmoid saturation function to characterize the physical process of a target evolving from feature vanishing to feature completeness:

[0070] ,

[0071] in, The effective resolution product is such that when the effective resolution product approaches 0, the accuracy approaches 0, which is consistent with the physical pruning boundary mentioned above.

[0072] Bitrate-entropy coupling coefficient Marginal coding gain characterizing bit resources. High texture entropy. This means that the image contains higher density information, resulting in a decrease in the precision gain (i.e., distortion attenuation) per unit bit rate. This example models it as the inverse function of entropy:

[0073] ,

[0074] Combining the above two points, the spatial detection accuracy model is as follows:

[0075] .

[0076] 2. Time-domain tracking accuracy model:

[0077] Time domain precision It depends on the target displacement within the sampling interval. Assuming the target moves at a uniform linear velocity between frames, the tracking success rate is modeled as a function of the normalized displacement. Exponential decay function:

[0078] ,

[0079] This model shows that, in order to maintain constant tracking stability, the frame sampling rate... It depends on the intensity of exercise Linear growth. In summary, the fine-grained accuracy model in this embodiment is defined as the combined inference accuracy, which is the combined effect of spatial detection accuracy and temporal tracking accuracy:

[0080] ,

[0081] Among them, parameters , , , , , All data were obtained through offline data fitting. This white-box model is not only computationally efficient, but it can also directly derive the optimal configuration that maximizes accuracy when the content undergoes sudden changes, without the need for retraining.

[0082] 4. Formalization of System Optimization Problems

[0083] This example further constructs a system cost model that includes energy consumption and latency, and formalizes the resource scheduling problem into a constrained long-run average utility maximization problem.

[0084] 1. Communication and Computing Cost Model

[0085] Total cost of the system Energy consumption based on reasoning and transmission energy consumption It consists of two parts:

[0086] Inference energy consumption model: depends on the selected DNN model The computational complexity is given. (Joules / frame) is the model The single-frame inference energy consumption is then expressed as:

[0087] ,

[0088] in, These are binary decision variables, indicating whether to select the model. ;

[0089] Transmission energy consumption model: related to the amount of data transmitted Proportional. Based on the characteristics of video encoding, the data volume is modeled as follows: ,in It is a bit density function of texture entropy; therefore, the transmission energy consumption is:

[0090] ,

[0091] in, It is the energy consumption coefficient for transmitting each bit of data.

[0092] System end-to-end delay Due to transmission delay and reasoning delay constitute:

[0093] Transmission delay: ,in To be assigned to users bandwidth;

[0094] Inference delay: ,in For the model The processing time per frame.

[0095] 2. Formalizing the resource scheduling problem

[0096] This invention models the resource scheduling problem of edge video analytics as a real-time online decision-making process. The core objective is to minimize the time-averaged system weighted cost, i.e., cost minus accuracy utility, while satisfying long-term average latency QoS constraints and instantaneous bandwidth capacity constraints. This represents the set of joint decision variables, including configuration parameters, model selection, and bandwidth allocation, for the optimization problem. Formalized as follows:

[0097] ,

[0098] in, It is a balance of precision utility Energy consumption cost The weighting factors. The formalization of this problem reflects the complex trade-off between multidimensional resources (computation, communication) and multidimensional objectives (accuracy, energy consumption, latency) in this invention. Due to the inclusion of discrete variables (model selection) and long-term coupling constraints (average latency), this is a typical mixed-integer nonlinear programming (MINLP) problem, which needs to be solved by subsequent Lyapunov optimization algorithms.

[0099] 5. Online Cooperative Scheduling Based on Lyapunov Optimization

[0100] like Figure 3 As shown, the online optimization algorithm flow for this example is as follows:

[0101] 1. The long-term constraint based on Lyapunov optimization is transformed into a decoupled long-term average delay constraint.

[0102] The system first assigns each user Build a virtual queue This is used to dynamically track the cumulative degree of delayed defaults in the system, and its evolution equation is defined as:

[0103] ,

[0104] in, For actual end-to-end delay, To establish a long-term average delay QoS threshold, this example introduces a quadratic Lyapunov function to characterize the system's congestion state, and defines a single-slot drift-plus-Penalty term as follows:

[0105] ,

[0106] The positive parameter V is used to adjust the tradeoff between minimizing delay and minimizing system cost. By deriving the upper bound of the drift penalty function, this example will include optimization problems with long-term constraints. Successfully transformed into a deterministic optimization problem involving only the current time slot variable. :

[0107] ,

[0108] Wherein, objective function The dynamic balance between utility maximization and queue stability is achieved; when the queue length... When the delay is large, the algorithm automatically increases the weight of the delay item to prioritize smoothing out the queue backlog.

[0109] 2. Solving based on Lagrange dual decomposition

[0110] although Temporal decoupling is achieved, but the total bandwidth capacity is limited. This still results in resource coupling between multiple users. Therefore, this embodiment introduces Lagrange multipliers. Using the shadow price of bandwidth resources to relax bandwidth constraints, a Lagrangian function is constructed:

[0111] ,

[0112] Leveraging the additivity of the objective function, the global problem is decomposed into parallel user-side local subproblems and edge-side master problems:

[0113] On the user side, for a given shadow price Each user selects the optimal configuration locally in parallel. To minimize generalized cost, in this step, the system enforces the application of the dynamic feasible region constructed based on video physical features (scale, texture entropy, motion intensity). Deterministic pruning greatly compresses the search space, allowing for a fast solution on the user side using the following formula:

[0114] ;

[0115] On the edge side, the server updates the shadow price through sub-gradient iterations based on the aggregated bandwidth demand of all users. To coordinate supply and demand balance:

[0116] ,

[0117] when Upon convergence, the system locks the optimal configuration parameters and bandwidth allocation scheme for all users in the current time slot.

[0118] The specific execution closed-loop process within a complete decision-making time slot in this example is as follows: Figure 2 , Figure 3 As shown: First, the edge device extracts the physical features of the current image and automatically defines the physically feasible configuration search range for initial pruning. Next, the edge server and the edge device engage in dynamic negotiation through interactive shadow pricing. Based on this, the edge device quickly calculates the most cost-effective configuration strategy locally, iterating continuously until the global bandwidth supply and demand reach a balance. Finally, the edge device strictly compresses and transmits the video according to the optimal parameters obtained through negotiation. The edge device completes the inference calculation, and the system dynamically updates the length of the virtual delay queue based on the actual processing latency, thus smoothly entering the cycle of the next decision slot.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An edge video perception analysis method based on encoding / decoding metadata, characterized in that, Includes the following steps: Step 1: In the end-side device, based on the codec metadata or the intermediate results of the tracking module, decouple and extract physical features from the video stream. The physical features include target scale, texture entropy, and motion intensity. Step 2: Based on the physical characteristics, construct a deterministic physical constraint model, pre-prune the search space of video configuration parameters, remove invalid configurations, and generate a set of physically feasible configurations. The video configuration parameters include resolution, bitrate, and frame rate. Step 3: Based on the physical features and the set of physically feasible configurations, construct an explicit analytical mapping model from video features to inference accuracy, and predict the comprehensive inference utility under different configuration combinations based on the current video features; Step 4: Based on the set of physically feasible configurations and comprehensive reasoning utility, perform online optimization to solve the resource scheduling problem of the multi-user edge system and lock the optimal configuration for the current time slot; Step 5: Execute the optimal configuration. The video stream is uploaded to the edge device for high-precision inference and tracking. The inference results and the updated network load status are then fed back to the edge device, forming millisecond-level adaptive closed-loop control.

2. The edge video perception analysis method based on encoding / decoding metadata as described in claim 1, characterized in that, In step 1, the target scale is calculated as follows: based on the target bounding box set predicted by the tracking module for the next frame, the ratio of the sum of pixel areas of all regions of interest to the total resolution of the current image is calculated; the texture entropy is calculated as follows: for the pixel regions inside the bounding box, the gray-level distribution probability is statistically analyzed, and the mean of the Shannon entropy is calculated. The motion intensity is calculated as follows: based on the inter-frame target center point coordinates provided by the tracker, the mean of the L2 norm of the target displacement vector is calculated.

3. The edge video perception analysis method based on encoding / decoding metadata as described in claim 1, characterized in that, In step 2, the deterministic physical constraint model includes the following deterministic pruning mechanism: Resolution scale pruning: Based on the effective receptive field theory of convolutional neural networks, a lower bound for resolution is set according to the target scale; Bitrate configuration pruning: Based on classical rate-distortion theory and hardware coding characteristics, a lower bound on the bitrate is set according to texture complexity; Temporal dynamic pruning: Based on the overlap requirements of target tracking, a lower bound for the frame rate is set according to the motion intensity.

4. The edge video perception analysis method based on encoding / decoding metadata as described in claim 1, characterized in that, In step 3, the explicit analytical mapping model includes: Spatial detection accuracy model: A product model including resolution-scale coupling coefficient and bit rate-entropy coupling coefficient is established, and the Sigmoid function and exponential function are used to describe the nonlinear effects of feature completeness and quantization loss on accuracy, respectively. Temporal tracking accuracy model: Establish an exponential decay model for normalized displacement, where the normalized displacement is the ratio of motion intensity to frame rate; Fine-grained accuracy model: combining spatial detection accuracy and temporal tracking accuracy with overall inference accuracy.

5. The edge video perception analysis method based on encoding / decoding metadata as described in claim 1, characterized in that, In step 4, the online optimization solution adopts online cooperative scheduling based on Lyapunov optimization. The specific process is as follows: a virtual queue mechanism is introduced to dynamically track the cumulative delay default of each user, and the long-term average delay constraint is transformed into a single-slot queue stability problem; a drift penalty strategy is used to balance system cost and queue backlog, and a Lagrange dual decomposition technique is used to decompose the global bandwidth coupling problem into parallel user-side sub-problems and edge-side main problems.

6. The edge video perception analysis method based on encoding / decoding metadata as described in claim 1, characterized in that, In step 4, the bandwidth shadow price is solved online based on local feature information and feedback from edge devices.

7. The edge video perception analysis method based on encoding / decoding metadata as described in claim 1, characterized in that, In step 4, the resource scheduling problem is formalized as a constrained long-term average utility maximization problem. The core objective is to minimize the time-averaged system weighted cost while satisfying the long-term average delay QoS constraint and the instantaneous bandwidth capacity constraint.

8. An edge video perception analysis system based on encoding / decoding metadata, characterized in that, For implementing the method as described in any one of claims 1 to 7, comprising: End-side devices are used to perform physical feature extraction and pruning, receive and execute optimal configurations, including codecs; Edge devices are used to perform online optimization solutions for resource scheduling, perform inference and tracking, and feed the results back to the end-side devices. They include detection modules, tracking modules, multi-dimensional evaluation modules, and decision-making modules.

Images